whylabs/whylogs-python

View on GitHub

Showing 1,724 of 1,736 total issues

DatasetProfile has 38 functions (exceeds 20 allowed). Consider refactoring.
Open

class DatasetProfile:
    """
    Statistics tracking for a dataset.

    A dataset refers to a collection of columns.
Severity: Minor
Found in src/whylogs/core/datasetprofile.py - About 5 hrs to fix

    Function estimate_segments has a Cognitive Complexity of 27 (exceeds 5 allowed). Consider refactoring.
    Open

    def estimate_segments(
        df: pyspark.sql.dataframe.DataFrame,
        target_field: str = None,
        max_segments: int = 30,
        include_columns: List[str] = [],
    Severity: Minor
    Found in java/spark/python/whyspark/preprocessing/autosegmentation.py - About 3 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Logger has 31 functions (exceeds 20 allowed). Consider refactoring.
    Open

    class Logger:
        """
        Class for logging whylogs statistics.
    
        :param session_id: The session ID value. Should be set by the Session boject
    Severity: Minor
    Found in src/whylogs/app/logger.py - About 3 hrs to fix

      Function _estimate_segments has a Cognitive Complexity of 25 (exceeds 5 allowed). Consider refactoring.
      Open

      def _estimate_segments(df: pd.DataFrame, target_field: str = None, max_segments: int = 30) -> Optional[Union[List[Dict], List[str]]]:
          """
          Estimates the most important features and values on which to segment
          data profiling using entropy-based methods.
      
      
      Severity: Minor
      Found in src/whylogs/features/autosegmentation.py - About 3 hrs to fix

      Cognitive Complexity

      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

      A method's cognitive complexity is based on a few simple rules:

      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
      • Code is considered more complex for each "break in the linear flow of the code"
      • Code is considered more complex when "flow breaking structures are nested"

      Further reading

      Function _check_and_init_table_shape_constraint has a Cognitive Complexity of 23 (exceeds 5 allowed). Consider refactoring.
      Open

          def _check_and_init_table_shape_constraint(self, reference_set):
              if self.first_field in ("columns", "total_row_number"):  # table shape constraint
      
                  if self.first_field == "columns":
                      if self.op == Op.EQ:
      Severity: Minor
      Found in src/whylogs/core/statistics/constraints.py - About 3 hrs to fix

      Cognitive Complexity

      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

      A method's cognitive complexity is based on a few simple rules:

      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
      • Code is considered more complex for each "break in the linear flow of the code"
      • Code is considered more complex when "flow breaking structures are nested"

      Further reading

      DatasetProfile has 26 methods (exceeds 20 allowed). Consider refactoring.
      Open

      @AllArgsConstructor
      public class DatasetProfile implements Serializable {
        // generated by IntelliJ
        private static final long serialVersionUID = -9221998596693275458L;
        public static final String TAG_PREFIX = "whylogs.tag.";
      Severity: Minor
      Found in java/core/src/main/java/com/whylogs/core/DatasetProfile.java - About 3 hrs to fix

        Function apply_summary_constraints has a Cognitive Complexity of 21 (exceeds 5 allowed). Consider refactoring.
        Open

            def apply_summary_constraints(self, summary_constraints: Optional[Mapping[str, SummaryConstraints]] = None):
                if summary_constraints is None:
                    summary_constraints = self.constraints.summary_constraint_map
                for k, v in summary_constraints.items():
                    if isinstance(v, list):
        Severity: Minor
        Found in src/whylogs/core/datasetprofile.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method calculateFormat has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
        Open

          private Instant calculateFormat(String firstInput) {
            val parsed = dateTimeFormatter.parse(firstInput);
            val hasYear = parsed.isSupported(ChronoField.YEAR);
            val hasMonth = parsed.isSupported(ChronoField.MONTH_OF_YEAR);
            val hasDayOfMonth = parsed.isSupported(ChronoField.DAY_OF_MONTH);

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Function to_protobuf has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
        Open

            def to_protobuf(self) -> MultiColumnValueConstraintMsg:
                value = None
                set_vals_message = None
                ref_cols = None
                dependent_single_col = None
        Severity: Minor
        Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Function add_drift_val_to_ref_profile_json has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
        Open

        def add_drift_val_to_ref_profile_json(target_profile, reference_profile, reference_profile_json):
            """
            Calculates drift value for reference profile based on profile type and inserts that data into reference profile
        
            Parameters
        Severity: Minor
        Found in src/whylogs/viz/utils/profile_viz_calculations.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Function _check_and_init_between_constraint has a Cognitive Complexity of 18 (exceeds 5 allowed). Consider refactoring.
        Open

            def _check_and_init_between_constraint(self):
                if self.op == Op.BTWN:
                    if all([v is not None for v in (self.value, self.upper_value)]) and all([v is None for v in (self.second_field, self.third_field)]):
                        # field-value summary comparison
                        if not isinstance(self.value, (int, float)) or not isinstance(self.upper_value, (int, float)):
        Severity: Minor
        Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        FrequentItemsSketch has 22 functions (exceeds 20 allowed). Consider refactoring.
        Open

        class FrequentItemsSketch:
            """
            A class to implement frequent item counting for mixed data types.
        
            Wraps `datasketches.frequent_strings_sketch` by encoding numbers as
        Severity: Minor
        Found in src/whylogs/util/dsketch.py - About 2 hrs to fix

          Consider simplifying this complex logical expression.
          Open

                  if msg.HasField("reference_set") and not any([msg.HasField(f) for f in ("value", "value_str", "second_field", "between")]):
                      return True
                  elif msg.HasField("value") and not any([msg.HasField(f) for f in ("value_str", "second_field", "between", "reference_set")]):
                      return True
                  elif msg.HasField("value_str") and not any([msg.HasField(f) for f in ("second_field", "between", "reference_set")]):
          Severity: Critical
          Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

            Function update has a Cognitive Complexity of 17 (exceeds 5 allowed). Consider refactoring.
            Open

                def update(self, column_values_dictionary):
                    columns = copy.deepcopy(column_values_dictionary)
                    self.total += 1
                    if isinstance(self.dependent_columns, str):
                        v1 = columns[self.dependent_columns]
            Severity: Minor
            Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function flush has a Cognitive Complexity of 17 (exceeds 5 allowed). Consider refactoring.
            Open

                def flush(self, rotation_suffix: Optional[str] = None):
                    """
                    Synchronously perform all remaining write tasks
                    """
                    if not self._active:
            Severity: Minor
            Found in src/whylogs/app/logger.py - About 2 hrs to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Method run has a Cognitive Complexity of 16 (exceeds 5 allowed). Consider refactoring.
            Open

              @SneakyThrows
              @Override
              public void run() {
                validateFiles();
            
            
            Severity: Minor
            Found in java/cli/src/main/java/com/whylogs/cli/Profiler.java - About 2 hrs to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function columnKLDivergenceLessThanConstraint has a Cognitive Complexity of 16 (exceeds 5 allowed). Consider refactoring.
            Open

            def columnKLDivergenceLessThanConstraint(reference_distribution: Union[List[Any], np.ndarray], threshold: float = 0.5, name=None, verbose: bool = False):
                """
                Defines a summary constraint specifying the expected
                upper limit of the threshold for the KL divergence of the specified feature.
            
            
            Severity: Minor
            Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function columnChiSquaredTestPValueGreaterThanConstraint has a Cognitive Complexity of 16 (exceeds 5 allowed). Consider refactoring.
            Open

            def columnChiSquaredTestPValueGreaterThanConstraint(
                reference_distribution: Union[List[Any], np.ndarray, Mapping[str, int]], p_value: float = 0.05, name=None, verbose: bool = False
            ):
                """
                Defines a summary constraint specifying the expected
            Severity: Minor
            Found in src/whylogs/core/statistics/constraints.py - About 2 hrs to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function log_segment_datum has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
            Open

                def log_segment_datum(self, feature_name, value, character_list: str = None, token_method: Optional[Callable] = None):
                    segment = [{"key": feature_name, "value": value}]
                    segment_profile = self.get_segment(segment)
                    if self.segment_type == "keys":
                        if feature_name in self.segments:
            Severity: Minor
            Found in src/whylogs/app/logger.py - About 1 hr to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function __init__ has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
            Open

                def __init__(
                    self,
                    first_field: str,
                    op: Op,
                    value=None,
            Severity: Minor
            Found in src/whylogs/core/statistics/constraints.py - About 1 hr to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Severity
            Category
            Status
            Source
            Language