Sign upLogin

dssg/triage

View on GitHub
Star
src/triage/component/catwalk/evaluation.py

Summary

Maintainability
D
2 days
Test Coverage

File evaluation.py has 775 lines of code (exceeds 400 allowed). Consider refactoring.
Open

import functools
import itertools
import verboselogs, logging

logger = verboselogs.VerboseLogger(__name__)
Severity: Major
Found in src/triage/component/catwalk/evaluation.py - About 1 day to fix

    Function evaluate has a Cognitive Complexity of 23 (exceeds 5 allowed). Consider refactoring.
    Open

        def evaluate(
        self, predictions_proba, matrix_store, model_id, protected_df=None, subset=None
    ):
        """Evaluate a model based on predictions, and save the results
    Severity: Minor
    Found in src/triage/component/catwalk/evaluation.py - About 3 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _compute_evaluations has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.
    Open

        def _compute_evaluations(self, predictions_proba, labels, metric_definitions):
        """Compute evaluations for a set of predictions and labels

        Args:
            predictions_proba (np.array) predictions, sorted by score descending
    Severity: Minor
    Found in src/triage/component/catwalk/evaluation.py - About 1 hr to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _write_audit_to_db has 10 arguments (exceeds 5 allowed). Consider refactoring.
    Open

        def _write_audit_to_db(
    Severity: Major
    Found in src/triage/component/catwalk/evaluation.py - About 1 hr to fix

      Function _write_to_db has 8 arguments (exceeds 5 allowed). Consider refactoring.
      Open

          def _write_to_db(
      Severity: Major
      Found in src/triage/component/catwalk/evaluation.py - About 50 mins to fix

        Function _write_audit_to_db has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
        Open

            def _write_audit_to_db(
        self,
        model_id,
        protected_df,
        predictions_proba,
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py - About 35 mins to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Module level import not at top of file
        Open

        from triage.util.db import scoped_session
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (95 > 88 characters)
        Open

                    Returned empty dataframe for model_id = {model_id}, and subset_hash = {subset_hash}
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        from . import metrics
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (126 > 88 characters)
        Open

                        f"Using all the predictions of model {model_id} on matrix {matrix_store.uuid} for evaluation (i.e. no subset)"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (156 > 88 characters)
        Open

                    f"For model {model_id}, {len(metric_defs_to_trial)} metric definitions need {SORT_TRIALS} random trials each as best/worst evals were different"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (111 > 88 characters)
        Open

                    tie_breaker: 'best' or 'worst' case tiebreaking rule that the predictions and labels were sorted by
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        import statistics
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from aequitas.preprocessing import preprocess_input_df
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from .utils import (
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from triage.util.random import generate_python_random_seed
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (98 > 88 characters)
        Open

                """Flatten lists of metrics, parameters, and thresholds into individual metric definitions
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        import math
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (89 > 88 characters)
        Open

                    threshold_unit (string) the type of threshold, either 'percentile' or 'top_n'
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (91 > 88 characters)
        Open

                        # using threshold configuration, convert probabilities to predicted classes
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        from collections import defaultdict
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (93 > 88 characters)
        Open

                        Should contain the key 'metrics', and optionally 'parameters' or 'thresholds'
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (89 > 88 characters)
        Open

                        "Not a thresholded group, generating evaluation based on all predictions"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (89 > 88 characters)
        Open

                    metric_config_groups (list) A list of metric group configuration dictionaries
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        import pandas as pd
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        import ohio.ext.pandas
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from aequitas.bias import Bias
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (90 > 88 characters)
        Open

                # if no bias config, no aequitas audits needed, so just return False at this point
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (127 > 88 characters)
        Open

                    f"Predictions from {model_id} sorted by best case scenario, i.e. all positive labels first, NULL labels at the end"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Multiple imports on one line
        Open

        import verboselogs, logging
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports on separate lines.

        Okay: import os\nimport sys
E401: import sys, os

Okay: from subprocess import Popen, PIPE
Okay: from myclas import MyClass
Okay: from foo.bar.yourclass import YourClass
Okay: import myclass
Okay: import foo.bar.yourclass

        Line too long (95 > 88 characters)
        Open

            Returns: (pandas.Series, np.array, pandas.DataFrame) The labels, predictions, and protected
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (90 > 88 characters)
        Open

                """Encode the metric parameters and threshold into a short, human-parseable string
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (127 > 88 characters)
        Open

                                f"{metric_def.metric} not defined for parameter {metric_def.parameter_combination} because all values "
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (102 > 88 characters)
        Open

                        f"Subsetting labels and predictions of model {model_id} on matrix {matrix_store.uuid}"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (100 > 88 characters)
        Open

                        f"Needed evaluations for model {model_id} on matrix {matrix_store.uuid} are missing"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (128 > 88 characters)
        Open

                                f"{metric_def.metric} not defined for parameter {metric_def.parameter_combination} because no entities "
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (113 > 88 characters)
        Open

                    f"Predictions from {model_id} sorted by worst case scenario, i.e. all negative and NULL labels first"
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        import typing
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from triage.component.catwalk.storage import MatrixStore
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (89 > 88 characters)
        Open

                    parameter_combination (dict) The non-threshold parameter keys and values used
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (113 > 88 characters)
        Open

                """Retrieve the correct metric config groups for the matrix type and flatten them into metric definitions
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (91 > 88 characters)
        Open

                # if we do have bias config, return True. Too complicated with aequitas' visibility
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        import numpy as np
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Module level import not at top of file
        Open

        from aequitas.fairness import Fairness
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (98 > 88 characters)
        Open

                """Flatten lists of metrics, parameters, and thresholds into individual metric definitions
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (100 > 88 characters)
        Open

                # The list of needed metrics and parameters are all the unique metric/params from the config
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        from aequitas.group import Group
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (89 > 88 characters)
        Open

                    threshold_unit (string) the type of threshold, either 'percentile' or 'top_n'
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (103 > 88 characters)
        Open

                        Each dict should contain the key 'metrics', and optionally 'parameters' or 'thresholds'
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (93 > 88 characters)
        Open

                    (bool) whether or not this matrix and model are missing any evaluations in the db
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Module level import not at top of file
        Open

        from sqlalchemy.orm import sessionmaker
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Place imports at the top of the file.

        Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x

Okay: if x:\n    import os

        Line too long (95 > 88 characters)
        Open

                Returns: (list of MetricEvaluationResult objects) One result for each metric definition
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        Line too long (131 > 88 characters)
        Open

                                    f"{metric_def.metric} not defined for parameter {metric_def.parameter_combination} because all labels "
        Severity: Minor
        Found in src/triage/component/catwalk/evaluation.py by pep8

        Limit all lines to a maximum of 79 characters.

        There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

        There are no issues that match your filters.

        Category
        Status