Sign upLogin

getindata/kedro-airflow-k8s

View on GitHub
Star

Showing 35 of 40 total issues

File config.py has 363 lines of code (exceeds 250 allowed). Consider refactoring.
Open

import os

from kedro.config import MissingConfigException

DEFAULT_CONFIG_TEMPLATE = """
Severity: Minor
Found in kedro_airflow_k8s/config.py - About 4 hrs to fix

    File task_group.py has 295 lines of code (exceeds 250 allowed). Consider refactoring.
    Open

    import logging
from collections import defaultdict
from inspect import signature
from typing import Callable, Dict, List, Optional, Set
    Severity: Minor
    Found in kedro_airflow_k8s/task_group.py - About 3 hrs to fix

      File cli.py has 286 lines of code (exceeds 250 allowed). Consider refactoring.
      Open

      import logging
import os
import webbrowser
from pathlib import Path
from typing import Dict, List, Optional
      Severity: Minor
      Found in kedro_airflow_k8s/cli.py - About 2 hrs to fix

        RunConfig has 24 functions (exceeds 20 allowed). Consider refactoring.
        Open

        class RunConfig(Config):
    @property
    def image(self):
        return self._get_or_fail("image")
        Severity: Minor
        Found in kedro_airflow_k8s/config.py - About 2 hrs to fix

          Function __init__ has 20 arguments (exceeds 4 allowed). Consider refactoring.
          Open

              def __init__(
          Severity: Major
          Found in kedro_airflow_k8s/operators/spark_submit_k8s.py - About 2 hrs to fix

            Function __init__ has 6 arguments (exceeds 4 allowed). Consider refactoring.
            Open

                def __init__(
            Severity: Minor
            Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py - About 45 mins to fix

              Function dump_init_script has 6 arguments (exceeds 4 allowed). Consider refactoring.
              Open

                  def dump_init_script(
              Severity: Minor
              Found in kedro_airflow_k8s/cli_helper.py - About 45 mins to fix

                Function _create_template_stream has 5 arguments (exceeds 4 allowed). Consider refactoring.
                Open

                def _create_template_stream(
                Severity: Minor
                Found in kedro_airflow_k8s/template.py - About 35 mins to fix

                  Function get_dag_filename_and_template_stream has 5 arguments (exceeds 4 allowed). Consider refactoring.
                  Open

                  def get_dag_filename_and_template_stream(
                  Severity: Minor
                  Found in kedro_airflow_k8s/template.py - About 35 mins to fix

                    Function run_once has 5 arguments (exceeds 4 allowed). Consider refactoring.
                    Open

                    def run_once(
                    Severity: Minor
                    Found in kedro_airflow_k8s/cli.py - About 35 mins to fix

                      Function submit_operator has 5 arguments (exceeds 4 allowed). Consider refactoring.
                      Open

                          def submit_operator(
                      Severity: Minor
                      Found in kedro_airflow_k8s/template_helper.py - About 35 mins to fix

                        Function execute has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                        Open

                            def execute(self, context: Dict):
        """
        On pipeline start it may be required to create experiment if it does not exist
        which happens in this method. Obtains experiment run_id and passes to xcom as
         `mlflow_run_id` so it can be referenced by ML related tasks.
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py - About 35 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Function setup_resources has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                        Open

                            def setup_resources(limits_cpu, limits_memory, requests_cpu):
        """
        Sets up driver and executor configuration for k8s limits and requests
        :param limits_cpu:
        :param limits_memory:
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/spark_submit_k8s.py - About 35 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Too many local variables (28/15)
                        Open

                            def __init__(  # pylint: disable=too-many-arguments
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/node_pod.py by pylint

                        Used when a function or method has too many local variables.

                        Missing class docstring
                        Open

                        class VarsAuthHandler(AuthHandler):
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a class has no docstring.Even an empty class must have a docstring.

                        Too few public methods (1/2)
                        Open

                        class VarsAuthHandler(AuthHandler):
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when class has too few public methods, so be sure it's really worth it.

                        Missing function or method docstring
                        Open

                            def instance():
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a function or method has no docstring.Some special methods like init do not require a docstring.

                        Variable name e doesn't conform to snake_case naming style
                        Open

                                    except Exception as e:
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when the name doesn't conform to naming rules associated to its type (constant, variable, class...).

                        Specify string format arguments as logging function parameters
                        Open

                                        self.log.error("Failed to obtain IAP access token. " + str(e))
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a logging statement has a call form of logging.<logging method>(format_string % (format_args...)). Such calls should leave string interpolation to the logging method itself and be written logging.<logging method>(format_string, format_args...) so that the program may avoid incurring the cost of the interpolation in those cases in which no message will be logged. For more, see http://www.python.org/dev/peps/pep-0282/.

                        Catching too general exception Exception
                        Open

                                    except Exception as e:
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when an except catches a too general exception, possibly burying unrelated errors.

                        Severity
                        Category
                        Status
                        Source
                        Language