Sign upLogin

getindata/kedro-airflow-k8s

View on GitHub
Star

Showing 35 of 40 total issues

File config.py has 363 lines of code (exceeds 250 allowed). Consider refactoring.
Open

import os

from kedro.config import MissingConfigException

DEFAULT_CONFIG_TEMPLATE = """
Severity: Minor
Found in kedro_airflow_k8s/config.py - About 4 hrs to fix

    File task_group.py has 295 lines of code (exceeds 250 allowed). Consider refactoring.
    Open

    import logging
from collections import defaultdict
from inspect import signature
from typing import Callable, Dict, List, Optional, Set
    Severity: Minor
    Found in kedro_airflow_k8s/task_group.py - About 3 hrs to fix

      File cli.py has 286 lines of code (exceeds 250 allowed). Consider refactoring.
      Open

      import logging
import os
import webbrowser
from pathlib import Path
from typing import Dict, List, Optional
      Severity: Minor
      Found in kedro_airflow_k8s/cli.py - About 2 hrs to fix

        RunConfig has 24 functions (exceeds 20 allowed). Consider refactoring.
        Open

        class RunConfig(Config):
    @property
    def image(self):
        return self._get_or_fail("image")
        Severity: Minor
        Found in kedro_airflow_k8s/config.py - About 2 hrs to fix

          Function __init__ has 20 arguments (exceeds 4 allowed). Consider refactoring.
          Open

              def __init__(
          Severity: Major
          Found in kedro_airflow_k8s/operators/spark_submit_k8s.py - About 2 hrs to fix

            Function dump_init_script has 6 arguments (exceeds 4 allowed). Consider refactoring.
            Open

                def dump_init_script(
            Severity: Minor
            Found in kedro_airflow_k8s/cli_helper.py - About 45 mins to fix

              Function __init__ has 6 arguments (exceeds 4 allowed). Consider refactoring.
              Open

                  def __init__(
              Severity: Minor
              Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py - About 45 mins to fix

                Function _create_template_stream has 5 arguments (exceeds 4 allowed). Consider refactoring.
                Open

                def _create_template_stream(
                Severity: Minor
                Found in kedro_airflow_k8s/template.py - About 35 mins to fix

                  Function get_dag_filename_and_template_stream has 5 arguments (exceeds 4 allowed). Consider refactoring.
                  Open

                  def get_dag_filename_and_template_stream(
                  Severity: Minor
                  Found in kedro_airflow_k8s/template.py - About 35 mins to fix

                    Function run_once has 5 arguments (exceeds 4 allowed). Consider refactoring.
                    Open

                    def run_once(
                    Severity: Minor
                    Found in kedro_airflow_k8s/cli.py - About 35 mins to fix

                      Function submit_operator has 5 arguments (exceeds 4 allowed). Consider refactoring.
                      Open

                          def submit_operator(
                      Severity: Minor
                      Found in kedro_airflow_k8s/template_helper.py - About 35 mins to fix

                        Function setup_resources has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                        Open

                            def setup_resources(limits_cpu, limits_memory, requests_cpu):
        """
        Sets up driver and executor configuration for k8s limits and requests
        :param limits_cpu:
        :param limits_memory:
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/spark_submit_k8s.py - About 35 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Function execute has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                        Open

                            def execute(self, context: Dict):
        """
        On pipeline start it may be required to create experiment if it does not exist
        which happens in this method. Obtains experiment run_id and passes to xcom as
         `mlflow_run_id` so it can be referenced by ML related tasks.
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py - About 35 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Too few public methods (1/2)
                        Open

                        class AuthHandler:
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when class has too few public methods, so be sure it's really worth it.

                        Too many arguments (20/5)
                        Open

                            def __init__(
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/spark_submit_k8s.py by pylint

                        Used when a function or method takes too many arguments.

                        Too many local variables (24/15)
                        Open

                            def __init__(
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/spark_submit_k8s.py by pylint

                        Used when a function or method has too many local variables.

                        Missing class docstring
                        Open

                        class GoogleOAuth2AuthHandler(AuthHandler):
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a class has no docstring.Even an empty class must have a docstring.

                        return statement in finally block may swallow exception
                        Open

                                        return {"MLFLOW_TRACKING_TOKEN": token}
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a break or a return statement is found inside the finally clause of a try...finally block: the exceptions raised in the try clause will be silently swallowed instead of being re-raised.

                        Unused argument 'context'
                        Open

                            def execute(self, context: Dict):
                        Severity: Minor
                        Found in kedro_airflow_k8s/operators/delete_pipeline_storage.py by pylint

                        Used when a function or method argument is not used.

                        Missing function or method docstring
                        Open

                            def instance():
                        Severity: Info
                        Found in kedro_airflow_k8s/operators/start_mlflow_experiment.py by pylint

                        Used when a function or method has no docstring.Some special methods like init do not require a docstring.

                        Severity
                        Category
                        Status
                        Source
                        Language