Sign upLogin

toodef/neural-pipeline

View on GitHub
Star
neural_pipeline/train.py

Summary

Maintainability
B
5 hrs
Test Coverage

File train.py has 277 lines of code (exceeds 250 allowed). Consider refactoring.
Open

"""
The main module for training process
"""
import json
import os
Severity: Minor
Found in neural_pipeline/train.py - About 2 hrs to fix

    Function train has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
    Open

        def train(self) -> None:
        """
        Run training process
        """
        if len(self._train_config.stages()) < 1:
    Severity: Minor
    Found in neural_pipeline/train.py - About 1 hr to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _save_state has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
    Open

        def _save_state(self, ckpts_manager: CheckpointsManager, best_ckpts_manager: CheckpointsManager or None,
                    cur_best_state: float or None, epoch_idx: int) -> float or None:
        """
        Internal method used for save states after epoch end
    Severity: Minor
    Found in neural_pipeline/train.py - About 45 mins to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _save_state has 5 arguments (exceeds 4 allowed). Consider refactoring.
    Open

        def _save_state(self, ckpts_manager: CheckpointsManager, best_ckpts_manager: CheckpointsManager or None,
    Severity: Minor
    Found in neural_pipeline/train.py - About 35 mins to fix

      Line too long (88 > 79 characters)
      Open

      from neural_pipeline.train_config.train_config import TrainConfig, ComparableTrainConfig
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (89 > 79 characters)
      Open

          This class provide lr decaying by defined metric value (by :arg:`target_value_clbk`).
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (103 > 79 characters)
      Open

              Enable rearing rate decaying. Learning rate decay when `target_val_clbk` returns doesn't update
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (94 > 79 characters)
      Open

              self._lr = DecayingLR(self._data_processor.get_lr(), coeff, patience, target_val_clbk)
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (92 > 79 characters)
      Open

              Define number of epoch for training. One epoch - one iteration over all train stages
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (92 > 79 characters)
      Open

              :param callback: method, that will be called. This method may not get any parameters
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (149 > 79 characters)
      Open

                  rule (callable): callable, that doesn't get params and return boolean. When one of rules returns `True` training loop will be interrupted
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (96 > 79 characters)
      Open

                              self.monitor_hub.update_metrics(stage.metrics_processor().get_metrics())
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (127 > 79 characters)
      Open

              self._data_processor = TrainDataProcessor(self._train_config, device).set_checkpoints_manager(self._checkpoint_manager)
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (102 > 79 characters)
      Open

          def enable_lr_decaying(self, coeff: float, patience: int, target_val_clbk: callable) -> 'Trainer':
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (108 > 79 characters)
      Open

          def _save_state(self, ckpts_manager: CheckpointsManager, best_ckpts_manager: CheckpointsManager or None,
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (113 > 79 characters)
      Open

          def __init__(self, start_value: float, decay_coefficient: float, patience: int, target_value_clbk: callable):
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (103 > 79 characters)
      Open

          def __init__(self, train_config: TrainConfig, fsm: FileStructManager, device: torch.device = None):
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (88 > 79 characters)
      Open

                  for epoch_idx in range(start_epoch_idx, self.__epoch_num + start_epoch_idx):
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (108 > 79 characters)
      Open

          If metric value doesn't update minimum after defined number of steps (:arg:`patience`) - lr was decaying
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (128 > 79 characters)
      Open

                      new_best_state = self._save_state(self._checkpoint_manager, best_checkpoints_manager, cur_best_state, epoch_idx)
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (93 > 79 characters)
      Open

              :param target_val_clbk: callback which returns the value that is used for lr decaying
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (119 > 79 characters)
      Open

                  raise NotImplementedError("Resume parameter may be only 'last' or 'best' not {}".format(self._resume_from))
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (84 > 79 characters)
      Open

                          cur_best_state: float or None, epoch_idx: int) -> float or None:
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (105 > 79 characters)
      Open

              :param rule: callback which returns the value that is used for define when need store best metric
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (80 > 79 characters)
      Open

          :param target_value_clbk: callable, that return target value for lr decaying
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (84 > 79 characters)
      Open

                          self._data_processor.set_checkpoints_manager(best_ckpts_manager)
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (83 > 79 characters)
      Open

                  trainer.add_stop_rule(lambda: trainer.data_processor().get_lr() < 1e-6)
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      Line too long (82 > 79 characters)
      Open

              :param best_ckpts_manager: checkpoints manager, used for store best stages
      Severity: Minor
      Found in neural_pipeline/train.py by pep8

      Limit all lines to a maximum of 79 characters.

      There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

      There are no issues that match your filters.

      Category
      Status