File migrate.py has 2830 lines of code (exceeds 250 allowed). Consider refactoring.

Open

""" Support schema migration

:Author: Arthur Goldberg <Arthur.Goldberg@mssm.edu>
:Date: 2018-11-18
:Copyright: 2018, Karr Lab

Function import_module_for_migration has a Cognitive Complexity of 116 (exceeds 5 allowed). Consider refactoring.

Open

    def import_module_for_migration(self, validate=True, required_attrs=None, debug=False,
                                    mod_patterns=None, print_code=False):
        """ Import a schema from a Python module in a file, which may be in a package

        Args:

• Read up
  Read up

Function validate has a Cognitive Complexity of 57 (exceeds 5 allowed). Consider refactoring.

Open

    def validate(self):
        """ Validate the attributes of a migration specification

        Returns:
            :obj:`list` of :obj:`str`: list of errors found

• Read up
  Read up

Function _get_inconsistencies has a Cognitive Complexity of 46 (exceeds 5 allowed). Consider refactoring.

Open

    def _get_inconsistencies(self, existing_model, migrated_model):
        """ Detect inconsistencies between `existing_model` and `migrated_model` model classes

        Detect inconsistencies between `existing_model` and `migrated_model`. Inconsistencies arise if the loaded `existing_model`
        or `migrated_model` definitions are not consistent with their model or attribute renaming specifications

• Read up
  Read up

Function find_schema_modules has a Cognitive Complexity of 30 (exceeds 5 allowed). Consider refactoring.

Open

    def find_schema_modules():
        """ Find the modules used by a schema

        Useful for creating schema changes files for a schema repo

• Read up
  Read up

Migrator has 28 functions (exceeds 20 allowed). Consider refactoring.

Open

class Migrator(object):
    """ Support schema migration

    Attributes:
        existing_schema (:obj:`SchemaModule`): the existing schema, and its properties

Function _connect_models has a Cognitive Complexity of 22 (exceeds 5 allowed). Consider refactoring.

Open

    def _connect_models(self, existing_migrated_pairs):
        """ Connect migrated model instances

        Migrate :obj:`obj_tables.RelatedAttribute` connections among existing models to migrated models

• Read up
  Read up

Function standardize has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.

Open

    def standardize(self):
        """ Standardize the attributes of a :obj:`MigrationSpec`

        In particular, standardize a :obj:`MigrationSpec` that has been read from a YAML config file
        """

• Read up
  Read up

Function validate has a Cognitive Complexity of 18 (exceeds 5 allowed). Consider refactoring.

Open

    def validate(schema_changes_kwargs):
        """ Check that the attributes of the arguments to :obj:`SchemaChanges` have the right structure

        Args:
            schema_changes_kwargs (:obj:`dict`): kwargs arguments to :obj:`SchemaChanges` generated by loading a schema

• Read up
  Read up

GitRepo has 22 functions (exceeds 20 allowed). Consider refactoring.

Open

class GitRepo(object):
    """ Methods for processing a git repo and its commit history

    Attributes:
        repo_dir (:obj:`str`): the repo's root directory

Function _validate_renamed_models has a Cognitive Complexity of 12 (exceeds 5 allowed). Consider refactoring.

Open

    def _validate_renamed_models(self):
        """ Validate renamed models

        Ensure that renamed models:

• Read up
  Read up

Function _migrate_expression has a Cognitive Complexity of 12 (exceeds 5 allowed). Consider refactoring.

Open

    def _migrate_expression(self, existing_analyzed_expr):
        """ Migrate a model instance's :obj:`ParsedExpression.expression`

        The :obj:`ParsedExpression` syntax supports model type names in a Name.model_id notation.
        If a model type name changes then these must be migrated.

• Read up
  Read up

Function _validate_renamed_attrs has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.

Open

    def _validate_renamed_attrs(self):
        """ Validate renamed attributes

        Ensure that renamed attributes:

• Read up
  Read up

Function __str__ has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.

Open

    def __str__(self):
        """ Provide a string representation

        Returns:
            :obj:`str`: a string representation of this `GitRepo`

• Read up
  Read up

Function _migrate_model has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.

Open

    def _migrate_model(self, existing_model, existing_model_def, migrated_model_def):
        """ Migrate a model instance's non-related attributes

        Args:
            existing_model (:obj:`obj_tables.Model`): an existing model instance

• Read up
  Read up

Function load_config_file has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.

Open

    def load_config_file(data_schema_migration_conf_file):
        """ Load a data-schema migration config file

        Args:
            data_schema_migration_conf_file (:obj:`str`): path to the data-schema migration config file

• Read up
  Read up

Function _check_imported_models has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.

Open

    def _check_imported_models(self, module=None):
        """ Check consistency of an imported module

        Args:
            module (:obj:`Module`, optional): a `Module` containing subclasses of `obj_tables.Model`;

• Read up
  Read up

Function make_data_schema_migration_conf_file_cmd has 29 lines of code (exceeds 25 allowed). Consider refactoring.

Open

    def make_data_schema_migration_conf_file_cmd(data_repo_dir, schema_file_url, files_to_migrate,
                                                 add_to_repo=True):
        """ Make a data-schema migration configuration file from CLI input

        Args:

Function migrate_over_schema_sequence has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.

Open

    def migrate_over_schema_sequence(migration_spec):
        """ Migrate some model files over a sequence of schemas

        Args:
            migration_spec (:obj:`MigrationSpec`): a migration specification

• Read up
  Read up

Function _migrate_all_analyzed_exprs has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.

Open

    def _migrate_all_analyzed_exprs(self, all_models):
        """ Migrate all model instances' :obj:`ParsedExpression`\ s

        This must be done after all migrated models have been created.

• Read up
  Read up

Function import_custom_IO_classes has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.

Open

    def import_custom_IO_classes(self, io_classes_file_basename=None):
        """ If the schema repo has an IO classes file, import custom IO classes for accessing data files

        Args:
            io_classes_file_basename (:obj:`str`, optional): custom basename for the custom IO classes file,

• Read up
  Read up

Function automated_migrate has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.

Open

    def automated_migrate(self, tmp_dir=None):
        """ Migrate the *data* repo's data files

        Migrate to the latest commit referenced by a schema changes file in the *schema* repo, and
        migrate data files in place.

• Read up
  Read up

Function __init__ has 6 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def __init__(self, schema_repo=None, schema_changes_file=None, commit_hash=None, renamed_models=None,

Function write_migrated_file has 6 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def write_migrated_file(self, migrated_models, model_order, existing_file, migrated_file=None,

Function __init__ has 6 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def __init__(self, existing_defs_file=None, migrated_defs_file=None, renamed_models=None,

Avoid deeply nested control flow statements.

Open

                            if len(pair) != 2 or not isinstance(pair[0], str) or not isinstance(pair[1], str):
                                errors.append(required_structure)
            except TypeError as e:

Function _get_model_defs has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.

Open

    def _get_model_defs(self, module):
        """ Obtain the `obj_tables.Model`\ s in a module

        Args:
            module (:obj:`Module`): a `Module` containing subclasses of `obj_tables.Model`

• Read up
  Read up

Avoid deeply nested control flow statements.

Open

                        for model in val:
                            migrated_val.append(getattr(model, self._migrated_copy_attr_name))
                    else:

Function _write has 6 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def _write(self, existing_file, migrated_models, schema_metadata_model=None, migrated_file=None,

Avoid deeply nested control flow statements.

Open

                        if existing_rel_attr != migrated_rel_attr:
                            inconsistencies.append("{}.{}.{} is '{}', which differs from the migrated value "
                                                   "of {}.{}.{}, which is '{}'".format(
                                                       existing_model, existing_attr_name, rel_attr,
                                                       existing_rel_attr, migrated_class, migrated_attr, rel_attr, migrated_rel_attr))

Avoid deeply nested control flow statements.

Open

                        if compiled_mod_pattern.search(name):
                            if hasattr(module, '__file__'):
                                print('\t', name, module.__file__)
                            else:
                                print('\t', name)

Avoid deeply nested control flow statements.

Open

                        if existing_rel_attr_name in self.deleted_models:
                            inconsistencies.append("existing model '{}' is not migrated, "
                                                   "but is referenced by migrated attribute {}.{}".format(existing_rel_attr_name,
                                                                                                          migrated_class, migrated_attr))
                        else:

Avoid deeply nested control flow statements.

Open

                        if os.path.isfile(path):
                            print_file(path)
                self.module_name, self.directory

Avoid deeply nested control flow statements.

Open

                            for attr_spec in attribute_renaming:
                                if len(attr_spec) != 2 or not isinstance(attr_spec[0], str) or \
                                        not isinstance(attr_spec[1], str):
                                    errors.append(required_structure)
            except TypeError as e:

Function import_module_for_migration has 5 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def import_module_for_migration(self, validate=True, required_attrs=None, debug=False,

Function make_template has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def make_template(self, schema_url=None, commit_hash=None):
        """ Make a template schema changes file

        The template includes the repo hash which it describes and empty values for :obj:`SchemaChanges`
        attributes.

• Read up
  Read up

Function __init__ has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def __init__(self, repo_location=None, repo_url=None, branch='master', search_parent_directories=False):
        """ Initialize a :obj:`GitRepo` from an existing Git repo

        If :obj:`repo_location` is a directory then use the Git repo in the directory. Otherwise it must
        be an URL and the repo is cloned into a temporary directory.

• Read up
  Read up

Function make_data_schema_migration_conf_file_cmd has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def make_data_schema_migration_conf_file_cmd(data_repo_dir, schema_file_url, files_to_migrate,
                                                 add_to_repo=True):
        """ Make a data-schema migration configuration file from CLI input

        Args:

• Read up
  Read up

Function all_schema_changes_with_commits has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def all_schema_changes_with_commits(schema_repo):
        """ Instantiate all schema changes in a git repo

        Obtain all validated schema change files.

• Read up
  Read up

Function make_migration_config_file has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def make_migration_config_file(data_git_repo, schema_repo_name, add_to_repo=True, **kwargs):
        """ Create a data-schema migration config file

        Args:
            data_git_repo (:obj:`GitRepo`): the data git repo that contains the data files to migrate

• Read up
  Read up

Function verify_schemas has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def verify_schemas(self):
        """ Verify that each schema can be independently imported

        It can be difficult to import a schema via `importlib.import_module()` in
        `import_module_for_migration()`. This method tests that proactively.

• Read up
  Read up

Function __str__ has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def __str__(self):
        """ Get string representation

        Returns:
            :obj:`str`: string representation of a :obj:`Migrator`; collections attributes are rendered

• Read up
  Read up

Function load has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def load(schema_changes_file):
        """ Read a schema changes file

        Args:
            schema_changes_file (:obj:`str`): path to the schema changes file

• Read up
  Read up

Function commits_as_graph has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def commits_as_graph(self):
        """ Make a DAG for this repo's commit dependencies - edges point from dependent commit to parent commit

        The DAG contains all commits in the repo on which the latest commit depends. Also creates
        `git_hash_map`, a map from all git hashes to their commits.

• Read up
  Read up

Function validate has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def validate(self):
        """ Validate files to migrate, and load all schema changes files

        Raises:
            :obj:`MigratorError`: if any files to migrate do not exist,

• Read up
  Read up

Function _get_mapped_attribute has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def _get_mapped_attribute(self, existing_class, existing_attribute):
        """ Get the corresponding migrated class and attribute for the existing class and attribute

        Args:
            existing_class (:obj:`obj_tables.core.ModelMeta` or :obj:`str`): an existing class

• Read up
  Read up