Sign upLogin

saltstack/salt

View on GitHub
Star
salt/fileserver/s3fs.py

Summary

Maintainability
F
1 wk
Test Coverage

Function _refresh_buckets_cache_file has a Cognitive Complexity of 98 (exceeds 5 allowed). Consider refactoring.
Open

def _refresh_buckets_cache_file(cache_file):
    '''
    Retrieve the content of all buckets and cache the metadata to the buckets
    cache file
    '''
Severity: Minor
Found in salt/fileserver/s3fs.py - About 1 day to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

File s3fs.py has 552 lines of code (exceeds 250 allowed). Consider refactoring.
Open

# -*- coding: utf-8 -*-
'''
Amazon S3 Fileserver Backend

.. versionadded:: 0.16.0
Severity: Major
Found in salt/fileserver/s3fs.py - About 1 day to fix

    Function _get_file_from_s3 has a Cognitive Complexity of 42 (exceeds 5 allowed). Consider refactoring.
    Open

    def _get_file_from_s3(metadata, saltenv, bucket_name, path, cached_file_path):
    '''
    Checks the local cache for the file, if it's old or missing go grab the
    file from S3 and update the cache
    '''
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 6 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Cyclomatic complexity is too high in function _refresh_buckets_cache_file. (33)
    Open

    def _refresh_buckets_cache_file(cache_file):
    '''
    Retrieve the content of all buckets and cache the metadata to the buckets
    cache file
    '''
    Severity: Minor
    Found in salt/fileserver/s3fs.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Function _find_dirs has a Cognitive Complexity of 23 (exceeds 5 allowed). Consider refactoring.
    Open

    def _find_dirs(metadata):
    '''
    Looks for all the directories in the S3 bucket cache metadata.

    Supports trailing '/' keys (as created by S3 console) as well as
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 3 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _find_files has a Cognitive Complexity of 16 (exceeds 5 allowed). Consider refactoring.
    Open

    def _find_files(metadata):
    '''
    Looks for all the files in the S3 bucket cache metadata
    '''
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 2 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function update has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
    Open

    def update():
    '''
    Update the cache file for the bucket.
    '''
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 1 hr to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function find_file has a Cognitive Complexity of 14 (exceeds 5 allowed). Consider refactoring.
    Open

    def find_file(path, saltenv='base', **kwargs):
    '''
    Look through the buckets cache file for a match.
    If the field is found, it is retrieved from S3 only if its cached version
    is missing, or if the MD5 does not match.
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 1 hr to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function _find_file_meta has a Cognitive Complexity of 11 (exceeds 5 allowed). Consider refactoring.
    Open

    def _find_file_meta(metadata, bucket_name, saltenv, path):
    '''
    Looks for a file's metadata in the S3 bucket cache file
    '''
    env_meta = metadata[saltenv] if saltenv in metadata else {}
    Severity: Minor
    Found in salt/fileserver/s3fs.py - About 1 hr to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Avoid deeply nested control flow statements.
    Open

                        for k in s3_meta:
                        if 'Code' in k or 'Message' in k:
                            # assumes no duplicate keys, consisdent with current erro response.
                            meta_response.update(k)
                    # attempt use of human readable output first.
    Severity: Major
    Found in salt/fileserver/s3fs.py - About 45 mins to fix

      Avoid deeply nested control flow statements.
      Open

                          if bucket_name in bucket_files:
                        bucket_files[bucket_name] += env_files
                        found = True
                        break
                if not found:
      Severity: Major
      Found in salt/fileserver/s3fs.py - About 45 mins to fix

        Avoid deeply nested control flow statements.
        Open

                            if bucket_name in bucket:
                        bucket[bucket_name] += list(dirpaths)
                        bucket[bucket_name] = list(set(bucket[bucket_name]))
                        break
    return ret
        Severity: Major
        Found in salt/fileserver/s3fs.py - About 45 mins to fix

          Function _get_s3_key has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
          Open

          def _get_s3_key():
    '''
    Get AWS keys from pillar or config
    '''
          Severity: Minor
          Found in salt/fileserver/s3fs.py - About 45 mins to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Avoid deeply nested control flow statements.
          Open

                              for file_path in files:
                        cached_file_path = _get_cached_file_name(bucket, saltenv, file_path)
                        log.info('%s - %s : %s', bucket, saltenv, file_path)

                        # load the file from S3 if it's not in the cache or it's old
          Severity: Major
          Found in salt/fileserver/s3fs.py - About 45 mins to fix

            Avoid deeply nested control flow statements.
            Open

                                    if 'Code' in meta_response:
                            log.warning(
                                "'%s' response for bucket '%s'",
                                meta_response['Code'], bucket_name
                            )
            Severity: Major
            Found in salt/fileserver/s3fs.py - About 45 mins to fix

              Avoid deeply nested control flow statements.
              Open

                                  if bucket_name in bucket:
                        bucket[bucket_name] += filepaths
                        break
    return ret
              Severity: Major
              Found in salt/fileserver/s3fs.py - About 45 mins to fix

                Avoid deeply nested control flow statements.
                Open

                                    if ret is not None:
                        for header_name, header_value in ret['headers'].items():
                            name = header_name.strip()
                            value = header_value.strip()
                            if six.text_type(name).lower() == 'last-modified':
                Severity: Major
                Found in salt/fileserver/s3fs.py - About 45 mins to fix

                  Avoid deeply nested control flow statements.
                  Open

                                      if 'Code' in k or 'Message' in k:
                        # assumes no duplicate keys, consisdent with current erro response.
                        meta_response.update(k)
                # attempt use of human readable output first.
                try:
                  Severity: Major
                  Found in salt/fileserver/s3fs.py - About 45 mins to fix

                    Function file_list has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                    Open

                    def file_list(load):
    '''
    Return a list of all files on the file server in a specified environment
    '''
    if 'env' in load:
                    Severity: Minor
                    Found in salt/fileserver/s3fs.py - About 35 mins to fix

                    Cognitive Complexity

                    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                    A method's cognitive complexity is based on a few simple rules:

                    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                    • Code is considered more complex for each "break in the linear flow of the code"
                    • Code is considered more complex when "flow breaking structures are nested"

                    Further reading

                    Function serve_file has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                    Open

                    def serve_file(load, fnd):
    '''
    Return a chunk from a file based on the data received
    '''
    if 'env' in load:
                    Severity: Minor
                    Found in salt/fileserver/s3fs.py - About 25 mins to fix

                    Cognitive Complexity

                    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                    A method's cognitive complexity is based on a few simple rules:

                    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                    • Code is considered more complex for each "break in the linear flow of the code"
                    • Code is considered more complex when "flow breaking structures are nested"

                    Further reading

                    Function dir_list has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                    Open

                    def dir_list(load):
    '''
    Return a list of all directories on the master
    '''
    if 'env' in load:
                    Severity: Minor
                    Found in salt/fileserver/s3fs.py - About 25 mins to fix

                    Cognitive Complexity

                    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                    A method's cognitive complexity is based on a few simple rules:

                    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                    • Code is considered more complex for each "break in the linear flow of the code"
                    • Code is considered more complex when "flow breaking structures are nested"

                    Further reading

                    Identical blocks of code found in 2 locations. Consider refactoring.
                    Open

                                if not files:
                meta_response = {}
                for k in s3_meta:
                    if 'Code' in k or 'Message' in k:
                        # assumes no duplicate keys, consisdent with current erro response.
                    Severity: Major
                    Found in salt/fileserver/s3fs.py and 1 other location - About 6 hrs to fix
                    salt/fileserver/s3fs.py on lines 460..482

                    Duplicated Code

                    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                    Tuning

                    This issue has a mass of 108.

                    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                    Refactorings

                    Further Reading

                    Identical blocks of code found in 2 locations. Consider refactoring.
                    Open

                                    if not bucket_files[bucket_name]:
                    meta_response = {}
                    for k in s3_meta:
                        if 'Code' in k or 'Message' in k:
                            # assumes no duplicate keys, consisdent with current erro response.
                    Severity: Major
                    Found in salt/fileserver/s3fs.py and 1 other location - About 6 hrs to fix
                    salt/fileserver/s3fs.py on lines 499..521

                    Duplicated Code

                    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                    Tuning

                    This issue has a mass of 108.

                    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                    Refactorings

                    Further Reading

                    Similar blocks of code found in 2 locations. Consider refactoring.
                    Open

                    def _get_cached_file_name(bucket_name, saltenv, path):
    '''
    Return the cached file name for a bucket path file
    '''
                    Severity: Major
                    Found in salt/fileserver/s3fs.py and 1 other location - About 3 hrs to fix
                    salt/pillar/s3.py on lines 242..253

                    Duplicated Code

                    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                    Tuning

                    This issue has a mass of 73.

                    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                    Refactorings

                    Further Reading

                    Identical blocks of code found in 2 locations. Consider refactoring.
                    Open

                                environments = [(os.path.dirname(k['Key']).split('/', 1))[0] for k in files]
                    Severity: Major
                    Found in salt/fileserver/s3fs.py and 1 other location - About 1 hr to fix
                    salt/pillar/s3.py on lines 296..296

                    Duplicated Code

                    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                    Tuning

                    This issue has a mass of 40.

                    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                    Refactorings

                    Further Reading

                    There are no issues that match your filters.

                    Category
                    Status