Sign upLogin

AlexMathew/scrapple

View on GitHub
Star

Showing 16 of 673 total issues

Function run has a Cognitive Complexity of 44 (exceeds 5 allowed). Consider refactoring.
Open

    def run(self):
        selectorClassMapping = {
            'xpath': XpathSelector,
            'css': CssSelector
        }
Severity: Minor
Found in scrapple/commands/run.py - About 6 hrs to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function traverse_next has a Cognitive Complexity of 32 (exceeds 5 allowed). Consider refactoring.
Open

def traverse_next(page, nextx, results, tabular_data_headers=[], verbosity=0):
    """
    Recursive generator to traverse through the next attribute and \
    crawl through the links to be followed.
Severity: Minor
Found in scrapple/utils/config.py - About 4 hrs to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function extract_columns has a Cognitive Complexity of 25 (exceeds 5 allowed). Consider refactoring.
Open

    def extract_columns(self, result={}, selector='', table_headers=[], attr='', connector='', default='', verbosity=0, *args, **kwargs):
        """
        Column data extraction for extract_tabular
        """
        result_list = []
Severity: Minor
Found in scrapple/selectors/selector.py - About 3 hrs to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function extract_rows has a Cognitive Complexity of 16 (exceeds 5 allowed). Consider refactoring.
Open

    def extract_rows(self, result={}, selector='', table_headers=[], attr='', connector='', default='', verbosity=0, *args, **kwargs):
        """
        Row data extraction for extract_tabular
        """
        result_list = []
Severity: Minor
Found in scrapple/selectors/selector.py - About 2 hrs to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function check_arguments has a Cognitive Complexity of 14 (exceeds 5 allowed). Consider refactoring.
Open

def check_arguments(args):
    """
    Validates the arguments passed through the CLI commands.

    :param args: The arguments passed in the CLI, parsed by the docopt module
Severity: Minor
Found in scrapple/utils/exceptions.py - About 1 hr to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function extract_rows has 9 arguments (exceeds 4 allowed). Consider refactoring.
Open

    def extract_rows(self, result={}, selector='', table_headers=[], attr='', connector='', default='', verbosity=0, *args, **kwargs):
Severity: Major
Found in scrapple/selectors/selector.py - About 1 hr to fix

Function extract_columns has 9 arguments (exceeds 4 allowed). Consider refactoring.
Open

    def extract_columns(self, result={}, selector='', table_headers=[], attr='', connector='', default='', verbosity=0, *args, **kwargs):
Severity: Major
Found in scrapple/selectors/selector.py - About 1 hr to fix

Function get_fields has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.
Open

def get_fields(config):
    """
    Recursive generator that yields the field names in the config file

    :param config: The configuration file that contains the specification of the extractor
Severity: Minor
Found in scrapple/utils/config.py - About 1 hr to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function extract_tabular has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.
Open

    def extract_tabular(self, header='', prefix='', suffix='', table_type='', *args, **kwargs):
        """
        Method for performing the tabular data extraction. \

        :param result: A dictionary containing the extracted data so far
Severity: Minor
Found in scrapple/selectors/selector.py - About 1 hr to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function extract_content has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
Open

    def extract_content(self, selector='', attr='', default='', connector='', *args, **kwargs):
        """
        Method for performing the content extraction for the particular selector type. \

        If the selector is "url", the URL of the current web page is returned.
Severity: Minor
Found in scrapple/selectors/selector.py - About 55 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Avoid deeply nested control flow statements.
Open

                            if not th in tabular_data_headers:
                                tabular_data_headers[th] = len(tabular_data_headers)
            if not self.config['scraping'].get('next'):
Severity: Major
Found in scrapple/commands/run.py - About 45 mins to fix

Function extract_content has 6 arguments (exceeds 4 allowed). Consider refactoring.
Open

    def extract_content(self, selector='', attr='', default='', connector='', *args, **kwargs):
Severity: Minor
Found in scrapple/selectors/selector.py - About 45 mins to fix

Function extract_tabular has 6 arguments (exceeds 4 allowed). Consider refactoring.
Open

    def extract_tabular(self, header='', prefix='', suffix='', table_type='', *args, **kwargs):
Severity: Minor
Found in scrapple/selectors/selector.py - About 45 mins to fix

Avoid deeply nested control flow statements.
Open

                            if not th in tabular_data_headers:
                                tabular_data_headers[th] = len(tabular_data_headers)
        except KeyboardInterrupt:
Severity: Major
Found in scrapple/commands/run.py - About 45 mins to fix

Function traverse_next has 5 arguments (exceeds 4 allowed). Consider refactoring.
Open

def traverse_next(page, nextx, results, tabular_data_headers=[], verbosity=0):
Severity: Minor
Found in scrapple/utils/config.py - About 35 mins to fix

Function form_to_json has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
Open

def form_to_json(form):
    """
    Takes the form from the POST request in the web interface, and generates the JSON config\
    file
Severity: Minor
Found in scrapple/utils/form.py - About 25 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Severity
Category
Status
Source
Language