Sign upLogin

HazyResearch/fonduer

View on GitHub
Star

Showing 224 of 224 total issues

Cyclomatic complexity is too high in function _get_direction_ngrams. (15)
Open

def _get_direction_ngrams(
    direction: str,
    c: Union[Candidate, Mention, TemporarySpanMention],
    attrib: str,
    n_min: int,
Severity: Minor
Found in src/fonduer/utils/data_model_utils/visual.py by radon

Cyclomatic Complexity

Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

Construct Effect on CC Reasoning
if +1 An if statement is a single decision.
elif +1 The elif statement adds another decision.
else +0 The else statement does not cause a new decision. The decision is at the if.
for +1 There is a decision at the start of the loop.
while +1 There is a decision at the while statement.
except +1 Each except branch adds a new conditional path of execution.
finally +0 The finally block is unconditionally executed.
with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
assert +1 The assert statement internally roughly equals a conditional statement.
Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

Source: http://radon.readthedocs.org/en/latest/intro.html

File pdf_visual_parser.py has 304 lines of code (exceeds 250 allowed). Consider refactoring.
Open

"""Fonduer visual parser that parses visual information from PDF."""
import logging
import os
import re
import shutil
Severity: Minor
Found in src/fonduer/parser/visual_parser/pdf_visual_parser.py - About 3 hrs to fix

    Cyclomatic complexity is too high in method _link_lists. (14)
    Open

        def _link_lists(
        self, search_max: int = 100, edit_cost: int = 20, offset_cost: int = 1
    ) -> None:
        # NOTE: there are probably some inefficiencies here from rehashing words
        # multiple times, but we're not going to worry about that for now
    Severity: Minor
    Found in src/fonduer/parser/visual_parser/pdf_visual_parser.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Cyclomatic complexity is too high in method enrich_sentences_with_NLP. (14)
    Open

        def enrich_sentences_with_NLP(
        self, sentences: Collection[Sentence]
    ) -> Iterator[Sentence]:
        """Enrich a list of fonduer Sentence objects with NLP features.
    Severity: Minor
    Found in src/fonduer/parser/lingual_parser/spacy_parser.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Cyclomatic complexity is too high in function get_neighbor_cell_ngrams. (13)
    Open

    def get_neighbor_cell_ngrams(
    mention: Union[Candidate, Mention, TemporarySpanMention],
    dist: int = 1,
    directions: bool = False,
    attrib: str = "words",
    Severity: Minor
    Found in src/fonduer/utils/data_model_utils/tabular.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Cyclomatic complexity is too high in method _apply. (12)
    Open

        def _apply(
        self, doc_loader: Collection[Document], parallelism: int, **kwargs: Any
    ) -> None:
        """Run the UDF multi-threaded using python multiprocessing."""
        if not Meta.postgres:
    Severity: Minor
    Found in src/fonduer/utils/udf.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Cyclomatic complexity is too high in method parse. (12)
    Open

        def parse(self, document: Document, text: str) -> Iterator[Sentence]:
        """Depth-first search over the provided tree.

        Implemented as an iterative procedure. The structure of the state
        needed to parse each node is also defined in this function.
    Severity: Minor
    Found in src/fonduer/parser/parser.py by radon

    Cyclomatic Complexity

    Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

    Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

    Construct Effect on CC Reasoning
    if +1 An if statement is a single decision.
    elif +1 The elif statement adds another decision.
    else +0 The else statement does not cause a new decision. The decision is at the if.
    for +1 There is a decision at the start of the loop.
    while +1 There is a decision at the while statement.
    except +1 Each except branch adds a new conditional path of execution.
    finally +0 The finally block is unconditionally executed.
    with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
    assert +1 The assert statement internally roughly equals a conditional statement.
    Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
    Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

    Source: http://radon.readthedocs.org/en/latest/intro.html

    Function get_head_ngrams has a Cognitive Complexity of 20 (exceeds 5 allowed). Consider refactoring.
    Open

    def get_head_ngrams(
    mention: Union[Candidate, Mention, TemporarySpanMention],
    axis: Optional[str] = None,
    attrib: str = "words",
    n_min: int = 1,
    Severity: Minor
    Found in src/fonduer/utils/data_model_utils/tabular.py - About 2 hrs to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    File featurizer.py has 278 lines of code (exceeds 250 allowed). Consider refactoring.
    Open

    """Fonduer featurizer."""
import itertools
import logging
from collections import defaultdict
from typing import (
    Severity: Minor
    Found in src/fonduer/features/featurizer.py - About 2 hrs to fix

      Cyclomatic complexity is too high in method drop_keys. (11)
      Open

          def drop_keys(
        self,
        keys: Iterable[Union[str, Callable]],
        candidate_classes: Optional[
            Union[Type[Candidate], List[Type[Candidate]]]
      Severity: Minor
      Found in src/fonduer/supervision/labeler.py by radon

      Cyclomatic Complexity

      Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

      Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

      Construct Effect on CC Reasoning
      if +1 An if statement is a single decision.
      elif +1 The elif statement adds another decision.
      else +0 The else statement does not cause a new decision. The decision is at the if.
      for +1 There is a decision at the start of the loop.
      while +1 There is a decision at the while statement.
      except +1 Each except branch adds a new conditional path of execution.
      finally +0 The finally block is unconditionally executed.
      with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
      assert +1 The assert statement internally roughly equals a conditional statement.
      Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
      Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

      Source: http://radon.readthedocs.org/en/latest/intro.html

      Cyclomatic complexity is too high in method upsert_keys. (11)
      Open

          def upsert_keys(
        self,
        keys: Iterable[Union[str, Callable]],
        candidate_classes: Optional[
            Union[Type[Candidate], List[Type[Candidate]]]
      Severity: Minor
      Found in src/fonduer/supervision/labeler.py by radon

      Cyclomatic Complexity

      Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

      Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

      Construct Effect on CC Reasoning
      if +1 An if statement is a single decision.
      elif +1 The elif statement adds another decision.
      else +0 The else statement does not cause a new decision. The decision is at the if.
      for +1 There is a decision at the start of the loop.
      while +1 There is a decision at the while statement.
      except +1 Each except branch adds a new conditional path of execution.
      finally +0 The finally block is unconditionally executed.
      with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
      assert +1 The assert statement internally roughly equals a conditional statement.
      Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
      Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

      Source: http://radon.readthedocs.org/en/latest/intro.html

      File implicit_span_mention.py has 276 lines of code (exceeds 250 allowed). Consider refactoring.
      Open

      """Fonduer implicit span mention model."""
from typing import Any, Dict, List, Optional, Type

from sqlalchemy import Column, ForeignKey, Integer, String, UniqueConstraint
from sqlalchemy.dialects import postgresql
      Severity: Minor
      Found in src/fonduer/candidates/models/implicit_span_mention.py - About 2 hrs to fix

        Function save_model has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
        Open

        def save_model(
    fonduer_model: FonduerModel,
    path: str,
    preprocessor: DocPreprocessor,
    parser: Parser,
        Severity: Minor
        Found in src/fonduer/packaging/fonduer_model.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Function create_task has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
        Open

        def create_task(
    task_names: Union[str, List[str]],
    n_arities: Union[int, List[int]],
    n_features: int,
    n_classes: Union[int, List[int]],
        Severity: Minor
        Found in src/fonduer/learning/task.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Cyclomatic complexity is too high in function _tablelib_unary_features. (10)
        Open

        def _tablelib_unary_features(span: SpanMention) -> Iterator[Tuple[str, int]]:
    """Table-/structure-related features for a single span."""
    if not span.sentence.is_tabular():
        return
    sentence = span.sentence
        Severity: Minor
        Found in src/fonduer/features/feature_libs/tabular_features.py by radon

        Cyclomatic Complexity

        Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

        Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

        Construct Effect on CC Reasoning
        if +1 An if statement is a single decision.
        elif +1 The elif statement adds another decision.
        else +0 The else statement does not cause a new decision. The decision is at the if.
        for +1 There is a decision at the start of the loop.
        while +1 There is a decision at the while statement.
        except +1 Each except branch adds a new conditional path of execution.
        finally +0 The finally block is unconditionally executed.
        with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
        assert +1 The assert statement internally roughly equals a conditional statement.
        Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
        Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

        Source: http://radon.readthedocs.org/en/latest/intro.html

        Cyclomatic complexity is too high in method get_candidates. (10)
        Open

            def get_candidates(
        self,
        docs: Union[Document, Iterable[Document], None] = None,
        split: int = 0,
        sort: bool = False,
        Severity: Minor
        Found in src/fonduer/candidates/candidates.py by radon

        Cyclomatic Complexity

        Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

        Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

        Construct Effect on CC Reasoning
        if +1 An if statement is a single decision.
        elif +1 The elif statement adds another decision.
        else +0 The else statement does not cause a new decision. The decision is at the if.
        for +1 There is a decision at the start of the loop.
        while +1 There is a decision at the while statement.
        except +1 Each except branch adds a new conditional path of execution.
        finally +0 The finally block is unconditionally executed.
        with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
        assert +1 The assert statement internally roughly equals a conditional statement.
        Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
        Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

        Source: http://radon.readthedocs.org/en/latest/intro.html

        Cyclomatic complexity is too high in class ParserUDF. (10)
        Open

        class ParserUDF(UDF):
    """Parser UDF class."""

    def __init__(
        self,
        Severity: Minor
        Found in src/fonduer/parser/parser.py by radon

        Cyclomatic Complexity

        Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

        Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

        Construct Effect on CC Reasoning
        if +1 An if statement is a single decision.
        elif +1 The elif statement adds another decision.
        else +0 The else statement does not cause a new decision. The decision is at the if.
        for +1 There is a decision at the start of the loop.
        while +1 There is a decision at the while statement.
        except +1 Each except branch adds a new conditional path of execution.
        finally +0 The finally block is unconditionally executed.
        with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
        assert +1 The assert statement internally roughly equals a conditional statement.
        Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
        Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

        Source: http://radon.readthedocs.org/en/latest/intro.html

        Function f has a Cognitive Complexity of 18 (exceeds 5 allowed). Consider refactoring.
        Open

            def f(self, m: TemporaryContext) -> bool:
        """Non-composed version of filter function f."""
        if not isinstance(m, TemporarySpanMention):
            raise ValueError(
                f"{self.__class__.__name__} only supports TemporarySpanMention"
        Severity: Minor
        Found in src/fonduer/candidates/matchers.py - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        File candidates.py has 267 lines of code (exceeds 250 allowed). Consider refactoring.
        Open

        """Fonduer candidate."""
import logging
from builtins import range
from itertools import product
from typing import (
        Severity: Minor
        Found in src/fonduer/candidates/candidates.py - About 2 hrs to fix

          Cyclomatic complexity is too high in function drop_keys. (9)
          Open

          def drop_keys(session: Session, key_table: Table, keys: Dict) -> None:
    """Bulk drop annotation keys to the specified table.

    Rather than directly dropping the keys, this removes the candidate_classes
    specified for the given keys only. If all candidate_classes are removed for
          Severity: Minor
          Found in src/fonduer/utils/utils_udf.py by radon

          Cyclomatic Complexity

          Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

          Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

          Construct Effect on CC Reasoning
          if +1 An if statement is a single decision.
          elif +1 The elif statement adds another decision.
          else +0 The else statement does not cause a new decision. The decision is at the if.
          for +1 There is a decision at the start of the loop.
          while +1 There is a decision at the while statement.
          except +1 Each except branch adds a new conditional path of execution.
          finally +0 The finally block is unconditionally executed.
          with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
          assert +1 The assert statement internally roughly equals a conditional statement.
          Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
          Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

          Source: http://radon.readthedocs.org/en/latest/intro.html

          Severity
          Category
          Status
          Source
          Language