Sign upLogin

megagonlabs/bunkai

View on GitHub
Star

Showing 55 of 55 total issues

Function tokenize has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.
Open

    def tokenize(self, text: str, *, never_split=None, **kwargs):
        """Tokenizes a piece of text."""
        if self.normalize_text:
            text = unicodedata.normalize("NFKC", text)
Severity: Minor
Found in bunkai/algorithm/lbd/custom_tokenizers.py - About 1 hr to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function __init__ has 8 arguments (exceeds 4 allowed). Consider refactoring.
Open

        def __init__(
Severity: Major
Found in bunkai/third/utils_ner.py - About 1 hr to fix

    Function annotate has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
    Open

        def annotate(self, original_text: str, spans: Annotations) -> Annotations:
        index2token_obj = self.__generate(list(spans.get_annotation_layer(MorphAnnotatorJanome.__name__)))

        __return_span_ann = []
        for target_rule_name in self.rule_targets:
    Severity: Minor
    Found in bunkai/algorithm/bunkai_sbd/annotator/indirect_quote_exception_annotator.py - About 55 mins to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function operation has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
    Open

    def operation(data: str) -> typing.Iterator[str]:
    root = ET.fromstring(data)
    for child in root:
        if child.tag != "article":
            continue
    Severity: Minor
    Found in bunkai/experiment/convert/bccwj.py - About 55 mins to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function func_filter_span has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
    Open

    def func_filter_span(
    spans_wide: typing.List[SpanAnnotation], spans_narrow: typing.List[SpanAnnotation]
) -> typing.List[SpanAnnotation]:
    """Compare spans_wide and spans_narrow. If there is an overlap, use wider one."""
    __filtered = []
    Severity: Minor
    Found in bunkai/base/annotator.py - About 55 mins to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function main has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
    Open

    def main() -> None:
    opts = get_opts()
    with opts.input as inf, opts.output as outf:
        for line in inf:
            data = json.loads(line)
    Severity: Minor
    Found in bunkai/experiment/convert/jalan.py - About 55 mins to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Function __init__ has 7 arguments (exceeds 4 allowed). Consider refactoring.
    Open

        def __init__(
    Severity: Major
    Found in bunkai/base/annotation.py - About 50 mins to fix

      Function __init__ has 7 arguments (exceeds 4 allowed). Consider refactoring.
      Open

              def __init__(
      Severity: Major
      Found in bunkai/third/utils_ner.py - About 50 mins to fix

        Function __find_emoji has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
        Open

            def __find_emoji(self, text: str) -> typing.List[EmojiText]:
        """:return: spans of emoji index. [[start-index, end-index]]."""
        emoji_spans = []
        emoji_categories = []
        is_emoji_span: bool = False
        Severity: Minor
        Found in bunkai/algorithm/bunkai_sbd/annotator/emoji_annotator.py - About 45 mins to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Avoid deeply nested control flow statements.
        Open

                            if not opts.nonfkc:
                        string = unicodedata.normalize("NFKC", string)
                    outf.write(string)
        Severity: Major
        Found in bunkai/experiment/convert/bccwj.py - About 45 mins to fix

          Function prepare_config has 6 arguments (exceeds 4 allowed). Consider refactoring.
          Open

          def prepare_config(
          Severity: Minor
          Found in bunkai/algorithm/lbd/train.py - About 45 mins to fix

            Function convert has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
            Open

            def convert(
    inpath: typing.IO,
    tokenizer: JanomeTokenizer,
    remove_trailing_lb: bool = True,
) -> typing.Iterator[Tokens]:
            Severity: Minor
            Found in bunkai/algorithm/lbd/corpus.py - About 45 mins to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Function check has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
            Open

                def check(self) -> bool:
        """
        Check following rule-order.

        1. MorphAnnotatorJanome must be before ExceptionParticle
            Severity: Minor
            Found in bunkai/algorithm/bunkai_sbd/bunkai_sbd.py - About 45 mins to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Avoid deeply nested control flow statements.
            Open

                                for line in f:
                        if line.startswith("-DOCSTART-") or line == "" or line == "\n":
                            writer.write(line)
                            if not preds_list[example_id]:
                                example_id += 1
            Severity: Major
            Found in bunkai/third/run_ner.py - About 45 mins to fix

              Function store_updater has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
              Open

              def store_updater(path_in: Path, base_model: str, path_out: Path):
    logging.disable(logging.WARN)
    orig_model = BertForTokenClassification.from_pretrained(base_model)
    assert isinstance(orig_model, BertForTokenClassification)
    new_model = BertForTokenClassification.from_pretrained(path_in)
              Severity: Minor
              Found in bunkai/algorithm/lbd/dist.py - About 45 mins to fix

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              Function train has 6 arguments (exceeds 4 allowed). Consider refactoring.
              Open

              def train(
              Severity: Minor
              Found in bunkai/algorithm/lbd/train.py - About 45 mins to fix

                Avoid deeply nested control flow statements.
                Open

                                    if unit.tag == "sampling":
                        pass
                    elif unit.tag == "correction":
                        yield unit.get("originalText")
                    elif unit.tag not in [
                Severity: Major
                Found in bunkai/experiment/convert/bccwj.py - About 45 mins to fix

                  Avoid deeply nested control flow statements.
                  Open

                                      if tid in token_ids_seps:
                        outf.write(bunkai.constant.METACHAR_SENTENCE_BOUNDARY)
                else:
                  Severity: Major
                  Found in bunkai/algorithm/lbd/predict.py - About 45 mins to fix

                    Function __init__ has 5 arguments (exceeds 4 allowed). Consider refactoring.
                    Open

                        def __init__(self, examples, labels, max_seq_length: int, tokenizer, is_xlnet: bool):
                    Severity: Minor
                    Found in bunkai/algorithm/lbd/train.py - About 35 mins to fix

                      Function operate_article has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                      Open

                      def operate_article(parent) -> typing.Iterator[str]:
    for node in parent:
        if node.tag in [
            "rejectedBlock",
            "info",
                      Severity: Minor
                      Found in bunkai/experiment/convert/bccwj.py - About 35 mins to fix

                      Cognitive Complexity

                      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                      A method's cognitive complexity is based on a few simple rules:

                      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                      • Code is considered more complex for each "break in the linear flow of the code"
                      • Code is considered more complex when "flow breaking structures are nested"

                      Further reading

                      Severity
                      Category
                      Status
                      Source
                      Language