Sign upLogin

fedspendingtransparency/usaspending-api

View on GitHub
Star

Showing 164 of 226 total issues

File spending_over_time.py has 415 lines of code (exceeds 250 allowed). Consider refactoring.
Open

import copy
import logging
from calendar import monthrange
from collections import OrderedDict
from datetime import datetime, timezone
Severity: Minor
Found in usaspending_api/search/v2/views/spending_over_time.py - About 5 hrs to fix

    Identical blocks of code found in 2 locations. Consider refactoring.
    Open

                    if row["awarding_agency_code"] is None or len(row["awarding_agency_code"].strip()) < 1:
                    awarding_subtier_agency_id = subtier_agency_map[row["awarding_sub_tier_agency_c"]]
                    awarding_toptier_agency_id = subtier_to_agency_map[awarding_subtier_agency_id]["toptier_agency_id"]
                    awarding_toptier_code = toptier_agency_map[awarding_toptier_agency_id]
                    row["awarding_agency_code"] = awarding_toptier_code
    Severity: Major
    Found in usaspending_api/broker/management/commands/update_transactions.py and 1 other location - About 5 hrs to fix
    usaspending_api/broker/management/commands/update_transactions.py on lines 110..114

    Duplicated Code

    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

    Tuning

    This issue has a mass of 97.

    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

    Refactorings

    Further Reading

    Identical blocks of code found in 2 locations. Consider refactoring.
    Open

                if row["awarding_agency_code"] is None or len(row["awarding_agency_code"].strip()) < 1:
                awarding_subtier_agency_id = subtier_agency_map[row["awarding_sub_tier_agency_c"]]
                awarding_toptier_agency_id = subtier_to_agency_map[awarding_subtier_agency_id]["toptier_agency_id"]
                awarding_toptier_code = toptier_agency_map[awarding_toptier_agency_id]
                row["awarding_agency_code"] = awarding_toptier_code
    Severity: Major
    Found in usaspending_api/broker/management/commands/update_transactions.py and 1 other location - About 5 hrs to fix
    usaspending_api/broker/management/commands/update_transactions.py on lines 278..282

    Duplicated Code

    Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

    Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

    When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

    Tuning

    This issue has a mass of 97.

    We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

    The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

    If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

    See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

    Refactorings

    Further Reading

    File test_award_index_elasticsearch.py has 413 lines of code (exceeds 250 allowed). Consider refactoring.
    Open

    import pytest

from model_bakery import baker
    Severity: Minor
    Found in usaspending_api/tests/integration/test_award_index_elasticsearch.py - About 5 hrs to fix

      Identical blocks of code found in 2 locations. Consider refactoring.
      Open

      def build_spending_update_query(query_base, update_data):
    values_string = ""
    for count, row in enumerate(update_data, 1):
        values_string += "("
        values_string += ",".join(["%s"] * len(row))
      Severity: Major
      Found in usaspending_api/database_scripts/job_archive/backfill_solicitation_date.py and 1 other location - About 5 hrs to fix
      usaspending_api/database_scripts/job_archive/backfill_per_transaction_exec_comp.py on lines 203..211

      Duplicated Code

      Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

      Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

      When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

      Tuning

      This issue has a mass of 95.

      We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

      The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

      If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

      See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

      Refactorings

      Further Reading

      Identical blocks of code found in 2 locations. Consider refactoring.
      Open

      def build_spending_update_query(query_base, update_data):
    values_string = ""
    for count, row in enumerate(update_data, 1):
        values_string += "("
        values_string += ",".join(["%s"] * len(row))
      Severity: Major
      Found in usaspending_api/database_scripts/job_archive/backfill_per_transaction_exec_comp.py and 1 other location - About 5 hrs to fix
      usaspending_api/database_scripts/job_archive/backfill_solicitation_date.py on lines 99..107

      Duplicated Code

      Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

      Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

      When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

      Tuning

      This issue has a mass of 95.

      We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

      The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

      If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

      See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

      Refactorings

      Further Reading

      File recipients.py has 405 lines of code (exceeds 250 allowed). Consider refactoring.
      Open

      import logging
import uuid

from decimal import Decimal
      Severity: Minor
      Found in usaspending_api/recipient/v2/views/recipients.py - About 5 hrs to fix

        File transaction_fpds.py has 402 lines of code (exceeds 250 allowed). Consider refactoring.
        Open

        from usaspending_api.common.data_classes import TransactionColumn

TRANSACTION_FPDS_COLUMN_INFO = [
    TransactionColumn("a_76_fair_act_action", "a_76_fair_act_action", "STRING"),
    TransactionColumn("a_76_fair_act_action_desc", "a_76_fair_act_action_desc", "STRING"),
        Severity: Minor
        Found in usaspending_api/transactions/delta_models/transaction_fpds.py - About 5 hrs to fix

          File disaster_covid19_file_d1_awards.py has 391 lines of code (exceeds 250 allowed). Consider refactoring.
          Open

          d1_awards_sql_string = """
SELECT
    award_search.generated_unique_award_id AS contract_award_unique_key,
    award_search.piid AS award_id_piid,
    latest_transaction.referenced_idv_agency_iden AS parent_award_agency_id,
          Severity: Minor
          Found in usaspending_api/disaster/management/sql/spark/disaster_covid19_file_d1_awards.py - About 5 hrs to fix

            Cyclomatic complexity is too high in function type_filter. (41)
            Open

            def type_filter(_type, filters, limit=None):
    _types = [
        "budget_function",
        "budget_subfunction",
        "federal_account",
            Severity: Minor
            Found in usaspending_api/spending_explorer/v2/filters/type_filter.py by radon

            Cyclomatic Complexity

            Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

            Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

            Construct Effect on CC Reasoning
            if +1 An if statement is a single decision.
            elif +1 The elif statement adds another decision.
            else +0 The else statement does not cause a new decision. The decision is at the if.
            for +1 There is a decision at the start of the loop.
            while +1 There is a decision at the while statement.
            except +1 Each except branch adds a new conditional path of execution.
            finally +0 The finally block is unconditionally executed.
            with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
            assert +1 The assert statement internally roughly equals a conditional statement.
            Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
            Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

            Source: http://radon.readthedocs.org/en/latest/intro.html

            File gsdm_maps.py has 381 lines of code (exceeds 250 allowed). Consider refactoring.
            Open

            """
This is a map of code enumerations, extracted from the GSDM

Each should be named following this pattern:
    - For a field that is unique among all models, <field>_map
            Severity: Minor
            Found in usaspending_api/data/gsdm_maps.py - About 5 hrs to fix

              Function get_download_sources has a Cognitive Complexity of 44 (exceeds 15 allowed). Consider refactoring.
              Open

              def get_download_sources(
    json_request: dict, download_job: DownloadJob = None, origination: Optional[str] = None
) -> List[DownloadSource]:
    download_sources = []
    for download_type in json_request["download_types"]:
              Severity: Minor
              Found in usaspending_api/download/filestreaming/download_generation.py - About 5 hrs to fix

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              File field_mappings_fpds.py has 378 lines of code (exceeds 250 allowed). Consider refactoring.
              Open

              from usaspending_api.broker.helpers.award_category_helper import award_types
from usaspending_api.etl.transaction_loaders.derived_field_functions_fpds import (
    calculate_fiscal_year,
    calculate_awarding_agency,
    calculate_funding_agency,
              Severity: Minor
              Found in usaspending_api/etl/transaction_loaders/field_mappings_fpds.py - About 5 hrs to fix

                Cyclomatic complexity is too high in method build_elasticsearch_result. (39)
                Open

                    def build_elasticsearch_result(self, response: dict) -> Dict[str, dict]:
        results = {}
        geo_info_buckets = response.get("group_by_agg_key", {}).get("buckets", [])

        for bucket in geo_info_buckets:
                Severity: Minor
                Found in usaspending_api/disaster/v2/views/spending_by_geography.py by radon

                Cyclomatic Complexity

                Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

                Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

                Construct Effect on CC Reasoning
                if +1 An if statement is a single decision.
                elif +1 The elif statement adds another decision.
                else +0 The else statement does not cause a new decision. The decision is at the if.
                for +1 There is a decision at the start of the loop.
                while +1 There is a decision at the while statement.
                except +1 Each except branch adds a new conditional path of execution.
                finally +0 The finally block is unconditionally executed.
                with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
                assert +1 The assert statement internally roughly equals a conditional statement.
                Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
                Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

                Source: http://radon.readthedocs.org/en/latest/intro.html

                Function process_data has a Cognitive Complexity of 42 (exceeds 15 allowed). Consider refactoring.
                Open

                    def process_data(self):
        # Make sure we weren't flagged to exit
        while True:
            row = self.data_queue.get()
            # If row is none, we need to die.
                Severity: Minor
                Found in usaspending_api/common/threaded_data_loader.py - About 4 hrs to fix

                Cognitive Complexity

                Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                A method's cognitive complexity is based on a few simple rules:

                • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                • Code is considered more complex for each "break in the linear flow of the code"
                • Code is considered more complex when "flow breaking structures are nested"

                Further reading

                Cyclomatic complexity is too high in method construct_es_response_for_prime_awards. (38)
                Open

                    def construct_es_response_for_prime_awards(self, response) -> dict:
        results = []
        should_return_display_award_id = "Award ID" in self.fields
        should_return_recipient_id = "recipient_id" in self.fields
        for res in response:
                Severity: Minor
                Found in usaspending_api/search/v2/views/spending_by_award.py by radon

                Cyclomatic Complexity

                Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

                Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

                Construct Effect on CC Reasoning
                if +1 An if statement is a single decision.
                elif +1 The elif statement adds another decision.
                else +0 The else statement does not cause a new decision. The decision is at the if.
                for +1 There is a decision at the start of the loop.
                while +1 There is a decision at the while statement.
                except +1 Each except branch adds a new conditional path of execution.
                finally +0 The finally block is unconditionally executed.
                with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
                assert +1 The assert statement internally roughly equals a conditional statement.
                Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
                Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

                Source: http://radon.readthedocs.org/en/latest/intro.html

                File settings.py has 356 lines of code (exceeds 250 allowed). Consider refactoring.
                Open

                """
For more information on this file: https://docs.djangoproject.com/en/3.2/topics/settings/
For the full list of settings and their values: https://docs.djangoproject.com/en/3.2/ref/settings/
"""
                Severity: Minor
                Found in usaspending_api/settings.py - About 4 hrs to fix

                  Cyclomatic complexity is too high in function _build_usas_data_for_spark. (37)
                  Open

                  def _build_usas_data_for_spark():
    """Create Django model data to be saved to Postgres, which will then get ingested into Delta Lake"""
    # Create recipient data for two transactions; the other two will generate ad hoc
    baker.make(
        "recipient.RecipientLookup",
                  Severity: Minor
                  Found in usaspending_api/tests/conftest_spark.py by radon

                  Cyclomatic Complexity

                  Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

                  Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

                  Construct Effect on CC Reasoning
                  if +1 An if statement is a single decision.
                  elif +1 The elif statement adds another decision.
                  else +0 The else statement does not cause a new decision. The decision is at the if.
                  for +1 There is a decision at the start of the loop.
                  while +1 There is a decision at the while statement.
                  except +1 Each except branch adds a new conditional path of execution.
                  finally +0 The finally block is unconditionally executed.
                  with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
                  assert +1 The assert statement internally roughly equals a conditional statement.
                  Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
                  Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

                  Source: http://radon.readthedocs.org/en/latest/intro.html

                  Function handle has a Cognitive Complexity of 40 (exceeds 15 allowed). Consider refactoring.
                  Open

                      def handle(self, *args, **options):
        """Run the application."""

        # Make sure
        #   settings.BULK_DOWNLOAD_S3_BUCKET_NAME
                  Severity: Minor
                  Found in usaspending_api/download/management/commands/populate_monthly_files.py - About 4 hrs to fix

                  Cognitive Complexity

                  Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                  A method's cognitive complexity is based on a few simple rules:

                  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                  • Code is considered more complex for each "break in the linear flow of the code"
                  • Code is considered more complex when "flow breaking structures are nested"

                  Further reading

                  Cyclomatic complexity is too high in method post. (36)
                  Open

                      @cache_response()
    def post(self, request, pk, format=None):
        # create response
        response = {"results": {}}
                  Severity: Minor
                  Found in usaspending_api/accounts/views/federal_accounts_v2.py by radon

                  Cyclomatic Complexity

                  Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.

                  Radon analyzes the AST tree of a Python program to compute Cyclomatic Complexity. Statements have the following effects on Cyclomatic Complexity:

                  Construct Effect on CC Reasoning
                  if +1 An if statement is a single decision.
                  elif +1 The elif statement adds another decision.
                  else +0 The else statement does not cause a new decision. The decision is at the if.
                  for +1 There is a decision at the start of the loop.
                  while +1 There is a decision at the while statement.
                  except +1 Each except branch adds a new conditional path of execution.
                  finally +0 The finally block is unconditionally executed.
                  with +1 The with statement roughly corresponds to a try/except block (see PEP 343 for details).
                  assert +1 The assert statement internally roughly equals a conditional statement.
                  Comprehension +1 A list/set/dict comprehension of generator expression is equivalent to a for loop.
                  Boolean Operator +1 Every boolean operator (and, or) adds a decision point.

                  Source: http://radon.readthedocs.org/en/latest/intro.html

                  Severity
                  Category
                  Status
                  Source
                  Language