File feature_generators.py
has 567 lines of code (exceeds 400 allowed). Consider refactoring. Open
import verboselogs, logging
logger = verboselogs.VerboseLogger(__name__)
from collections import OrderedDict
Function __init__
has 6 arguments (exceeds 5 allowed). Consider refactoring. Open
def __init__(
Function _validate_keys
has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring. Open
def _validate_keys(self, aggregation_config):
for key in [
"from_obj",
"intervals",
"knowledge_date_column",
- Read upRead up
Cognitive Complexity
Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.
A method's cognitive complexity is based on a few simple rules:
- Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
- Code is considered more complex for each "break in the linear flow of the code"
- Code is considered more complex when "flow breaking structures are nested"
Further reading
Function create_features_before_imputation
has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring. Open
def create_features_before_imputation(
self, feature_aggregation_config, feature_dates, state_table=None
):
"""Create features before imputation for a set of dates"""
all_tasks = self.generate_all_table_tasks(
- Read upRead up
Cognitive Complexity
Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.
A method's cognitive complexity is based on a few simple rules:
- Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
- Code is considered more complex for each "break in the linear flow of the code"
- Code is considered more complex when "flow breaking structures are nested"
Further reading
Function _validate_imputations
has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring. Open
def _validate_imputations(self, aggregation_config):
"""Validate the imputation rules in an aggregation config, looping
through all three types of aggregates. Most of the work here is
done by _validate_imputation_rule() to check the requirements of
each imputation rule found
- Read upRead up
Cognitive Complexity
Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.
A method's cognitive complexity is based on a few simple rules:
- Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
- Code is considered more complex for each "break in the linear flow of the code"
- Code is considered more complex when "flow breaking structures are nested"
Further reading
Function _validate_categoricals
has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring. Open
def _validate_categoricals(self, categoricals):
for categorical in categoricals:
if "choice_query" in categorical:
logger.spam("Validating choice query")
- Read upRead up
Cognitive Complexity
Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.
A method's cognitive complexity is based on a few simple rules:
- Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
- Code is considered more complex for each "break in the linear flow of the code"
- Code is considered more complex when "flow breaking structures are nested"
Further reading
Function _explain_selects
has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring. Open
def _explain_selects(self, aggregations):
with self.db_engine.begin() as conn:
for aggregation in aggregations:
for selectlist in aggregation.get_selects().values():
for select in selectlist:
- Read upRead up
Cognitive Complexity
Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.
A method's cognitive complexity is based on a few simple rules:
- Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
- Code is considered more complex for each "break in the linear flow of the code"
- Code is considered more complex when "flow breaking structures are nested"
Further reading
Module level import not at top of file Open
import sqlparse
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Module level import not at top of file Open
from triage.util.conf import convert_str_to_relativedelta
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Module level import not at top of file Open
from triage.component.collate import (
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Multiple imports on one line Open
import verboselogs, logging
- Read upRead up
- Exclude checks
Place imports on separate lines.
Okay: import os\nimport sys
E401: import sys, os
Okay: from subprocess import Popen, PIPE
Okay: from myclas import MyClass
Okay: from foo.bar.yourclass import YourClass
Okay: import myclass
Okay: import foo.bar.yourclass
Module level import not at top of file Open
from triage.database_reflection import table_exists
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Line too long (95 > 88 characters) Open
independently of the cohort. Takes longer but means that features can be reused
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Module level import not at top of file Open
from collections import OrderedDict
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Module level import not at top of file Open
import sqlalchemy
- Read upRead up
- Exclude checks
Place imports at the top of the file.
Always put imports at the top of the file, just after any module
comments and docstrings, and before module globals and constants.
Okay: import os
Okay: # this is a comment\nimport os
Okay: '''this is a module docstring'''\nimport os
Okay: r'''this is a module docstring'''\nimport os
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nelse:\n\tpass\nimport y
Okay:
try:\n\timport x\nexcept ImportError:\n\tpass\nfinally:\n\tpass\nimport y
E402: a=1\nimport os
E402: 'One string'\n"Two string"\nimport os
E402: a=1\nfrom sys import x
Okay: if x:\n import os
Expected 2 blank lines, found 1 Open
class FeatureGenerator:
- Read upRead up
- Exclude checks
Separate top-level function and class definitions with two blank lines.
Method definitions inside a class are separated by a single blank
line.
Extra blank lines may be used (sparingly) to separate groups of
related functions. Blank lines may be omitted between a bunch of
related one-liners (e.g. a set of dummy implementations).
Use blank lines in functions, sparingly, to indicate logical
sections.
Okay: def a():\n pass\n\n\ndef b():\n pass
Okay: def a():\n pass\n\n\nasync def b():\n pass
Okay: def a():\n pass\n\n\n# Foo\n# Bar\n\ndef b():\n pass
Okay: default = 1\nfoo = 1
Okay: classify = 1\nfoo = 1
E301: class Foo:\n b = 0\n def bar():\n pass
E302: def a():\n pass\n\ndef b(n):\n pass
E302: def a():\n pass\n\nasync def b(n):\n pass
E303: def a():\n pass\n\n\n\ndef b(n):\n pass
E303: def a():\n\n\n\n pass
E304: @decorator\n\ndef a():\n pass
E305: def a():\n pass\na()
E306: def a():\n def b():\n pass\n def c():\n pass
Line too long (94 > 88 characters) Open
features_ignore_cohort (boolean, optional) Whether or not features should be built
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (97 > 88 characters) Open
"Features can only be grouped at the {} level.".format(self.entity_id_column)
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (94 > 88 characters) Open
f"where {self.features_schema_name}.{imputed_table}.entity_id is null limit 1"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (120 > 88 characters) Open
"Computed list of categoricals: {self.categorical_cache[choice_query]} for choice query: {choice_query}"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (93 > 88 characters) Open
f"Insert query {query_num}: {sqlparse.format(str(query), reindent=True)}"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (99 > 88 characters) Open
f"Imputation failed for {len(nullcols)} columns. Null values remain in: {nullcols}"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (113 > 88 characters) Open
f'Building categorical choices for column {categorical["column"]}, metrics {categorical["metrics"]}',
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (93 > 88 characters) Open
state_table (string) schema.table_name for state table with all entity/date pairs
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (101 > 88 characters) Open
raise ValueError(f"Table task type must be aggregation or imputation, was: {table_task}")
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (114 > 88 characters) Open
logger.debug(f"Created tasks for aggregation {self._clean_table_name(aggregation.get_table_name())}" )
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (115 > 88 characters) Open
f"{len(prepares)} prepare queries, {len(inserts)} insert queries, {len(finalize)} finalize queries"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (139 > 88 characters) Open
return f"CREATE INDEX ON {aggregation.get_table_name(imputed=imputed)} ({self.entity_id_column}, {aggregation.output_date_column})"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (119 > 88 characters) Open
logger.debug(f"Skipping feature creation for table {self._clean_table_name(aggregation.get_table_name())}")
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (114 > 88 characters) Open
def _generate_imp_table_tasks_for(self, aggregation, impute_cols=None, nonimpute_cols=None, drop_preagg=True):
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (95 > 88 characters) Open
f"Prepare query {query_num}: {sqlparse.format(str(query), reindent=True)}",
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (122 > 88 characters) Open
f"Building collate.SpacetimeAggregation for config {aggregation_config} and {len(feature_dates)} as_of_dates",
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (94 > 88 characters) Open
each with keys for different stages of table creation (prepare, inserts, finalize)
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (93 > 88 characters) Open
"In the future, please exclude this key from your feature configuration."
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (91 > 88 characters) Open
f"Imputed feature table {imputed_table} did not contain rows from the "
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (95 > 88 characters) Open
f"Finalize query {query_num}: {sqlparse.format(str(query), reindent=True)}"
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (108 > 88 characters) Open
f"No state table defined in aggregation, cannot create imputation table for {imp_tbl_name}",
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (123 > 88 characters) Open
f"State table {aggregation.state_table} does not exist, cannot create imputation table for {imp_tbl_name}",
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Line too long (92 > 88 characters) Open
table_tasks_aggregate = self.generate_all_table_tasks(aggs, task_type="aggregation")
- Read upRead up
- Exclude checks
Limit all lines to a maximum of 79 characters.
There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side. The default wrapping on such
devices looks ugly. Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.
Reports error E501.
Whitespace before ')' Open
logger.debug(f"Created tasks for aggregation {self._clean_table_name(aggregation.get_table_name())}" )
- Read upRead up
- Exclude checks
Avoid extraneous whitespace.
Avoid extraneous whitespace in these situations:
- Immediately inside parentheses, brackets or braces.
- Immediately before a comma, semicolon, or colon.
Okay: spam(ham[1], {eggs: 2})
E201: spam( ham[1], {eggs: 2})
E201: spam(ham[ 1], {eggs: 2})
E201: spam(ham[1], { eggs: 2})
E202: spam(ham[1], {eggs: 2} )
E202: spam(ham[1 ], {eggs: 2})
E202: spam(ham[1], {eggs: 2 })
E203: if x == 4: print x, y; x, y = y , x
E203: if x == 4: print x, y ; x, y = y, x
E203: if x == 4 : print x, y; x, y = y, x