Sign upLogin

LucaCappelletti94/tinycrawler

View on GitHub
Star

Showing 81 of 90 total issues

Missing whitespace around parameter equals
Open

    def set(self, category: str, name: str, value: int=1):
Severity: Minor
Found in tinycrawler/statistics/statistics.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Line too long (351 > 79 characters)
Open

    def __init__(self, process_spawn_event: Event, process_callback_event: Event, pages_number: Value, urls_number: Value, responses: Queue, urls: Urls, robots: Robots, file_parser: Callable[[str, BeautifulSoup, Log], None], url_validator: Callable[[str, Log], bool], statistics: Statistics, logger: Log, follow_robots_txt: bool, parser_library: str):
Severity: Minor
Found in tinycrawler/process/parser.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Missing whitespace around operator
Open

    def get(self, domains: Set[str])->Tuple[str, bool]:
Severity: Minor
Found in tinycrawler/urls/urls.py by pep8

Surround operators with a single space on either side.

- Always surround these binary operators with a single space on
  either side: assignment (=), augmented assignment (+=, -= etc.),
  comparisons (==, <, >, !=, <=, >=, in, not in, is, is not),
  Booleans (and, or, not).

- If operators with different priorities are used, consider adding
  whitespace around the operators with the lowest priorities.

Okay: i = i + 1
Okay: submitted += 1
Okay: x = x * 2 - 1
Okay: hypot2 = x * x + y * y
Okay: c = (a + b) * (a - b)
Okay: foo(bar, key='word', *args, **kwargs)
Okay: alpha[:-i]

E225: i=i+1
E225: submitted +=1
E225: x = x /2 - 1
E225: z = x **y
E225: z = 1and 1
E226: c = (a+b) * (a-b)
E226: hypot2 = x*x + y*y
E227: c = a|b
E228: msg = fmt%(errno, errmsg)

Line too long (92 > 79 characters)
Open

            work_name = "{} and other {} domains".format(get_domain(seed[0]), len(seed) - 1)
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Missing whitespace around operator
Open

    def _get_proxy(self, url: str)->Proxy:
Severity: Minor
Found in tinycrawler/process/downloader.py by pep8

Surround operators with a single space on either side.

- Always surround these binary operators with a single space on
  either side: assignment (=), augmented assignment (+=, -= etc.),
  comparisons (==, <, >, !=, <=, >=, in, not in, is, is not),
  Booleans (and, or, not).

- If operators with different priorities are used, consider adding
  whitespace around the operators with the lowest priorities.

Okay: i = i + 1
Okay: submitted += 1
Okay: x = x * 2 - 1
Okay: hypot2 = x * x + y * y
Okay: c = (a + b) * (a - b)
Okay: foo(bar, key='word', *args, **kwargs)
Okay: alpha[:-i]

E225: i=i+1
E225: submitted +=1
E225: x = x /2 - 1
E225: z = x **y
E225: z = 1and 1
E226: c = (a+b) * (a-b)
E226: hypot2 = x*x + y*y
E227: c = a|b
E228: msg = fmt%(errno, errmsg)

Missing whitespace around parameter equals
Open

            use_cli: bool=True, bloom_filters_capacity: int=1e9, responses_queue_max_size: int= 10000, download_attempts: int=10, follow_robots_txt: bool=True,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Missing whitespace around parameter equals
Open

            use_cli: bool=True, bloom_filters_capacity: int=1e9, responses_queue_max_size: int= 10000, download_attempts: int=10, follow_robots_txt: bool=True,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Line too long (82 > 79 characters)
Open

        self._load_proxies(proxy_path, proxy_list, proxy_timeout, domains_timeout,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Missing whitespace around parameter equals
Open

    def __init__(self, timeout: float, custom_timeout: Callable[[object], float]=None):
Severity: Minor
Found in tinycrawler/eta/eta.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Line too long (170 > 79 characters)
Open

    def __init__(self, data: Dict, proxy_timeout: float, domains_timeout: float, custom_domains_timeout: Callable[[str], float], follow_robots_txt: bool, robots: Robots):
Severity: Minor
Found in tinycrawler/proxy/proxy.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (368 > 79 characters)
Open

        self._urls, self._responses, self._process_callback_event, self._pages_number, self._urls_number, self._robots, self._follow_robots_txt, self._file_parser,  self._url_validator, self._logger, self._parser_library = urls, responses, process_callback_event, pages_number, urls_number, robots, follow_robots_txt, file_parser, url_validator, logger, parser_library
Severity: Minor
Found in tinycrawler/process/parser.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (86 > 79 characters)
Open

    def __init__(self, name: str, statistics: Statistics, process_spawn_event: Event):
Severity: Minor
Found in tinycrawler/process/process_handler.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (175 > 79 characters)
Open

            proxy_timeout: float=10, domains_timeout: float=10, robots_timeout: float=60*60*24, connection_timeout: float=5, cooldown_time_beetween_download_attempts: float=1,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Missing whitespace around parameter equals
Open

            proxy_timeout: float=10, domains_timeout: float=10, robots_timeout: float=60*60*24, connection_timeout: float=5, cooldown_time_beetween_download_attempts: float=1,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Missing whitespace around parameter equals
Open

            proxy_timeout: float=10, domains_timeout: float=10, robots_timeout: float=60*60*24, connection_timeout: float=5, cooldown_time_beetween_download_attempts: float=1,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Line too long (97 > 79 characters)
Open

        self._cooldown_time_beetween_download_attempts = cooldown_time_beetween_download_attempts
Severity: Minor
Found in tinycrawler/process/downloader.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Missing whitespace around operator
Open

    def _download(self, proxy: Proxy, url: str)->Response:
Severity: Minor
Found in tinycrawler/process/downloader.py by pep8

Surround operators with a single space on either side.

- Always surround these binary operators with a single space on
  either side: assignment (=), augmented assignment (+=, -= etc.),
  comparisons (==, <, >, !=, <=, >=, in, not in, is, is not),
  Booleans (and, or, not).

- If operators with different priorities are used, consider adding
  whitespace around the operators with the lowest priorities.

Okay: i = i + 1
Okay: submitted += 1
Okay: x = x * 2 - 1
Okay: hypot2 = x * x + y * y
Okay: c = (a + b) * (a - b)
Okay: foo(bar, key='word', *args, **kwargs)
Okay: alpha[:-i]

E225: i=i+1
E225: submitted +=1
E225: x = x /2 - 1
E225: z = x **y
E225: z = 1and 1
E226: c = (a+b) * (a-b)
E226: hypot2 = x*x + y*y
E227: c = a|b
E228: msg = fmt%(errno, errmsg)

Missing whitespace around parameter equals
Open

            use_cli: bool=True, bloom_filters_capacity: int=1e9, responses_queue_max_size: int= 10000, download_attempts: int=10, follow_robots_txt: bool=True,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Missing whitespace around parameter equals
Open

            proxy_timeout: float=10, domains_timeout: float=10, robots_timeout: float=60*60*24, connection_timeout: float=5, cooldown_time_beetween_download_attempts: float=1,
Severity: Minor
Found in tinycrawler/tinycrawler.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):

Missing whitespace around parameter equals
Open

    def __init__(self, timeout: float, custom_timeout: Callable[[object], float]=None):
Severity: Minor
Found in tinycrawler/eta/eta.py by pep8

Don't use spaces around the '=' sign in function arguments.

Don't use spaces around the '=' sign when used to indicate a
keyword argument or a default parameter value, except when
using a type annotation.

Okay: def complex(real, imag=0.0):
Okay: return magic(r=real, i=imag)
Okay: boolean(a == b)
Okay: boolean(a != b)
Okay: boolean(a <= b)
Okay: boolean(a >= b)
Okay: def foo(arg: int = 42):
Okay: async def foo(arg: int = 42):

E251: def complex(real, imag = 0.0):
E251: return magic(r = real, i = imag)
E252: def complex(real, image: float=0.0):
Severity
Category
Status
Source
Language