Sign upLogin

LucaCappelletti94/tinycrawler

View on GitHub
Star

Showing 5 of 90 total issues

Function _print_all has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
Open

    def _print_all(self):
        max_len = self._max_len + self.PADDING
        for k, v in self._outputs.items():
            self._window.addstr(k, 0, "|", curses.A_DIM)
            self._window.addstr(k, max_len, "|", curses.A_DIM)
Severity: Minor
Found in tinycrawler/cli/cli.py - About 45 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function _url_parser has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
Open

    def _url_parser(self, page_url: str, soup: BeautifulSoup) -> Set[str]:
        urls = set()
        for a in soup.findAll("a", href=True):
            url = urljoin(page_url, a.get("href"))
            if (
Severity: Minor
Found in tinycrawler/process/parser.py - About 45 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function get has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
Open

    def get(self, domains: Set[str])->Tuple[str, bool]:
        for domain in set(self._domains.keys()) - set(domains):
            try:
                return (self._domains[domain].get_nowait(), True)
            except Empty:
Severity: Minor
Found in tinycrawler/urls/urls.py - About 25 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function _target has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
Open

    def _target(self, proxy: Proxy, url: str):
        """Tries to download an url with a proxy n times"""
        for attempts in range(self._download_attempts):
            response = self._download(proxy, url)
            proxy.used(bool(response), url)
Severity: Minor
Found in tinycrawler/process/downloader.py - About 25 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Function put has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
Open

    def put(self, urls: List[str]):
        n = 0
        self._put_lock.acquire()
        for url in urls:
            if url not in self._bloom:
Severity: Minor
Found in tinycrawler/urls/urls.py - About 25 mins to fix

Cognitive Complexity

Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

A method's cognitive complexity is based on a few simple rules:

  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
  • Code is considered more complex for each "break in the linear flow of the code"
  • Code is considered more complex when "flow breaking structures are nested"

Further reading

Severity
Category
Status
Source
Language