Function open has 5 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def open(self, url, splash_args:dict=None, reuse_session=False, *args, **kwargs):

Function open has 5 arguments (exceeds 4 allowed). Consider refactoring.

Open

    def open(self, url, splash_args:dict=None, reuse_session=False, *args, **kwargs):

Function _init_tasks has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def _init_tasks(self, kwargs):
        """
        Create individual task queues for the workers.

        If, Type[StatefulBook] is passed as the `tasks` parameter, the tracker with

• Read up
  Read up

Function _response_callback has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def _response_callback(self, resp):
        """
        Callback for after response received. If status code is not 200 then
        recursively retry. Retry up to five times.

• Read up
  Read up

Function _write_headers_and_set_fields_to_export has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def _write_headers_and_set_fields_to_export(self, item):
        if self.include_headers_line:
            if not self.fields_to_export:
                if isinstance(item, dict):
                    # for dicts try using fields of the first item

• Read up
  Read up

Function _next_page has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.

Open

    def _next_page(self):
        """
        Find the url to the next page from the pagination link.
        """
        if self.page:

• Read up
  Read up

Avoid too many return statements within this function.

Open

        return value

Avoid too many return statements within this function.

Open

            return f"<{type(o).__name__}, {o.status_code}, {o.url}>"

Avoid too many return statements within this function.

Open

            return dict(o)

Avoid too many return statements within this function.

Open

            return str(o)

Avoid too many return statements within this function.

Open

            return f"<{type(o).__name__}, {o.method}, {o.url}>"

Avoid too many return statements within this function.

Open

            return super(TransistorJSONEncoder, self).default(o)

Function stateful_post has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def stateful_post(self, url, *args, **kwargs):
        """Post to the URL and store the Browser's state, as received from
        the response object, in this object.

        All arguments are forwarded to :func:`SplashCrawleraBrowser.post`.

• Read up
  Read up

Function get_oldest has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

def get_oldest(class_name):
    """Get the oldest object for a specific class name"""
    for cls, wdict in live_refs.items():
        if cls.__name__ == class_name:
            if not wdict:

• Read up
  Read up

Function _add_soup has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

Open

    def _add_soup(self, response, soup_config):
        """Attaches a soup object to a requests response."""
        if self.resp_headers:
            if ("text/html" in self.resp_content_type_header or
                    SplashBrowser.__looks_like_html(self.html)):

• Read up
  Read up