Sign upLogin

unb-cic-esw/facebook-page-post-scraper

View on GitHub
Star
scraper/page_scraper.py

Summary

Maintainability
C
1 day
Test Coverage

Showing 8 of 10 total issues

File page_scraper.py has 363 lines of code (exceeds 250 allowed). Consider refactoring.
Open

"""@page_scraper Responsavel por definir Scraper e seus metodos."""
 
import os
import json
import csv
Severity: Minor
Found in scraper/page_scraper.py - About 4 hrs to fix

    Function get_reactions has a Cognitive Complexity of 26 (exceeds 5 allowed). Consider refactoring.
    Open

        def get_reactions(self, page=None, since_date=None, until_date=None):
            """Raspa informaƧƵes da pƔgina referentes a reaƧƵes."""
            if page is None:
                page = self.page
            if not self.valid_page(page):
    Severity: Minor
    Found in scraper/page_scraper.py - About 3 hrs to fix

    Cyclomatic complexity is too high in method get_reactions. (15)
    Open

        def get_reactions(self, page=None, since_date=None, until_date=None):
            """Raspa informaƧƵes da pƔgina referentes a reaƧƵes."""
            if page is None:
                page = self.page
            if not self.valid_page(page):
    Severity: Minor
    Found in scraper/page_scraper.py by radon

    Cyclomatic complexity is too high in method write_to_csv. (11)
    Open

        def write_to_csv(self, file_name='scraped'):
            """Grava informaƧƵes da pƔgina raspada em um arquivo CSV."""
     
            def dict_to_list():
                content = []
    Severity: Minor
    Found in scraper/page_scraper.py by radon

    Cyclomatic complexity is too high in method scrape_current_page. (6)
    Open

        def scrape_current_page(self, page=None, feed=False, query=''):
            """Raspa dados de uma pƔgina selecionada."""
            if page is not None:
                self.set_page(page)
            graph = facebook.GraphAPI(access_token=self.token, version="2.12")
    Severity: Minor
    Found in scraper/page_scraper.py by radon

    Function write_to_csv has a Cognitive Complexity of 12 (exceeds 5 allowed). Consider refactoring.
    Open

        def write_to_csv(self, file_name='scraped'):
            """Grava informaƧƵes da pƔgina raspada em um arquivo CSV."""
     
            def dict_to_list():
                content = []
    Severity: Minor
    Found in scraper/page_scraper.py - About 1 hr to fix

    Similar blocks of code found in 2 locations. Consider refactoring.
    Wontfix

            with open(
                    'json/' + strftime("%Y-%m-%d") + '/' + file + '.json',
                    'r', encoding='utf8'
    Severity: Major
    Found in scraper/page_scraper.py and 1 other location - About 1 hr to fix
    scraper/page_scraper.py on lines 87..89

    Similar blocks of code found in 2 locations. Consider refactoring.
    Wontfix

            with open(
                    'json/' + strftime("%Y-%m-%d") + '/' + file + '.json',
                    'w', encoding='utf8'
    Severity: Major
    Found in scraper/page_scraper.py and 1 other location - About 1 hr to fix
    scraper/page_scraper.py on lines 351..353

    Similar blocks of code found in 2 locations. Consider refactoring.
    Open

            num_comments = 0 if 'comments' not in status else \
                status['comments']['summary']['total_count']
    Severity: Minor
    Found in scraper/page_scraper.py and 1 other location - About 45 mins to fix
    scraper/page_scraper.py on lines 194..195

    Similar blocks of code found in 2 locations. Consider refactoring.
    Open

            num_reactions = 0 if 'reactions' not in status else \
                status['reactions']['summary']['total_count']
    Severity: Minor
    Found in scraper/page_scraper.py and 1 other location - About 45 mins to fix
    scraper/page_scraper.py on lines 196..197
    Category
    Status