Sign upLogin

kymmt90/bookmeter_scraper

View on GitHub
Star

Showing 17 of 17 total issues

Class Scraper has 23 methods (exceeds 20 allowed). Consider refactoring.
Open

  class Scraper
    NUM_BOOKS_PER_PAGE = 40
    NUM_USERS_PER_PAGE = 20

    attr_accessor :agent
Severity: Minor
Found in lib/bookmeter_scraper/scraper.rb - About 2 hrs to fix

    File scraper.rb has 263 lines of code (exceeds 250 allowed). Consider refactoring.
    Open

    require 'mechanize'
require 'yasuri'

require 'bookmeter_scraper/book'
require 'bookmeter_scraper/profile'
    Severity: Minor
    Found in lib/bookmeter_scraper/scraper.rb - About 2 hrs to fix

      Method fetch_target_books_from has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
      Open

          def fetch_target_books_from(page, target_year_month)
      raise ArgumentError if target_year_month.nil? || page.nil?

      target_books = Books.new
      1.upto(NUM_BOOKS_PER_PAGE) do |i|
      Severity: Minor
      Found in lib/bookmeter_scraper/scraper.rb - About 1 hr to fix

      Cognitive Complexity

      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

      A method's cognitive complexity is based on a few simple rules:

      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
      • Code is considered more complex for each "break in the linear flow of the code"
      • Code is considered more complex when "flow breaking structures are nested"

      Further reading

      Method fetch_read_books_in has a Cognitive Complexity of 12 (exceeds 5 allowed). Consider refactoring.
      Open

          def fetch_read_books_in(target_year_month, user_id)
      raise ArgumentError unless user_id =~ USER_ID_REGEX
      raise ArgumentError if target_year_month.nil?

      result = Books.new
      Severity: Minor
      Found in lib/bookmeter_scraper/scraper.rb - About 1 hr to fix

      Cognitive Complexity

      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

      A method's cognitive complexity is based on a few simple rules:

      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
      • Code is considered more complex for each "break in the linear flow of the code"
      • Code is considered more complex when "flow breaking structures are nested"

      Further reading

      Method extract_books has 29 lines of code (exceeds 25 allowed). Consider refactoring.
      Open

          def extract_books(page)
      raise ArgumentError if page.nil?

      books = []
      1.upto(NUM_BOOKS_PER_PAGE) do |i|
      Severity: Minor
      Found in lib/bookmeter_scraper/scraper.rb - About 1 hr to fix

        Method extract_books has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.
        Open

            def extract_books(page)
      raise ArgumentError if page.nil?

      books = []
      1.upto(NUM_BOOKS_PER_PAGE) do |i|
        Severity: Minor
        Found in lib/bookmeter_scraper/scraper.rb - About 1 hr to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method fetch_target_books_from has 27 lines of code (exceeds 25 allowed). Consider refactoring.
        Open

            def fetch_target_books_from(page, target_year_month)
      raise ArgumentError if target_year_month.nil? || page.nil?

      target_books = Books.new
      1.upto(NUM_BOOKS_PER_PAGE) do |i|
        Severity: Minor
        Found in lib/bookmeter_scraper/scraper.rb - About 1 hr to fix

          Method scrape_books_pages has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
          Open

              def scrape_books_pages(user_id, uri_method, agent = @agent)
      unless user_id =~ USER_ID_REGEX && \
             BookmeterScraper.methods.include?(uri_method)
        raise ArgumentError
      end
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb - About 45 mins to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Similar blocks of code found in 2 locations. Consider refactoring.
          Open

              def scrape_reread_date(book_uri, agent = @agent)
      book_reread_date = Yasuri.struct_reread_date '//*[@id="book_edit_area"]/div/form[1]/div[2]' do
        text_reread_year  '//div[@class="reread_box"]/form[1]/div[2]/select[1]/option[1]', truncate: /\d+/, proc: :to_i
        text_reread_month '//div[@class="reread_box"]/form[1]/div[2]/select[2]/option[1]', truncate: /\d+/, proc: :to_i
        text_reread_day   '//div[@class="reread_box"]/form[1]/div[2]/select[3]/option[1]', truncate: /\d+/, proc: :to_i
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 35 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 230..236

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 36.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Similar blocks of code found in 2 locations. Consider refactoring.
          Open

              def scrape_read_date(book_uri, agent = @agent)
      book_date = Yasuri.struct_date '//*[@id="book_edit_area"]/form[1]/div[2]' do
        text_year  '//*[@id="read_date_y"]/option[1]', truncate: /\d+/, proc: :to_i
        text_month '//*[@id="read_date_m"]/option[1]', truncate: /\d+/, proc: :to_i
        text_day   '//*[@id="read_date_d"]/option[1]', truncate: /\d+/, proc: :to_i
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 35 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 239..245

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 36.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Method fetch_followings has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
          Open

              def fetch_followings(user_id, agent = @agent)
      raise ArgumentError unless user_id =~ USER_ID_REGEX
      raise ScraperError if agent.nil?
      return [] unless agent.logged_in?
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb - About 35 mins to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Method scrape_users_listing_page has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
          Open

              def scrape_users_listing_page(user_id, uri_method, agent = @agent)
      raise ArgumentError unless user_id =~ USER_ID_REGEX
      raise ArgumentError unless BookmeterScraper.methods.include?(uri_method)
      return [] unless agent.logged_in?
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb - About 25 mins to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Method fetch_books has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
          Open

              def fetch_books(user_id, uri_method, agent = @agent)
      unless user_id =~ USER_ID_REGEX && \
             BookmeterScraper.methods.include?(uri_method)
        raise ArgumentError
      end
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb - About 25 mins to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Similar blocks of code found in 2 locations. Consider refactoring.
          Open

                followings_root = Yasuri.struct_books '//*[@id="main_left"]/div' do
        1.upto(NUM_USERS_PER_PAGE) do |i|
          send("text_user_#{i}_name", "//*[@id=\"main_left\"]/div/div[#{i}]/a/@title")
          send("text_user_#{i}_link", "//*[@id=\"main_left\"]/div/div[#{i}]/a/@href")
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 25 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 307..310

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 29.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Similar blocks of code found in 2 locations. Consider refactoring.
          Open

                root = Yasuri.struct_users '//*[@id="main_left"]/div' do
        1.upto(NUM_USERS_PER_PAGE) do |i|
          send("text_user_#{i}_name", "//*[@id=\"main_left\"]/div/div[#{i}]/div/div[2]/a/@title")
          send("text_user_#{i}_link", "//*[@id=\"main_left\"]/div/div[#{i}]/div/div[2]/a/@href")
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 25 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 282..285

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 29.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Identical blocks of code found in 2 locations. Consider refactoring.
          Open

                  1.upto(NUM_BOOKS_PER_PAGE) do |i|
          send("text_book_#{i}_name", "//*[@id=\"main_left\"]/div/div[#{i + 1}]/div[2]/a")
          send("text_book_#{i}_link", "//*[@id=\"main_left\"]/div/div[#{i + 1}]/div[2]/a/@href")
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 20 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 79..81

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 28.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Identical blocks of code found in 2 locations. Consider refactoring.
          Open

                    1.upto(NUM_BOOKS_PER_PAGE) do |i|
            send("text_book_#{i}_name", "//*[@id=\"main_left\"]/div/div[#{i + 1}]/div[2]/a")
            send("text_book_#{i}_link", "//*[@id=\"main_left\"]/div/div[#{i + 1}]/div[2]/a/@href")
          Severity: Minor
          Found in lib/bookmeter_scraper/scraper.rb and 1 other location - About 20 mins to fix
          lib/bookmeter_scraper/scraper.rb on lines 89..91

          Duplicated Code

          Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

          Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

          When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

          Tuning

          This issue has a mass of 28.

          We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

          The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

          If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

          See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

          Refactorings

          Further Reading

          Severity
          Category
          Status
          Source
          Language