Sign upLogin

CartoDB/cartodb20

View on GitHub
Star
app/models/data_import.rb

Summary

Maintainability
F
1 wk
Test Coverage

File data_import.rb has 927 lines of code (exceeds 250 allowed). Consider refactoring.
Open

require 'sequel'
require 'fileutils'
require_relative './user'
require_relative './table'
require_relative './visualization/member'
Severity: Major
Found in app/models/data_import.rb - About 2 days to fix

    Class DataImport has 59 methods (exceeds 20 allowed). Consider refactoring.
    Open

    class DataImport < Sequel::Model
  include Carto::DataImportConstants
  include Carto::Configuration
  include Carto::UUIDHelper
  include Carto::DataImportCommons
    Severity: Major
    Found in app/models/data_import.rb - About 1 day to fix

      Method store_results has a Cognitive Complexity of 24 (exceeds 5 allowed). Consider refactoring.
      Open

        def store_results(importer=nil, runner=nil, datasource_provider=nil, manual_fields={})
    if importer.nil?
      set_error(manual_fields.fetch(:error_code, 99999), manual_fields.fetch(:log_info, nil))
    else
      self.results    = importer.results
      Severity: Minor
      Found in app/models/data_import.rb - About 3 hrs to fix

      Cognitive Complexity

      Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

      A method's cognitive complexity is based on a few simple rules:

      • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
      • Code is considered more complex for each "break in the linear flow of the code"
      • Code is considered more complex when "flow breaking structures are nested"

      Further reading

      Method new_importer has 83 lines of code (exceeds 25 allowed). Consider refactoring.
      Open

        def new_importer
    manual_fields = {}
    had_errors = false
    log.append('new_importer()')
      Severity: Major
      Found in app/models/data_import.rb - About 3 hrs to fix

        Method track_results has a Cognitive Complexity of 21 (exceeds 5 allowed). Consider refactoring.
        Open

          def track_results(results, import_id, import_properties)
    current_user_id = current_user.id
    return unless current_user_id

    if visualization_id
        Severity: Minor
        Found in app/models/data_import.rb - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method run_import! has a Cognitive Complexity of 20 (exceeds 5 allowed). Consider refactoring.
        Open

          def run_import!
    self.resque_ppid = Process.ppid
    self.server = Socket.gethostname
    log.append("Running on server #{server} with PID: #{Process.pid}")
        Severity: Minor
        Found in app/models/data_import.rb - About 2 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method notify has 68 lines of code (exceeds 25 allowed). Consider refactoring.
        Open

          def notify(results, import_id=nil)
    owner = ::User.where(:id => self.user_id).first
    imported_tables = results.select {|r| r.success }.length
    failed_tables = results.length - imported_tables
        Severity: Major
        Found in app/models/data_import.rb - About 2 hrs to fix

          Method run_import! has 65 lines of code (exceeds 25 allowed). Consider refactoring.
          Open

            def run_import!
    self.resque_ppid = Process.ppid
    self.server = Socket.gethostname
    log.append("Running on server #{server} with PID: #{Process.pid}")
          Severity: Major
          Found in app/models/data_import.rb - About 2 hrs to fix

            Method notify has a Cognitive Complexity of 15 (exceeds 5 allowed). Consider refactoring.
            Open

              def notify(results, import_id=nil)
    owner = ::User.where(:id => self.user_id).first
    imported_tables = results.select {|r| r.success }.length
    failed_tables = results.length - imported_tables
            Severity: Minor
            Found in app/models/data_import.rb - About 1 hr to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Method from_table has a Cognitive Complexity of 13 (exceeds 5 allowed). Consider refactoring.
            Open

              def from_table
    log.append('from_table()')

    number_of_tables = 1
    quota_checker = CartoDB::QuotaChecker.new(current_user)
            Severity: Minor
            Found in app/models/data_import.rb - About 1 hr to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Method update_synchronization has a Cognitive Complexity of 12 (exceeds 5 allowed). Consider refactoring.
            Open

              def update_synchronization(importer)
    if synchronization_id
      log.type = Carto::Log::TYPE_SYNCHRONIZATION
      log.store
      log.append("synchronization_id: #{synchronization_id}")
            Severity: Minor
            Found in app/models/data_import.rb - About 1 hr to fix

            Cognitive Complexity

            Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

            A method's cognitive complexity is based on a few simple rules:

            • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
            • Code is considered more complex for each "break in the linear flow of the code"
            • Code is considered more complex when "flow breaking structures are nested"

            Further reading

            Method new_importer_with_runner has 30 lines of code (exceeds 25 allowed). Consider refactoring.
            Open

              def new_importer_with_runner(downloader, unpacker, post_import_handler)
    runner = CartoDB::Importer2::Runner.new(
      pg: pg_options,
      downloader: downloader,
      log: log,
            Severity: Minor
            Found in app/models/data_import.rb - About 1 hr to fix

              Method migrate_existing has 30 lines of code (exceeds 25 allowed). Consider refactoring.
              Open

                def migrate_existing(imported_name)
    log.append('migrate_existing()')

    table         = ::Table.new
    table.user_id = user_id
              Severity: Minor
              Found in app/models/data_import.rb - About 1 hr to fix

                Method import_from_query has 29 lines of code (exceeds 25 allowed). Consider refactoring.
                Open

                  def import_from_query(name, query)
    log.append('import_from_query()')

    self.data_type    = TYPE_QUERY
    self.data_source  = query
                Severity: Minor
                Found in app/models/data_import.rb - About 1 hr to fix

                  Method new_importer_with_connector has 28 lines of code (exceeds 25 allowed). Consider refactoring.
                  Open

                    def new_importer_with_connector
    provider_name = get_provider_name_from_id(service_item_id)
    CartoDB::Importer2::ConnectorRunner.check_availability!(current_user, provider_name)

    database_options = pg_options
                  Severity: Minor
                  Found in app/models/data_import.rb - About 1 hr to fix

                    Method track_results has 27 lines of code (exceeds 25 allowed). Consider refactoring.
                    Open

                      def track_results(results, import_id, import_properties)
    current_user_id = current_user.id
    return unless current_user_id

    if visualization_id
                    Severity: Minor
                    Found in app/models/data_import.rb - About 1 hr to fix

                      Method handle_failure has 26 lines of code (exceeds 25 allowed). Consider refactoring.
                      Open

                        def handle_failure(supplied_exception = nil)
    self.success    = false
    self.state      = STATE_FAILURE
    if !supplied_exception.nil? && supplied_exception.respond_to?(:error_code)
      self.error_code = supplied_exception.error_code
                      Severity: Minor
                      Found in app/models/data_import.rb - About 1 hr to fix

                        Method data_source= has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def data_source=(data_source)
    if data_source.nil?
      values[:data_type] = TYPE_DATASOURCE
      values[:data_source] = ''
    elsif self.data_type != TYPE_QUERY
                        Severity: Minor
                        Found in app/models/data_import.rb - About 55 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Method execute_importer has a Cognitive Complexity of 8 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def execute_importer(importer, runner, datasource_provider = nil, manual_fields = nil)
    if importer
      tracker = lambda do |state|
        self.state = state
        save
                        Severity: Minor
                        Found in app/models/data_import.rb - About 45 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Method get_downloader has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def get_downloader(datasource_provider)
    log.append("Fetching datasource #{datasource_provider} metadata for item id #{service_item_id}")

    metadata = datasource_provider.get_resource_metadata(service_item_id)
                        Severity: Minor
                        Found in app/models/data_import.rb - About 35 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Method ogr2ogr_options has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def ogr2ogr_options
    options = Cartodb.config.fetch(:ogr2ogr, {})
    if options['binary'].nil? || options['csv_guessing'].nil?
      {}
    else
                        Severity: Minor
                        Found in app/models/data_import.rb - About 25 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Method running_import_ids has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def running_import_ids
    Resque::Worker.all.map do |worker|
      next unless worker.job['queue'] == 'imports'
      worker.job['payload']['args'].first['job_id'] rescue nil
    end.compact
                        Severity: Minor
                        Found in app/models/data_import.rb - About 25 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Method handle_failure has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                        Open

                          def handle_failure(supplied_exception = nil)
    self.success    = false
    self.state      = STATE_FAILURE
    if !supplied_exception.nil? && supplied_exception.respond_to?(:error_code)
      self.error_code = supplied_exception.error_code
                        Severity: Minor
                        Found in app/models/data_import.rb - About 25 mins to fix

                        Cognitive Complexity

                        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                        A method's cognitive complexity is based on a few simple rules:

                        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                        • Code is considered more complex for each "break in the linear flow of the code"
                        • Code is considered more complex when "flow breaking structures are nested"

                        Further reading

                        Identical blocks of code found in 2 locations. Consider refactoring.
                        Open

                          def content_guessing_options
    guessing_config = Cartodb.config.fetch(:importer, {}).deep_symbolize_keys.fetch(:content_guessing, {})
    geocoder_config = Cartodb.config.fetch(:geocoder, {}).deep_symbolize_keys
    if guessing_config[:enabled] and self.content_guessing and geocoder_config
      { guessing: guessing_config, geocoder: geocoder_config }
                        Severity: Minor
                        Found in app/models/data_import.rb and 1 other location - About 40 mins to fix
                        app/models/synchronization/member.rb on lines 537..544

                        Duplicated Code

                        Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                        Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                        When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                        Tuning

                        This issue has a mass of 38.

                        We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                        The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                        If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                        See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                        Refactorings

                        Further Reading

                        Identical blocks of code found in 2 locations. Consider refactoring.
                        Open

                          def running_import_ids
    Resque::Worker.all.map do |worker|
      next unless worker.job['queue'] == 'imports'
      worker.job['payload']['args'].first['job_id'] rescue nil
    end.compact
                        Severity: Minor
                        Found in app/models/data_import.rb and 1 other location - About 25 mins to fix
                        app/controllers/home_controller.rb on lines 175..179

                        Duplicated Code

                        Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                        Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                        When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                        Tuning

                        This issue has a mass of 31.

                        We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                        The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                        If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                        See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                        Refactorings

                        Further Reading

                        There are no issues that match your filters.

                        Category
                        Status