Sign upLogin

uken/fluent-plugin-elasticsearch

View on GitHub
Star
lib/fluent/plugin/out_elasticsearch.rb

Summary

Maintainability
F
1 wk
Test Coverage

File out_elasticsearch.rb has 1022 lines of code (exceeds 250 allowed). Consider refactoring.
Open

require 'date'
require 'excon'
require 'elasticsearch'
require 'set'
require 'json'
Severity: Major
Found in lib/fluent/plugin/out_elasticsearch.rb - About 2 days to fix

    Method configure has a Cognitive Complexity of 99 (exceeds 5 allowed). Consider refactoring.
    Open

        def configure(conf)
      compat_parameters_convert(conf, :buffer)

      super
      if placeholder_substitution_needed_for_template?
    Severity: Minor
    Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 day to fix

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Method configure has 190 lines of code (exceeds 25 allowed). Consider refactoring.
    Open

        def configure(conf)
      compat_parameters_convert(conf, :buffer)

      super
      if placeholder_substitution_needed_for_template?
    Severity: Major
    Found in lib/fluent/plugin/out_elasticsearch.rb - About 7 hrs to fix

      Class ElasticsearchOutput has 45 methods (exceeds 20 allowed). Consider refactoring.
      Open

        class ElasticsearchOutput < Output
    class RecoverableRequestFailure < StandardError; end
    class UnrecoverableRequestFailure < Fluent::UnrecoverableError; end
    class RetryStreamEmitFailure < StandardError; end
      Severity: Minor
      Found in lib/fluent/plugin/out_elasticsearch.rb - About 6 hrs to fix

        Method process_message has a Cognitive Complexity of 40 (exceeds 5 allowed). Consider refactoring.
        Open

            def process_message(tag, meta, header, time, record, affinity_target_indices, extracted_values)
      logstash_prefix, logstash_dateformat, index_name, type_name, _template_name, _customize_template, _deflector_alias, application_name, pipeline, _ilm_policy_id = extracted_values

      if @flatten_hashes
        record = flatten_record(record)
        Severity: Minor
        Found in lib/fluent/plugin/out_elasticsearch.rb - About 6 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method get_connection_options has a Cognitive Complexity of 29 (exceeds 5 allowed). Consider refactoring.
        Open

            def get_connection_options(con_host=nil)

      hosts = if con_host || @hosts
        (con_host || @hosts).split(',').map do |host_str|
          # Support legacy hosts format host:port,host:port,host:port...
        Severity: Minor
        Found in lib/fluent/plugin/out_elasticsearch.rb - About 4 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method write has a Cognitive Complexity of 28 (exceeds 5 allowed). Consider refactoring.
        Open

            def write(chunk)
      bulk_message_count = Hash.new { |h,k| h[k] = 0 }
      bulk_message = Hash.new { |h,k| h[k] = '' }
      header = {}
      meta = {}
        Severity: Minor
        Found in lib/fluent/plugin/out_elasticsearch.rb - About 4 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method send_bulk has a Cognitive Complexity of 27 (exceeds 5 allowed). Consider refactoring.
        Open

            def send_bulk(data, tag, chunk, bulk_message_count, extracted_values, info, unpacked_msg_arr)
      _logstash_prefix, _logstash_dateformat, index_name, _type_name, template_name, customize_template, deflector_alias, application_name, _pipeline, ilm_policy_id = extracted_values
      if deflector_alias
        template_installation(deflector_alias, template_name, customize_template, application_name, index_name, ilm_policy_id, info.host)
      else
        Severity: Minor
        Found in lib/fluent/plugin/out_elasticsearch.rb - About 3 hrs to fix

        Cognitive Complexity

        Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

        A method's cognitive complexity is based on a few simple rules:

        • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
        • Code is considered more complex for each "break in the linear flow of the code"
        • Code is considered more complex when "flow breaking structures are nested"

        Further reading

        Method process_message has 82 lines of code (exceeds 25 allowed). Consider refactoring.
        Open

            def process_message(tag, meta, header, time, record, affinity_target_indices, extracted_values)
      logstash_prefix, logstash_dateformat, index_name, type_name, _template_name, _customize_template, _deflector_alias, application_name, pipeline, _ilm_policy_id = extracted_values

      if @flatten_hashes
        record = flatten_record(record)
        Severity: Major
        Found in lib/fluent/plugin/out_elasticsearch.rb - About 3 hrs to fix

          Method template_installation_actual has a Cognitive Complexity of 22 (exceeds 5 allowed). Consider refactoring.
          Open

              def template_installation_actual(deflector_alias, template_name, customize_template, application_name, target_index, ilm_policy_id, host=nil)
      if template_name && @template_file
        if !@logstash_format && (deflector_alias.nil? || (@alias_indexes.include? deflector_alias)) && (@template_names.include? template_name)
          if deflector_alias
            log.debug("Index alias #{deflector_alias} and template #{template_name} already exist (cached)")
          Severity: Minor
          Found in lib/fluent/plugin/out_elasticsearch.rb - About 3 hrs to fix

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Method write has 53 lines of code (exceeds 25 allowed). Consider refactoring.
          Open

              def write(chunk)
      bulk_message_count = Hash.new { |h,k| h[k] = 0 }
      bulk_message = Hash.new { |h,k| h[k] = '' }
      header = {}
      meta = {}
          Severity: Major
          Found in lib/fluent/plugin/out_elasticsearch.rb - About 2 hrs to fix

            Method client has 45 lines of code (exceeds 25 allowed). Consider refactoring.
            Open

                def client(host = nil, compress_connection = false)
      return cloud_client if @cloud_id

      # check here to see if we already have a client connection for the given host
      connection_options = get_connection_options(host)
            Severity: Minor
            Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

              Method expand_placeholders has a Cognitive Complexity of 14 (exceeds 5 allowed). Consider refactoring.
              Open

                  def expand_placeholders(chunk)
      logstash_prefix = extract_placeholders(@logstash_prefix, chunk)
      logstash_dateformat = extract_placeholders(@logstash_dateformat, chunk)
      index_name = extract_placeholders(@index_name, chunk)
      if @type_name
              Severity: Minor
              Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              Method create_time_parser has a Cognitive Complexity of 14 (exceeds 5 allowed). Consider refactoring.
              Open

                  def create_time_parser
      if @time_key_format
        begin
          # Strptime doesn't support all formats, but for those it does it's
          # blazingly fast.
              Severity: Minor
              Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              Method get_connection_options has 43 lines of code (exceeds 25 allowed). Consider refactoring.
              Open

                  def get_connection_options(con_host=nil)

      hosts = if con_host || @hosts
        (con_host || @hosts).split(',').map do |host_str|
          # Support legacy hosts format host:port,host:port,host:port...
              Severity: Minor
              Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

                Method expand_placeholders has 39 lines of code (exceeds 25 allowed). Consider refactoring.
                Open

                    def expand_placeholders(chunk)
      logstash_prefix = extract_placeholders(@logstash_prefix, chunk)
      logstash_dateformat = extract_placeholders(@logstash_dateformat, chunk)
      index_name = extract_placeholders(@index_name, chunk)
      if @type_name
                Severity: Minor
                Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

                  Method send_bulk has 36 lines of code (exceeds 25 allowed). Consider refactoring.
                  Open

                      def send_bulk(data, tag, chunk, bulk_message_count, extracted_values, info, unpacked_msg_arr)
      _logstash_prefix, _logstash_dateformat, index_name, _type_name, template_name, customize_template, deflector_alias, application_name, _pipeline, ilm_policy_id = extracted_values
      if deflector_alias
        template_installation(deflector_alias, template_name, customize_template, application_name, index_name, ilm_policy_id, info.host)
      else
                  Severity: Minor
                  Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

                    Method get_affinity_target_indices has 28 lines of code (exceeds 25 allowed). Consider refactoring.
                    Open

                        def get_affinity_target_indices(chunk)
      indices = Hash.new
      if target_index_affinity_enabled?()
        id_key_accessor = record_accessor_create(@id_key)
        ids = Set.new
                    Severity: Minor
                    Found in lib/fluent/plugin/out_elasticsearch.rb - About 1 hr to fix

                      Method template_installation has 7 arguments (exceeds 4 allowed). Consider refactoring.
                      Open

                          def template_installation(deflector_alias, template_name, customize_template, application_name, ilm_policy_id, target_index, host)
                      Severity: Major
                      Found in lib/fluent/plugin/out_elasticsearch.rb - About 50 mins to fix

                        Method process_message has 7 arguments (exceeds 4 allowed). Consider refactoring.
                        Open

                            def process_message(tag, meta, header, time, record, affinity_target_indices, extracted_values)
                        Severity: Major
                        Found in lib/fluent/plugin/out_elasticsearch.rb - About 50 mins to fix

                          Method send_bulk has 7 arguments (exceeds 4 allowed). Consider refactoring.
                          Open

                              def send_bulk(data, tag, chunk, bulk_message_count, extracted_values, info, unpacked_msg_arr)
                          Severity: Major
                          Found in lib/fluent/plugin/out_elasticsearch.rb - About 50 mins to fix

                            Method template_installation_actual has 7 arguments (exceeds 4 allowed). Consider refactoring.
                            Open

                                def template_installation_actual(deflector_alias, template_name, customize_template, application_name, target_index, ilm_policy_id, host=nil)
                            Severity: Major
                            Found in lib/fluent/plugin/out_elasticsearch.rb - About 50 mins to fix

                              Method append_record_to_messages has 5 arguments (exceeds 4 allowed). Consider refactoring.
                              Open

                                  def append_record_to_messages(op, meta, header, record, msgs)
                              Severity: Minor
                              Found in lib/fluent/plugin/out_elasticsearch.rb - About 35 mins to fix

                                Method is_existing_connection has a Cognitive Complexity of 7 (exceeds 5 allowed). Consider refactoring.
                                Open

                                    def is_existing_connection(host)
      # check if the host provided match the current connection
      return false if @_es.nil?
      return false if @current_config.nil?
      return false if host.length != @current_config.length
                                Severity: Minor
                                Found in lib/fluent/plugin/out_elasticsearch.rb - About 35 mins to fix

                                Cognitive Complexity

                                Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                                A method's cognitive complexity is based on a few simple rules:

                                • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                                • Code is considered more complex for each "break in the linear flow of the code"
                                • Code is considered more complex when "flow breaking structures are nested"

                                Further reading

                                Avoid too many return statements within this method.
                                Open

                                      return true
                                Severity: Major
                                Found in lib/fluent/plugin/out_elasticsearch.rb - About 30 mins to fix

                                  Method client has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                                  Open

                                      def client(host = nil, compress_connection = false)
      return cloud_client if @cloud_id

      # check here to see if we already have a client connection for the given host
      connection_options = get_connection_options(host)
                                  Severity: Minor
                                  Found in lib/fluent/plugin/out_elasticsearch.rb - About 25 mins to fix

                                  Cognitive Complexity

                                  Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                                  A method's cognitive complexity is based on a few simple rules:

                                  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                                  • Code is considered more complex for each "break in the linear flow of the code"
                                  • Code is considered more complex when "flow breaking structures are nested"

                                  Further reading

                                  Method get_affinity_target_indices has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.
                                  Open

                                      def get_affinity_target_indices(chunk)
      indices = Hash.new
      if target_index_affinity_enabled?()
        id_key_accessor = record_accessor_create(@id_key)
        ids = Set.new
                                  Severity: Minor
                                  Found in lib/fluent/plugin/out_elasticsearch.rb - About 25 mins to fix

                                  Cognitive Complexity

                                  Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                                  A method's cognitive complexity is based on a few simple rules:

                                  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                                  • Code is considered more complex for each "break in the linear flow of the code"
                                  • Code is considered more complex when "flow breaking structures are nested"

                                  Further reading

                                  Identical blocks of code found in 2 locations. Consider refactoring.
                                  Open

                                      def is_existing_connection(host)
      # check if the host provided match the current connection
      return false if @_es.nil?
      return false if @current_config.nil?
      return false if host.length != @current_config.length
                                  Severity: Major
                                  Found in lib/fluent/plugin/out_elasticsearch.rb and 1 other location - About 1 hr to fix
                                  lib/fluent/plugin/in_elasticsearch.rb on lines 245..257

                                  Duplicated Code

                                  Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                                  Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                                  When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                                  Tuning

                                  This issue has a mass of 55.

                                  We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                                  The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                                  If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                                  See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                                  Refactorings

                                  Further Reading

                                  Similar blocks of code found in 2 locations. Consider refactoring.
                                  Open

                                      def parse_time(value, event_time, tag)
      @time_parser.call(value)
    rescue => e
      router.emit_error_event(@time_parse_error_tag, Fluent::Engine.now, {'tag' => tag, 'time' => event_time, 'format' => @time_key_format, 'value' => value}, e)
      return Time.at(event_time).to_datetime
                                  Severity: Minor
                                  Found in lib/fluent/plugin/out_elasticsearch.rb and 1 other location - About 25 mins to fix
                                  lib/fluent/plugin/in_elasticsearch.rb on lines 200..205

                                  Duplicated Code

                                  Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                                  Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                                  When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                                  Tuning

                                  This issue has a mass of 31.

                                  We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                                  The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                                  If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                                  See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                                  Refactorings

                                  Further Reading

                                  Identical blocks of code found in 2 locations. Consider refactoring.
                                  Open

                                      def get_escaped_userinfo(host_str)
      if m = host_str.match(/(?<scheme>.*)%{(?<user>.*)}:%{(?<password>.*)}(?<path>@.*)/)
        m["scheme"] +
          URI.encode_www_form_component(m["user"]) +
          ':' +
                                  Severity: Minor
                                  Found in lib/fluent/plugin/out_elasticsearch.rb and 1 other location - About 25 mins to fix
                                  lib/fluent/plugin/in_elasticsearch.rb on lines 115..123

                                  Duplicated Code

                                  Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                                  Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                                  When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                                  Tuning

                                  This issue has a mass of 29.

                                  We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                                  The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                                  If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                                  See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                                  Refactorings

                                  Further Reading

                                  Identical blocks of code found in 3 locations. Consider refactoring.
                                  Open

                                          host.merge!(user: @user, password: @password) if !host[:user] && @user
        host.merge!(path: @path) if !host[:path] && @path
      end
                                  Severity: Minor
                                  Found in lib/fluent/plugin/out_elasticsearch.rb and 2 other locations - About 20 mins to fix
                                  lib/fluent/plugin/in_elasticsearch.rb on lines 150..152
                                  lib/fluent/plugin/out_elasticsearch_dynamic.rb on lines 104..106

                                  Duplicated Code

                                  Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                                  Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                                  When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                                  Tuning

                                  This issue has a mass of 27.

                                  We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                                  The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                                  If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                                  See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                                  Refactorings

                                  Further Reading

                                  There are no issues that match your filters.

                                  Category
                                  Status