deeplearning4j/deeplearning4j

View on GitHub
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java

Summary

Maintainability
F
1 wk
Test Coverage

File OpValidation.java has 962 lines of code (exceeds 250 allowed). Consider refactoring.
Open

/*
 *  ******************************************************************************
 *  *
 *  *
 *  * This program and the accompanying materials are made available under the

    Method logCoverageInformation has a Cognitive Complexity of 82 (exceeds 5 allowed). Consider refactoring.
    Open

        public static void logCoverageInformation(boolean logAdequatelyTested, boolean logInadequate, boolean logUnmappedLibnd4jOps,
                                                  boolean logUntestedTFImport, boolean logUnmappedTFOps) {
            //Set of ops that we can't gradient check
            Set<Class> excludedFromBackpropCoverage = excludedFromGradientCheckCoverage();
            Set<Class> excludedFromAllTestCoverage = excludedFromAllTests();

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Method checkDeserializedEquality has a Cognitive Complexity of 62 (exceeds 5 allowed). Consider refactoring.
    Open

        public static void checkDeserializedEquality(SameDiff original, ByteBuffer bbSerialized, TestCase tc) {
            SameDiff deserialized;
            try{
                deserialized = SameDiff.fromFlatBuffers(bbSerialized);
            } catch (IOException e){

    Cognitive Complexity

    Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

    A method's cognitive complexity is based on a few simple rules:

    • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
    • Code is considered more complex for each "break in the linear flow of the code"
    • Code is considered more complex when "flow breaking structures are nested"

    Further reading

    Method logCoverageInformation has 145 lines of code (exceeds 25 allowed). Consider refactoring.
    Open

        public static void logCoverageInformation(boolean logAdequatelyTested, boolean logInadequate, boolean logUnmappedLibnd4jOps,
                                                  boolean logUntestedTFImport, boolean logUnmappedTFOps) {
            //Set of ops that we can't gradient check
            Set<Class> excludedFromBackpropCoverage = excludedFromGradientCheckCoverage();
            Set<Class> excludedFromAllTestCoverage = excludedFromAllTests();

      Method excludedFromAllTests has 137 lines of code (exceeds 25 allowed). Consider refactoring.
      Open

          private static Set<Class> excludedFromAllTests() {
              List list = Arrays.asList(
                      //Exclude misc
                      DynamicCustomOp.class,
                      GradientBackwardsMarker.class,

        Method checkDeserializedEquality has 111 lines of code (exceeds 25 allowed). Consider refactoring.
        Open

            public static void checkDeserializedEquality(SameDiff original, ByteBuffer bbSerialized, TestCase tc) {
                SameDiff deserialized;
                try{
                    deserialized = SameDiff.fromFlatBuffers(bbSerialized);
                } catch (IOException e){

          Method validateHelper has a Cognitive Complexity of 30 (exceeds 5 allowed). Consider refactoring.
          Open

              private static String validateHelper(TestCase testCase) {
                  testCase.assertConfigValid();
          
                  //First: collect coverage information
                  collectCoverageInformation(testCase);

          Cognitive Complexity

          Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

          A method's cognitive complexity is based on a few simple rules:

          • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
          • Code is considered more complex for each "break in the linear flow of the code"
          • Code is considered more complex when "flow breaking structures are nested"

          Further reading

          Method excludeFromTfImportCoverage has 95 lines of code (exceeds 25 allowed). Consider refactoring.
          Open

              private static Set<String> excludeFromTfImportCoverage(){
                  List<String> list = Arrays.asList(
                          "Reverse",      //Can be excluded because "Reverse_v2" is synonym that TF uses with tf.reverse(...); ReverseV2 is also Java op that is synonym for same op
                          "LogSigmoid",    //Not in ops.proto. Have tests for tf.log_sigmoid, but can't test LogSigmoid op directly: tf.log_sigmoid actually just uses "y = -tf.nn.softplus(-x)" - i.e., 3 separate ops :/
                          "HardSigmoid",   //Also implemented as python, NOT a single native op

            Method excludedFromGradientCheckCoverage has 76 lines of code (exceeds 25 allowed). Consider refactoring.
            Open

                private static Set<Class> excludedFromGradientCheckCoverage() {
                    List list = Arrays.asList(
                            //Exclude misc
                            DynamicCustomOp.class,
                            EqualsWithEps.class,

              Method initializeCoverage has a Cognitive Complexity of 21 (exceeds 5 allowed). Consider refactoring.
              Open

                  private static void initializeCoverage() {
                      //Scan classpath to find all DifferentialFunction instances, so tensorflow/onnx mappings can be made
                      //We're assuming here that all instances with such mappings are defined in ND4J
                      //As of 04/2018 all DifferentialFunction classes are defined in org.nd4j.linalg.api.ops - with the exception
                      // of ILossFunction instances, which don't have TF/Onnx import working anyway

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              Method collectCoverageInformation has a Cognitive Complexity of 21 (exceeds 5 allowed). Consider refactoring.
              Open

                  private static void collectCoverageInformation(TestCase testCase) {
                      SameDiff sd = testCase.sameDiff();
              
                      //NOTE: Count on a per-test-case basis, not on a 'per function seen' basis
                      //i.e., don't double count if a SameDiff instance has multiple copies of the same op type

              Cognitive Complexity

              Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

              A method's cognitive complexity is based on a few simple rules:

              • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
              • Code is considered more complex for each "break in the linear flow of the code"
              • Code is considered more complex when "flow breaking structures are nested"

              Further reading

              Method initializeCoverage has 65 lines of code (exceeds 25 allowed). Consider refactoring.
              Open

                  private static void initializeCoverage() {
                      //Scan classpath to find all DifferentialFunction instances, so tensorflow/onnx mappings can be made
                      //We're assuming here that all instances with such mappings are defined in ND4J
                      //As of 04/2018 all DifferentialFunction classes are defined in org.nd4j.linalg.api.ops - with the exception
                      // of ILossFunction instances, which don't have TF/Onnx import working anyway

                Method collectTensorflowImportCoverage has a Cognitive Complexity of 19 (exceeds 5 allowed). Consider refactoring.
                Open

                    public static void collectTensorflowImportCoverage(SameDiff graph){
                        for(SameDiffOp op : graph.getOps().values()){
                            DifferentialFunction d = op.getOp();
                            String[] tfNames = null;
                            try{

                Cognitive Complexity

                Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                A method's cognitive complexity is based on a few simple rules:

                • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                • Code is considered more complex for each "break in the linear flow of the code"
                • Code is considered more complex when "flow breaking structures are nested"

                Further reading

                Method validateHelper has 61 lines of code (exceeds 25 allowed). Consider refactoring.
                Open

                    private static String validateHelper(TestCase testCase) {
                        testCase.assertConfigValid();
                
                        //First: collect coverage information
                        collectCoverageInformation(testCase);

                  Method validate has a Cognitive Complexity of 13 (exceeds 5 allowed). Consider refactoring.
                  Open

                      public static String validate(OpTestCase testCase) {
                          collectCoverageInformation(testCase);
                  
                          //Check shape function:
                          List<LongShapeDescriptor> outShapes;

                  Cognitive Complexity

                  Cognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.

                  A method's cognitive complexity is based on a few simple rules:

                  • Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one
                  • Code is considered more complex for each "break in the linear flow of the code"
                  • Code is considered more complex when "flow breaking structures are nested"

                  Further reading

                  Method validate has 38 lines of code (exceeds 25 allowed). Consider refactoring.
                  Open

                      public static String validate(OpTestCase testCase) {
                          collectCoverageInformation(testCase);
                  
                          //Check shape function:
                          List<LongShapeDescriptor> outShapes;

                    Method collectCoverageInformation has 32 lines of code (exceeds 25 allowed). Consider refactoring.
                    Open

                        private static void collectCoverageInformation(TestCase testCase) {
                            SameDiff sd = testCase.sameDiff();
                    
                            //NOTE: Count on a per-test-case basis, not on a 'per function seen' basis
                            //i.e., don't double count if a SameDiff instance has multiple copies of the same op type

                      Method collectTensorflowImportCoverage has 28 lines of code (exceeds 25 allowed). Consider refactoring.
                      Open

                          public static void collectTensorflowImportCoverage(SameDiff graph){
                              for(SameDiffOp op : graph.getOps().values()){
                                  DifferentialFunction d = op.getOp();
                                  String[] tfNames = null;
                                  try{

                        Method logCoverageInformation has 5 arguments (exceeds 4 allowed). Consider refactoring.
                        Open

                            public static void logCoverageInformation(boolean logAdequatelyTested, boolean logInadequate, boolean logUnmappedLibnd4jOps,
                                                                      boolean logUntestedTFImport, boolean logUnmappedTFOps) {

                          Avoid too many return statements within this method.
                          Open

                                  return null;    //OK

                            Identical blocks of code found in 2 locations. Consider refactoring.
                            Open

                                            if (countFwdValidation > 0 && (countBackpropSeen > 0 || gradExcluded)) {
                                                //At least 1 forward test, and 1 gradient check
                            
                                                if (gradExcluded) {
                                                    log.info("Forward: {} tests, GradCheck: <excluded> for op {}", String.format(numFormat, countFwdValidation), c.getName());
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 689..698

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 58.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Identical blocks of code found in 2 locations. Consider refactoring.
                            Open

                                            if (countFwdValidation == 0 || (countBackpropSeen == 0 && !gradExcluded)) {
                                                //0 forward test OR 0 gradient check (and not excluded from grad checks)
                            
                                                if (gradExcluded) {
                                                    log.info("Forward: {} tests, GradCheck: <excluded> for op {}", String.format(numFormat, countFwdValidation), c.getName());
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 667..676

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 58.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Identical blocks of code found in 2 locations. Consider refactoring.
                            Open

                                    if(testCase.testFlatBufferSerialization() == TestCase.TestSerialization.BEFORE_EXEC || testCase.testFlatBufferSerialization() == TestCase.TestSerialization.BOTH){
                                        serializedBeforeExec = testCase.sameDiff().asFlatBuffers(true);
                                        Preconditions.checkNotNull(serializedBeforeExec, "Serialization failed? Null output");
                                    }
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 189..192

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 54.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Identical blocks of code found in 2 locations. Consider refactoring.
                            Open

                                        if(testCase.testFlatBufferSerialization() == TestCase.TestSerialization.BEFORE_EXEC || testCase.testFlatBufferSerialization() == TestCase.TestSerialization.BOTH){
                                            serializedAfterExec = testCase.sameDiff().asFlatBuffers(true);
                                            Preconditions.checkNotNull(serializedAfterExec, "Serialization failed? Null output");
                                        }
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 142..145

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 54.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Identical blocks of code found in 3 locations. Consider refactoring.
                            Open

                                    for (Class c : backpropSeen) {
                                        if(gradCheckCoverageCountPerClass.containsKey(c))
                                            gradCheckCoverageCountPerClass.put(c, gradCheckCoverageCountPerClass.get(c) + 1);
                                        else
                                            gradCheckCoverageCountPerClass.put(c, 1);
                            deeplearning4j/deeplearning4j-modelimport/src/main/java/org/deeplearning4j/nn/modelimport/keras/preprocessing/text/KerasTokenizer.java on lines 187..192
                            deeplearning4j/deeplearning4j-modelimport/src/main/java/org/deeplearning4j/nn/modelimport/keras/preprocessing/text/KerasTokenizer.java on lines 194..199

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 46.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Similar blocks of code found in 2 locations. Consider refactoring.
                            Open

                                        if(!Objects.equals(exp.dataType(), act.dataType())) {
                                            return "Shape function check failed for output " + i + ": expected shape " + exp + ", actual shape " + act;
                                        }
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 404..406

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 42.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            Similar blocks of code found in 2 locations. Consider refactoring.
                            Open

                                        if(!Arrays.equals(act.getShape(), exp.getShape())){
                                            return "Shape function check failed for output " + i + ": expected shape " + exp + ", actual shape " + act;
                                        }
                            nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/validation/OpValidation.java on lines 401..403

                            Duplicated Code

                            Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

                            Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

                            When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

                            Tuning

                            This issue has a mass of 42.

                            We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

                            The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

                            If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

                            See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

                            Refactorings

                            Further Reading

                            There are no issues that match your filters.

                            Category