Sign upLogin

fabiocicerchia/salmonjs

View on GitHub
Star

Showing 88 of 88 total issues

Function analiseRedisResponse has 45 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.analiseRedisResponse = function (err, reply, redisId, container) {
        var id               = redisId.substr(0, 8),
            winstonCrawlerId = '[' + id.cyan + '-' + currentCrawler.idCrawler.magenta + ']',
            newId;
Severity: Minor
Found in src/crawler.js - About 1 hr to fix

Function execSubProcess has 45 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.execSubProcess = function () {
        var idRequest = utils.sha1(this.url + this.type + JSON.stringify(this.data) + this.evt + this.xPath),
            subprocess,
            params  = {
                idCrawler:       this.idUri,
Severity: Minor
Found in src/crawler.js - About 1 hr to fix

Function processQueue has 44 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.processQueue = function () {
        if (currentPool.queue.length === 0) {
            return;
        }
Severity: Minor
Found in src/pool.js - About 1 hr to fix

Function handleError has 41 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.handleError = function () {
        var winstonCrawlerId = '[' + currentCrawler.idUri.cyan + '-' + this.idCrawler.magenta + ']';

        if (currentCrawler.tries < config.crawler.attempts) {
            winston.info('%s' + ' Trying again in %s msec'.grey, winstonCrawlerId, config.crawler.delay);
Severity: Minor
Found in src/crawler.js - About 1 hr to fix

Function parsePost has 40 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.parsePost = function () {
        this.handleQueryString();

        this.setUpPage(this.page);
Severity: Minor
Found in src/parser/phantom.js - About 1 hr to fix

Function checkAndRun has 37 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.checkAndRun = function (settings) {
        var container   = {},
            redisId,
            id,
            winstonCrawlerId;
Severity: Minor
Found in src/crawler.js - About 1 hr to fix

Identical blocks of code found in 2 locations. Consider refactoring.
Open

        for (header in this.data.HEADERS) {
            if (this.data.HEADERS.hasOwnProperty(header)) {
                headers[header] = this.data.HEADERS[header];
            }
        }
Severity: Major
Found in src/parser/phantom.js and 1 other location - About 1 hr to fix
src/parser/phantom.js on lines 276..280

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 63.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Identical blocks of code found in 2 locations. Consider refactoring.
Open

        for (header in this.data.HEADERS) {
            if (this.data.HEADERS.hasOwnProperty(header)) {
                headers[header] = this.data.HEADERS[header];
            }
        }
Severity: Major
Found in src/parser/phantom.js and 1 other location - About 1 hr to fix
src/parser/phantom.js on lines 229..233

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 63.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Function dump has 33 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.dump = function (callback) {
        var dump = {conf: conf, redis: {}, pool: {}};

        dump.pool.size            = pool.size;
        dump.pool.queue           = pool.queue;
Severity: Minor
Found in src/session.js - About 1 hr to fix

Function parseGet has 29 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.parseGet = function () {
        this.handleQueryString();

        // InitPhantomJs
        this.setUpPage(this.page);
Severity: Minor
Found in src/parser/phantom.js - About 1 hr to fix

Function start has 29 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.start = function() {
        var uri = currentInstance.resolveURI(argv.uri);

        winston.info('Start processing "' + uri.green + '"...');
Severity: Minor
Found in src/main.js - About 1 hr to fix

Similar blocks of code found in 4 locations. Consider refactoring.
Open

    this.readFileSync = function (filename) {
        var method = (fs.readFileSync !== undefined) ? 'readFileSync' : 'read';
        return fs[method].call(fs, filename);
    };
Severity: Major
Found in src/fs.js and 3 other locations - About 1 hr to fix
src/fs.js on lines 104..107
src/fs.js on lines 134..137
src/fs.js on lines 147..150

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 58.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Similar blocks of code found in 4 locations. Consider refactoring.
Open

    this.unlinkSync = function (path) {
        var method = (fs.unlinkSync !== undefined) ? 'unlinkSync' : 'remove';
        return fs[method].call(fs, path);
    };
Severity: Major
Found in src/fs.js and 3 other locations - About 1 hr to fix
src/fs.js on lines 53..56
src/fs.js on lines 104..107
src/fs.js on lines 134..137

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 58.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Similar blocks of code found in 4 locations. Consider refactoring.
Open

    this.readdirSync = function (path) {
        var method = (fs.readdirSync !== undefined) ? 'readdirSync' : 'list';
        return fs[method].call(fs, path);
    };
Severity: Major
Found in src/fs.js and 3 other locations - About 1 hr to fix
src/fs.js on lines 53..56
src/fs.js on lines 104..107
src/fs.js on lines 147..150

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 58.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Similar blocks of code found in 4 locations. Consider refactoring.
Open

    this.existsSync = function (filename) {
        var method = (fs.existsSync !== undefined) ? 'existsSync' : 'exists';
        return fs[method].call(fs, filename);
    };
Severity: Major
Found in src/fs.js and 3 other locations - About 1 hr to fix
src/fs.js on lines 53..56
src/fs.js on lines 134..137
src/fs.js on lines 147..150

Duplicated Code

Duplicated code can lead to software that is hard to understand and difficult to change. The Don't Repeat Yourself (DRY) principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

When you violate DRY, bugs and maintenance problems are sure to follow. Duplicated code has a tendency to both continue to replicate and also to diverge (leaving bugs as two similar implementations differ in subtle ways).

Tuning

This issue has a mass of 58.

We set useful threshold defaults for the languages we support but you may want to adjust these settings based on your project guidelines.

The threshold configuration represents the minimum mass a code block must have to be analyzed for duplication. The lower the threshold, the more fine-grained the comparison.

If the engine is too easily reporting duplication, try raising the threshold. If you suspect that the engine isn't catching enough duplication, try lowering the threshold. The best setting tends to differ from language to language.

See codeclimate-duplication's documentation for more information about tuning the mass threshold in your .codeclimate.yml.

Refactorings

Further Reading

Function run has 27 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.run = function (settings) {
        this.url   = settings.url;
        this.type  = settings.type || 'GET';
        this.data  = settings.data || {
            GET:     {},
Severity: Minor
Found in src/crawler.js - About 1 hr to fix

Function onEvaluateNonHtml has 27 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.onEvaluateNonHtml = function () {
        var urls          = {
                mixed_full: [],
                mixed_rel:  [],
            },
Severity: Minor
Found in src/parser/phantom.js - About 1 hr to fix

Function list has 26 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    list: function (dir, done) {
        var FSWrapper = require('./fs'),
            fsWrapper = new FSWrapper(),
            results   = [],
            list      = fsWrapper.readdirSync(dir),
Severity: Minor
Found in src/glob.js - About 1 hr to fix

Function onLoadFinished has 26 lines of code (exceeds 25 allowed). Consider refactoring.
Open

    this.onLoadFinished = function () {
        if (settings.sanitise !== undefined && settings.sanitise.toString() === 'true') {
            var tmp_fn  = fs.workingDirectory + '/file_' + ((new Date()).getTime()) + '.html',
                args    = [ fs.workingDirectory + '/src/tidy.js', tmp_fn ],
                process;
Severity: Minor
Found in src/parser/phantom.js - About 1 hr to fix

Function Crawler has 8 arguments (exceeds 4 allowed). Consider refactoring.
Open

var Crawler = function (config, spawn, test, client, winston, fs, optimist, utils) {
Severity: Major
Found in src/crawler.js - About 1 hr to fix
Severity
Category
Status
Source
Language