Showing 248 of 248 total issues

Double quote to prevent globbing and word splitting.
Open

-v $VERSION \
Severity: Minor
Found in scripts/ssm-build-deb.sh by shellcheck

Double quote to prevent globbing and word splitting.

Problematic code:

echo $1
for i in $*; do :; done # this done and the next one also applies to expanding arrays.
for i in $@; do :; done

Correct code:

echo "$1"
for i in "$@"; do :; done # or, 'for i; do'

Rationale

The first code looks like "print the first argument". It's actually "Split the first argument by IFS (spaces, tabs and line feeds). Expand each of them as if it was a glob. Join all the resulting strings and filenames with spaces. Print the result."

The second one looks like "iterate through all arguments". It's actually "join all the arguments by the first character of IFS (space), split them by IFS and expand each of them as globs, and iterate on the resulting list". The third one skips the joining part.

Quoting variables prevents word splitting and glob expansion, and prevents the script from breaking when input contains spaces, line feeds, glob characters and such.

Strictly speaking, only expansions themselves need to be quoted, but for stylistic reasons, entire arguments with multiple variable and literal parts are often quoted as one:

$HOME/$dir/dist/bin/$file        # Unquoted (bad)
"$HOME"/"$dir"/dist/bin/"$file"  # Minimal quoting (good)
"$HOME/$dir/dist/bin/$file"      # Canonical quoting (good)

When quoting composite arguments, make sure to exclude globs and brace expansions, which lose their special meaning in double quotes: "$HOME/$dir/src/*.c" will not expand, but "$HOME/$dir/src"/*.c will.

Note that $( ) starts a new context, and variables in it have to be quoted independently:

echo "This $variable is quoted $(but this $variable is not)"
echo "This $variable is quoted $(and now this "$variable" is too)"

Exceptions

Sometimes you want to split on spaces, like when building a command line:

options="-j 5 -B"
make $options file

Just quoting this doesn't work. Instead, you should have used an array (bash, ksh, zsh):

options=(-j 5 -B) # ksh: set -A options -- -j 5 -B
make "${options[@]}" file

or a function (POSIX):

make_with_flags() { make -j 5 -B "$@"; }
make_with_flags file

To split on spaces but not perform glob expansion, Posix has a set -f to disable globbing. You can disable word splitting by setting IFS=''.

Similarly, you might want an optional argument:

debug=""
[[ $1 == "--trace-commands" ]] && debug="-x"
bash $debug script

Quoting this doesn't work, since in the default case, "$debug" would expand to one empty argument while $debug would expand into zero arguments. In this case, you can use an array with zero or one elements as outlined above, or you can use an unquoted expansion with an alternate value:

debug=""
[[ $1 == "--trace-commands" ]] && debug="yes"
bash ${debug:+"-x"} script

This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}.


As always, this warning can be [[ignore]]d on a case-by-case basis.

this is especially relevant when BASH many not be available for the array work around. For example, use in eval or in command options where script has total control of the variables...

FLAGS="-av -e 'ssh -x' --delete --delete-excluded"
...
# shellcheck disable=SC2086
eval rsync $FLAGS ~/dir remote_host:dir

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Inconsistent indentation for list items at the same level
Open

   * `gLite-APEL` for Grid Accounting
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD005 - Inconsistent indentation for list items at the same level

Tags: bullet, ul, indentation

Aliases: list-indent

This rule is triggered when list items are parsed as being at the same level, but don't have the same indentation:

* Item 1
    * Nested Item 1
    * Nested Item 2
   * A misaligned item

Usually this rule will be triggered because of a typo. Correct the indentation for the list to fix it:

* Item 1
  * Nested Item 1
  * Nested Item 2
  * Nested Item 3

Line length
Open

- The 'Host DN' listed in the GOCDB 'Service' entry must exactly match the certificate DN of the host used for accounting. Make sure there are no leading or trailing spaces in the 'Host DN' field.
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Line length
Open

2018-09-19 14:18:06,424 - ssmsend - INFO - No AMS token provided, using cert/key pair instead.
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Line too long (83 > 79 characters)
Open

        logging.warning('Separate logging config file option has been deprecated.')
Severity: Minor
Found in bin/receiver.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (80 > 79 characters)
Open

        log.warning("No option set for 'protocol'. Defaulting to %s.", protocol)
Severity: Minor
Found in ssm/agents.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Expected 2 blank lines, found 1
Open

def sign(text, certpath, keypath):
Severity: Minor
Found in ssm/crypto.py by pep8

Separate top-level function and class definitions with two blank lines.

Method definitions inside a class are separated by a single blank
line.

Extra blank lines may be used (sparingly) to separate groups of
related functions.  Blank lines may be omitted between a bunch of
related one-liners (e.g. a set of dummy implementations).

Use blank lines in functions, sparingly, to indicate logical
sections.

Okay: def a():\n    pass\n\n\ndef b():\n    pass
Okay: def a():\n    pass\n\n\nasync def b():\n    pass
Okay: def a():\n    pass\n\n\n# Foo\n# Bar\n\ndef b():\n    pass
Okay: default = 1\nfoo = 1
Okay: classify = 1\nfoo = 1

E301: class Foo:\n    b = 0\n    def bar():\n        pass
E302: def a():\n    pass\n\ndef b(n):\n    pass
E302: def a():\n    pass\n\nasync def b(n):\n    pass
E303: def a():\n    pass\n\n\n\ndef b(n):\n    pass
E303: def a():\n\n\n\n    pass
E304: @decorator\n\ndef a():\n    pass
E305: def a():\n    pass\na()
E306: def a():\n    def b():\n        pass\n    def c():\n        pass

In POSIX sh, $".." is undefined.
Open

    echo -n $"Stopping $prog: "
Severity: Minor
Found in bin/apel-ssm by shellcheck

In POSIX sh, something is undefined.

You have declared that your script works with /bin/sh, but you are using features that have undefined behavior according to the POSIX specification.

It may currently work for you, but it can or will fail on other OS, the same OS with different configurations, from different contexts (like initramfs/chroot), or in different versions of the same OS, including future updates to your current system.

Either declare that your script requires a specific shell like #!/bin/bash or #!/bin/dash, or rewrite the script in a portable way.

For help with rewrites, the Ubuntu wiki has a list of portability issues that broke people's #!/bin/sh scripts when Ubuntu switched from Bash to Dash. See also Bashism on wooledge's wiki. ShellCheck may not warn about all these issues.

$'c-style-escapes'

bash, ksh:

a=$' \t\n'

POSIX:

a="$(printf '%b_' ' \t\n')"; a="${a%_}" # protect trailing \n

Want some good news? See http://austingroupbugs.net/view.php?id=249#c590.

$"msgid"

Bash:

echo $"foo $(bar) baz"

POSIX:

. gettext.sh # GNU Gettext sh library
# ...
barout=$(bar)
eval_gettext 'foo $barout baz' # See GNU Gettext doc for more info.

Or you can change them to normal double quotes so you go without gettext.

Arithmetic for loops

Bash:

for ((init; test; next)); do foo; done

POSIX:

: $((init))
while [ $((test)) -ne 0 ]; do foo; : $((next)); done

Arithmetic exponentiation

Bash:

printf "%s\n" "$(( 2**63 ))"

POSIX:

The POSIX standard does not allow for exponents. However, you can replicate them completely built-in using a POSIX compatible function. As an example, the pow function from here.

pow () {
    set "$1" "$2" 1
    while [ "$2" -gt 0 ]; do
      set "$1" $(($2-1)) $(($1*$3))
    done
    # %d = signed decimal, %u = unsigned decimal
    # Either should overflow to 0
    printf "%d\n" "$3"
}

To compare:

$ echo "$(( 2**62 ))"
4611686018427387904
$ pow 2 62
4611686018427387904

Alternatively, if you don't mind using an external program, you can use bc. Be aware though: bash and other programs may abide by a certain maximum integer that bc does not (for bash that's: 64-bit signed long int, failing back to 32-bit signed long int).

Example:

# Note the overflow that gives a negative number
$ echo "$(( 2**63 ))"
-9223372036854775808

# No such problem
$ echo 2^63 | bc
9223372036854775808

# 'bc' just keeps on going
$ echo 2^1280 | bc
20815864389328798163850480654728171077230524494533409610638224700807\
21611934672059602447888346464836968484322790856201558276713249664692\
98162798132113546415258482590187784406915463666993231671009459188410\
95379622423387354295096957733925002768876520583464697770622321657076\
83317005651120933244966378183760369413644440628104205339687097746591\
6057756101739472373801429441421111406337458176

standalone ((..))

Bash:

((a=c+d))
((d)) && echo d is true.

POSIX:

: $((a=c+d)) # discard the output of the arith expn with `:` command
[ $((d)) -ne 0 ] && echo d is true. # manually check non-zero => true

select loops

It takes extra care over terminal columns to make select loop look like bash's, which generates a list with multiple items on one line, or like ls.

It is, however, still possible to make a naive translation for select foo in bar baz; do eat; done:

while
  _i=0 _foo= foo=
  for _name in bar baz; do echo "$((_i+=1))) $_name"; done
  printf '$# '; read _foo
do
  case _foo in 1) foo=bar;; 2) foo=baz;; *) continue;; esac
  eat
done

Here-strings

Bash, ksh:

grep aaa <<< "$g"

POSIX:

# not exactly the same -- <<< adds a trailing \n if $g doesn't end with \n
printf '%s' "$g" | grep aaa

echo flags

See https://unix.stackexchange.com/tags/echo/info.

${var/pat/replacement}

Bash:

echo "${TERM/%-256*}"

POSIX:

echo "$TERM" | sed -e 's/-256.*$//g'
# Special case for this since we are matching the end:
echo "${TERM%-256*}"

printf %q

Bash:

printf '%q ' "$@"

POSIX:

# TODO: Interpret it back to printf escapes for hard-to-copy chars like \t?
# See also: http://git.savannah.gnu.org/cgit/libtool.git/tree/gl/build-aux/funclib.sh?id=c60e054#n1029
reuse_quote()(
  for i; do
    __i_quote=$(printf '%s\n' "$i" | sed -e "s/'/'\\\\''/g"; echo x)
    printf "'%s'" "${__i_quote%x}"
  done
)
reuse_quote "$@"

Exception

Depends on what your expected POSIX shell providers would use.

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Line length
Open

2018-09-19 14:18:06,424 - ssmsend - INFO - No server certificate supplied.  Will not encrypt messages.
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Line length
Open

- Grid sites using the APEL accounting client need to be using APEL 1.9.0 and SSM 3.2.0. They can then modify their APEL client script, usually located at `/usr/bin/apelclient`. At the moment, this requires a manual change, but will become a configuration option in the next version of APEL. For example, to halve the number of records per message from the default of 1000, add the line `unloader.records_per_message = 500` after the call to `DbUnloader`:
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Fenced code blocks should be surrounded by blank lines
Open

   ```
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD031 - Fenced code blocks should be surrounded by blank lines

Tags: code, blank_lines

Aliases: blanks-around-fences

This rule is triggered when fenced code blocks are either not preceded or not followed by a blank line:

Some text
```
Code block
```

```
Another code block
```
Some more text

To fix this, ensure that all fenced code blocks have a blank line both before and after (except where the block is at the beginning or end of the document):

Some text

```
Code block
```

```
Another code block
```

Some more text

Rationale: Aside from aesthetic reasons, some parsers, including kramdown, will not parse fenced code blocks that don't have blank lines before and after them.

Line too long (80 > 79 characters)
Open

                            help='DEPRECATED - location of logging config file',
Severity: Minor
Found in bin/receiver.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (86 > 79 characters)
Open

    # Using the vars function to output a dict-like view rather than Namespace object.
Severity: Minor
Found in bin/receiver.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Continuation line over-indented for visual indent
Open

                        pkg_resources.get_distribution('argo_ams_library').version)
Severity: Minor
Found in ssm/ssm2.py by pep8

Continuation lines indentation.

Continuation lines should align wrapped elements either vertically
using Python's implicit line joining inside parentheses, brackets
and braces, or using a hanging indent.

When using a hanging indent these considerations should be applied:
- there should be no arguments on the first line, and
- further indentation should be used to clearly distinguish itself
  as a continuation line.

Okay: a = (\n)
E123: a = (\n    )

Okay: a = (\n    42)
E121: a = (\n   42)
E122: a = (\n42)
E123: a = (\n    42\n    )
E124: a = (24,\n     42\n)
E125: if (\n    b):\n    pass
E126: a = (\n        42)
E127: a = (24,\n      42)
E128: a = (24,\n    42)
E129: if (a or\n    b):\n    pass
E131: a = (\n    42\n 24)

Double quote to prevent globbing and word splitting.
Open

--iteration $ITERATION \
Severity: Minor
Found in scripts/ssm-build-deb.sh by shellcheck

Double quote to prevent globbing and word splitting.

Problematic code:

echo $1
for i in $*; do :; done # this done and the next one also applies to expanding arrays.
for i in $@; do :; done

Correct code:

echo "$1"
for i in "$@"; do :; done # or, 'for i; do'

Rationale

The first code looks like "print the first argument". It's actually "Split the first argument by IFS (spaces, tabs and line feeds). Expand each of them as if it was a glob. Join all the resulting strings and filenames with spaces. Print the result."

The second one looks like "iterate through all arguments". It's actually "join all the arguments by the first character of IFS (space), split them by IFS and expand each of them as globs, and iterate on the resulting list". The third one skips the joining part.

Quoting variables prevents word splitting and glob expansion, and prevents the script from breaking when input contains spaces, line feeds, glob characters and such.

Strictly speaking, only expansions themselves need to be quoted, but for stylistic reasons, entire arguments with multiple variable and literal parts are often quoted as one:

$HOME/$dir/dist/bin/$file        # Unquoted (bad)
"$HOME"/"$dir"/dist/bin/"$file"  # Minimal quoting (good)
"$HOME/$dir/dist/bin/$file"      # Canonical quoting (good)

When quoting composite arguments, make sure to exclude globs and brace expansions, which lose their special meaning in double quotes: "$HOME/$dir/src/*.c" will not expand, but "$HOME/$dir/src"/*.c will.

Note that $( ) starts a new context, and variables in it have to be quoted independently:

echo "This $variable is quoted $(but this $variable is not)"
echo "This $variable is quoted $(and now this "$variable" is too)"

Exceptions

Sometimes you want to split on spaces, like when building a command line:

options="-j 5 -B"
make $options file

Just quoting this doesn't work. Instead, you should have used an array (bash, ksh, zsh):

options=(-j 5 -B) # ksh: set -A options -- -j 5 -B
make "${options[@]}" file

or a function (POSIX):

make_with_flags() { make -j 5 -B "$@"; }
make_with_flags file

To split on spaces but not perform glob expansion, Posix has a set -f to disable globbing. You can disable word splitting by setting IFS=''.

Similarly, you might want an optional argument:

debug=""
[[ $1 == "--trace-commands" ]] && debug="-x"
bash $debug script

Quoting this doesn't work, since in the default case, "$debug" would expand to one empty argument while $debug would expand into zero arguments. In this case, you can use an array with zero or one elements as outlined above, or you can use an unquoted expansion with an alternate value:

debug=""
[[ $1 == "--trace-commands" ]] && debug="yes"
bash ${debug:+"-x"} script

This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}.


As always, this warning can be [[ignore]]d on a case-by-case basis.

this is especially relevant when BASH many not be available for the array work around. For example, use in eval or in command options where script has total control of the variables...

FLAGS="-av -e 'ssh -x' --delete --delete-excluded"
...
# shellcheck disable=SC2086
eval rsync $FLAGS ~/dir remote_host:dir

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Line length
Open

1. To send to the central APEL Accounting server, change `destination` to one of the following depending on your type of accounting:
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Expected 2 blank lines, found 1
Open

def _get_subject_components(subject_x509name):
Severity: Minor
Found in ssm/crypto.py by pep8

Separate top-level function and class definitions with two blank lines.

Method definitions inside a class are separated by a single blank
line.

Extra blank lines may be used (sparingly) to separate groups of
related functions.  Blank lines may be omitted between a bunch of
related one-liners (e.g. a set of dummy implementations).

Use blank lines in functions, sparingly, to indicate logical
sections.

Okay: def a():\n    pass\n\n\ndef b():\n    pass
Okay: def a():\n    pass\n\n\nasync def b():\n    pass
Okay: def a():\n    pass\n\n\n# Foo\n# Bar\n\ndef b():\n    pass
Okay: default = 1\nfoo = 1
Okay: classify = 1\nfoo = 1

E301: class Foo:\n    b = 0\n    def bar():\n        pass
E302: def a():\n    pass\n\ndef b(n):\n    pass
E302: def a():\n    pass\n\nasync def b(n):\n    pass
E303: def a():\n    pass\n\n\n\ndef b(n):\n    pass
E303: def a():\n\n\n\n    pass
E304: @decorator\n\ndef a():\n    pass
E305: def a():\n    pass\na()
E306: def a():\n    def b():\n        pass\n    def c():\n        pass

Line length
Open

1. Follow the steps 1 to 4 as per the [Sender documentation](#Sender) but editing your receiver configuration instead, usually found under `/etc/apel/receiver.cfg`, naming the section `[receiver]` rather than `[sender]`.
Severity: Info
Found in migrating_to_ams.md by markdownlint

MD013 - Line length

Tags: line_length

Aliases: line-length Parameters: linelength, codeblocks, tables (number; default 80, boolean; default true)

This rule is triggered when there are lines that are longer than the configured line length (default: 80 characters). To fix this, split the line up into multiple lines.

This rule has an exception where there is no whitespace beyond the configured line length. This allows you to still include items such as long URLs without being forced to break them in the middle.

You also have the option to exclude this rule for code blocks and tables. To do this, set the code_blocks and/or tables parameters to false.

Code blocks are included in this rule by default since it is often a requirement for document readability, and tentatively compatible with code rules. Still, some languages do not lend themselves to short lines.

Line too long (85 > 79 characters)
Open

                # If option not set, resort to value of verify_server_cert set above.
Severity: Minor
Found in ssm/agents.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.

Line too long (85 > 79 characters)
Open

                "server certificate is the same as host certificate in config file. "
Severity: Minor
Found in ssm/agents.py by pep8

Limit all lines to a maximum of 79 characters.

There are still many devices around that are limited to 80 character
lines; plus, limiting windows to 80 characters makes it possible to
have several windows side-by-side.  The default wrapping on such
devices looks ugly.  Therefore, please limit all lines to a maximum
of 79 characters. For flowing long blocks of text (docstrings or
comments), limiting the length to 72 characters is recommended.

Reports error E501.
Severity
Category
Status
Source
Language