sjsucohort6/amigo-chatbot

View on GitHub
deploy/run_local.sh

Summary

Maintainability
Test Coverage

Quote this to prevent word splitting.
Open

eval $(docker-machine env default)
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Quote this to prevent word splitting

Problematic code:

ls -l $(getfilename)

Correct code:

# getfilename outputs 1 file
ls -l "$(getfilename)"

# getfilename outputs multiple files, linefeed separated
getfilename | while IFS='' read -r line
do
  ls -l "$line"
done

Rationale:

When command expansions are unquoted, word splitting and globbing will occur. This often manifests itself by breaking when filenames contain spaces.

Trying to fix it by adding quotes or escapes to the data will not work. Instead, quote the command substitution itself.

If the command substitution outputs multiple pieces of data, use a loop instead.

Exceptions

In rare cases you actually want word splitting, such as in

gcc $(pkg-config --libs openssl) client.c

This is because pkg-config outputs -lssl -lcrypto, which you want to break up by spaces into -lssl and -lcrypto. An alternative is to put the variables to an array and expand it:

args=( $(pkg-config --libs openssl) )
gcc "${args[@]}" client.c

The power of using an array becomes evident when you want to combine, for example, the command result with user-provided arguments:

compile () {
    args=( $(pkg-config --libs openssl) "${@}" )
    gcc "${args[@]}" client.c
}
compile -DDEBUG
+ gcc -lssl -lcrypto -DDEBUG client.c

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Double quote to prevent globbing and word splitting.
Open

echo $BUILD_IMAGES
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Double quote to prevent globbing and word splitting.

Problematic code:

echo $1
for i in $*; do :; done # this done and the next one also applies to expanding arrays.
for i in $@; do :; done

Correct code:

echo "$1"
for i in "$@"; do :; done # or, 'for i; do'

Rationale

The first code looks like "print the first argument". It's actually "Split the first argument by IFS (spaces, tabs and line feeds). Expand each of them as if it was a glob. Join all the resulting strings and filenames with spaces. Print the result."

The second one looks like "iterate through all arguments". It's actually "join all the arguments by the first character of IFS (space), split them by IFS and expand each of them as globs, and iterate on the resulting list". The third one skips the joining part.

Quoting variables prevents word splitting and glob expansion, and prevents the script from breaking when input contains spaces, line feeds, glob characters and such.

Strictly speaking, only expansions themselves need to be quoted, but for stylistic reasons, entire arguments with multiple variable and literal parts are often quoted as one:

$HOME/$dir/dist/bin/$file        # Unquoted (bad)
"$HOME"/"$dir"/dist/bin/"$file"  # Minimal quoting (good)
"$HOME/$dir/dist/bin/$file"      # Canonical quoting (good)

When quoting composite arguments, make sure to exclude globs and brace expansions, which lose their special meaning in double quotes: "$HOME/$dir/src/*.c" will not expand, but "$HOME/$dir/src"/*.c will.

Note that $( ) starts a new context, and variables in it have to be quoted independently:

echo "This $variable is quoted $(but this $variable is not)"
echo "This $variable is quoted $(and now this "$variable" is too)"

Exceptions

Sometimes you want to split on spaces, like when building a command line:

options="-j 5 -B"
make $options file

Just quoting this doesn't work. Instead, you should have used an array (bash, ksh, zsh):

options=(-j 5 -B) # ksh: set -A options -- -j 5 -B
make "${options[@]}" file

or a function (POSIX):

make_with_flags() { make -j 5 -B "$@"; }
make_with_flags file

To split on spaces but not perform glob expansion, Posix has a set -f to disable globbing. You can disable word splitting by setting IFS=''.

Similarly, you might want an optional argument:

debug=""
[[ $1 == "--trace-commands" ]] && debug="-x"
bash $debug script

Quoting this doesn't work, since in the default case, "$debug" would expand to one empty argument while $debug would expand into zero arguments. In this case, you can use an array with zero or one elements as outlined above, or you can use an unquoted expansion with an alternate value:

debug=""
[[ $1 == "--trace-commands" ]] && debug="yes"
bash ${debug:+"-x"} script

This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}.


As always, this warning can be [[ignore]]d on a case-by-case basis.

this is especially relevant when BASH many not be available for the array work around. For example, use in eval or in command options where script has total control of the variables...

FLAGS="-av -e 'ssh -x' --delete --delete-excluded"
...
# shellcheck disable=SC2086
eval rsync $FLAGS ~/dir remote_host:dir

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Double quote to prevent globbing and word splitting.
Open

if [ -z $BUILD_IMAGES ]; then
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Double quote to prevent globbing and word splitting.

Problematic code:

echo $1
for i in $*; do :; done # this done and the next one also applies to expanding arrays.
for i in $@; do :; done

Correct code:

echo "$1"
for i in "$@"; do :; done # or, 'for i; do'

Rationale

The first code looks like "print the first argument". It's actually "Split the first argument by IFS (spaces, tabs and line feeds). Expand each of them as if it was a glob. Join all the resulting strings and filenames with spaces. Print the result."

The second one looks like "iterate through all arguments". It's actually "join all the arguments by the first character of IFS (space), split them by IFS and expand each of them as globs, and iterate on the resulting list". The third one skips the joining part.

Quoting variables prevents word splitting and glob expansion, and prevents the script from breaking when input contains spaces, line feeds, glob characters and such.

Strictly speaking, only expansions themselves need to be quoted, but for stylistic reasons, entire arguments with multiple variable and literal parts are often quoted as one:

$HOME/$dir/dist/bin/$file        # Unquoted (bad)
"$HOME"/"$dir"/dist/bin/"$file"  # Minimal quoting (good)
"$HOME/$dir/dist/bin/$file"      # Canonical quoting (good)

When quoting composite arguments, make sure to exclude globs and brace expansions, which lose their special meaning in double quotes: "$HOME/$dir/src/*.c" will not expand, but "$HOME/$dir/src"/*.c will.

Note that $( ) starts a new context, and variables in it have to be quoted independently:

echo "This $variable is quoted $(but this $variable is not)"
echo "This $variable is quoted $(and now this "$variable" is too)"

Exceptions

Sometimes you want to split on spaces, like when building a command line:

options="-j 5 -B"
make $options file

Just quoting this doesn't work. Instead, you should have used an array (bash, ksh, zsh):

options=(-j 5 -B) # ksh: set -A options -- -j 5 -B
make "${options[@]}" file

or a function (POSIX):

make_with_flags() { make -j 5 -B "$@"; }
make_with_flags file

To split on spaces but not perform glob expansion, Posix has a set -f to disable globbing. You can disable word splitting by setting IFS=''.

Similarly, you might want an optional argument:

debug=""
[[ $1 == "--trace-commands" ]] && debug="-x"
bash $debug script

Quoting this doesn't work, since in the default case, "$debug" would expand to one empty argument while $debug would expand into zero arguments. In this case, you can use an array with zero or one elements as outlined above, or you can use an unquoted expansion with an alternate value:

debug=""
[[ $1 == "--trace-commands" ]] && debug="yes"
bash ${debug:+"-x"} script

This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}.


As always, this warning can be [[ignore]]d on a case-by-case basis.

this is especially relevant when BASH many not be available for the array work around. For example, use in eval or in command options where script has total control of the variables...

FLAGS="-av -e 'ssh -x' --delete --delete-excluded"
...
# shellcheck disable=SC2086
eval rsync $FLAGS ~/dir remote_host:dir

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

cd ../docker-images/kafka
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

    cd ../user-service
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Not following: ./build.sh: openFile: does not exist (No such file or directory)
Open

    source ./build.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Not following: (error message here)

Reasons include: file not found, no permissions, not included on the command line, not allowing shellcheck to follow files with -x, etc.

Problematic code:

source somefile

Correct code:

# shellcheck disable=SC1091
source somefile

Rationale:

ShellCheck, for whichever reason, is not able to access the source file.

This could be because you did not include it on the command line, did not use shellcheck -x to allow following other files, don't have permissions or a variety of other problems.

Feel free to ignore the error with a [[directive]].

Exceptions:

If you're fine with it, ignore the message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Not following: ./build.sh: openFile: does not exist (No such file or directory)
Open

    source ./build.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Not following: (error message here)

Reasons include: file not found, no permissions, not included on the command line, not allowing shellcheck to follow files with -x, etc.

Problematic code:

source somefile

Correct code:

# shellcheck disable=SC1091
source somefile

Rationale:

ShellCheck, for whichever reason, is not able to access the source file.

This could be because you did not include it on the command line, did not use shellcheck -x to allow following other files, don't have permissions or a variety of other problems.

Feel free to ignore the error with a [[directive]].

Exceptions:

If you're fine with it, ignore the message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

read without -r will mangle backslashes.
Open

read BUILD_IMAGES
Severity: Minor
Found in deploy/run_local.sh by shellcheck

read without -r mangle backslashes

Problematic code:

echo "Enter name:"
read name

Correct code:

echo "Enter name:"
read -r name

Rationale:

By default, read will interpret backslashes before spaces and line feeds, and otherwise strip them. This is rarely expected or desired.

Normally you just want to read data, which is what read -r does. You should always use -r unless you have a good reason not to.

Note that read -r will still strip leading and trailing spaces. IFS="" read -r prevents this.

Exceptions:

If you want backslashes to affect field splitting and line terminators instead of being read, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

    cd ../command-processor-service
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Can't follow non-constant source. Use a directive to specify location.
Open

source ~/env.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Can't follow non-constant source. Use a directive to specify location.

Problematic code:

. "$(find_install_dir)/lib.sh"

Correct code:

# shellcheck source=src/lib.sh
. "$(find_install_dir)/lib.sh"

Rationale:

ShellCheck is not able to include sourced files from paths that are determined at runtime. The file will not be read, potentially resulting in warnings about unassigned variables and similar.

Use a [[Directive]] to point shellcheck to a fixed location it can read instead.

Exceptions:

If you don't care that ShellCheck is unable to account for the file, specify # shellcheck source=/dev/null.

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

    cd ../slackbot-service
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Not following: ./build.sh: openFile: does not exist (No such file or directory)
Open

    source ./build.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Not following: (error message here)

Reasons include: file not found, no permissions, not included on the command line, not allowing shellcheck to follow files with -x, etc.

Problematic code:

source somefile

Correct code:

# shellcheck disable=SC1091
source somefile

Rationale:

ShellCheck, for whichever reason, is not able to access the source file.

This could be because you did not include it on the command line, did not use shellcheck -x to allow following other files, don't have permissions or a variety of other problems.

Feel free to ignore the error with a [[directive]].

Exceptions:

If you're fine with it, ignore the message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

    cd ../riabot-service
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Not following: ./build.sh: openFile: does not exist (No such file or directory)
Open

    source ./build.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Not following: (error message here)

Reasons include: file not found, no permissions, not included on the command line, not allowing shellcheck to follow files with -x, etc.

Problematic code:

source somefile

Correct code:

# shellcheck disable=SC1091
source somefile

Rationale:

ShellCheck, for whichever reason, is not able to access the source file.

This could be because you did not include it on the command line, did not use shellcheck -x to allow following other files, don't have permissions or a variety of other problems.

Feel free to ignore the error with a [[directive]].

Exceptions:

If you're fine with it, ignore the message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Not following: ./build.sh: openFile: does not exist (No such file or directory)
Open

    source ./build.sh
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Not following: (error message here)

Reasons include: file not found, no permissions, not included on the command line, not allowing shellcheck to follow files with -x, etc.

Problematic code:

source somefile

Correct code:

# shellcheck disable=SC1091
source somefile

Rationale:

ShellCheck, for whichever reason, is not able to access the source file.

This could be because you did not include it on the command line, did not use shellcheck -x to allow following other files, don't have permissions or a variety of other problems.

Feel free to ignore the error with a [[directive]].

Exceptions:

If you're fine with it, ignore the message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Open

    cd ../chatbot-service
Severity: Minor
Found in deploy/run_local.sh by shellcheck

Use cd ... || exit in case cd fails.

Problematic code:

cd generated_files
rm -r *.c

func(){
  cd foo
  do_something
}

Correct code:

cd generated_files || exit
rm -r *.c

# For functions, you may want to use return:
func(){
  cd foo || return
  do_something
}

Rationale:

cd can fail for a variety of reasons: misspelled paths, missing directories, missing permissions, broken symlinks and more.

If/when it does, the script will keep going and do all its operations in the wrong directory. This can be messy, especially if the operations involve creating or deleting a lot of files.

To avoid this, make sure you handle the cases when cd fails. Ways to do this include

  • cd foo || exit as suggested to just abort immediately
  • if cd foo; then echo "Ok"; else echo "Fail"; fi for custom handling
  • <(cd foo && cmd) as an alternative to <(cd foo || exit; cmd) in <(..), $(..) or ( )

Exceptions:

ShellCheck does not give this warning when cd is on the left of a || or &&, or the condition of a if, while or until loop. Having a set -e command anywhere in the script will disable this message, even though it won't necessarily prevent the issue.

If you are accounting for cd failures in a way shellcheck doesn't realize, you can disable this message with a [[directive]].

Notice

Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.

There are no issues that match your filters.

Category
Status