Quotes/backslashes will be treated literally. Use an array. Open
updates='{"repositories": [{"packagist.org": false}, {"type": "path", "url": "'$GITHUB_WORKSPACE'"}, {"type": "composer", "url": "https://packagist.org"}], "prefer-stable": true, "minimum-stability": "dev"}'
- Read upRead up
- Exclude checks
Quotes/backslashes will be treated literally. Use an array.
Problematic code:
args='-lh "My File.txt"'
ls $args
Correct code:
args=(-lh "My File.txt")
ls "${args[@]}"
Rationale:
Bash does not interpret data as code. Consider almost any other languages, such as Python:
print 1+1 # prints 2
a="1+1"
print a # prints 1+1, not 2
Here, 1+1
is Python syntax for adding numbers. However, passing a literal string containing this expression does not cause Python to interpret it, see the +
and produce the calculated result.
Similarly, "My File.txt"
is Bash syntax for a single word with a space in it. However, passing a literal string containing this expression does not cause Bash to interpret it, see the quotes and produce the tokenized result.
The solution is to use an array instead, whenever possible.
If due to sh
compatibility you can't use arrays, you can use eval
instead. However, this is very insecure and easy to get wrong, leading to various forms of security vulnerabilities and breakage:
quote() { local q=${1//\'/\'\\\'\'}; echo "'$q'"; }
args="-lh $(quote "My File.txt")"
eval ls "$args" # Do not use unless you understand implications
If you ever accidentally forget to use proper quotes, such as with:
for f in *.txt; do
args="-lh '$1'" # Example security exploit
eval ls "$args" # Do not copy and use
done
Then you can use touch "'; rm -rf \$'\x2F'; '.txt"
(or someone can trick you into downloading a file with this name, or create a zip file or git repo containing it, or changing their nick and have your chat client create the file for a chat log, or...), and running the script to list your files will run the command rm -rf /
.
Exceptions
Few and far between.
Additional resources
Wooledge BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!
Notice
Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.
Quote this to prevent word splitting. Open
if test $(grep -c nb_uniq_visitors "$output_file") -gt 0; then
- Read upRead up
- Exclude checks
Quote this to prevent word splitting
Problematic code:
ls -l $(getfilename)
Correct code:
# getfilename outputs 1 file
ls -l "$(getfilename)"
# getfilename outputs multiple files, linefeed separated
getfilename | while IFS='' read -r line
do
ls -l "$line"
done
Rationale:
When command expansions are unquoted, word splitting and globbing will occur. This often manifests itself by breaking when filenames contain spaces.
Trying to fix it by adding quotes or escapes to the data will not work. Instead, quote the command substitution itself.
If the command substitution outputs multiple pieces of data, use a loop instead.
Exceptions
In rare cases you actually want word splitting, such as in
gcc $(pkg-config --libs openssl) client.c
This is because pkg-config
outputs -lssl -lcrypto
, which you want to break up by spaces into -lssl
and -lcrypto
. An alternative is to put the variables to an array and expand it:
args=( $(pkg-config --libs openssl) )
gcc "${args[@]}" client.c
The power of using an array becomes evident when you want to combine, for example, the command result with user-provided arguments:
compile () {
args=( $(pkg-config --libs openssl) "${@}" )
gcc "${args[@]}" client.c
}
compile -DDEBUG
+ gcc -lssl -lcrypto -DDEBUG client.c
Notice
Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.
Double quote to prevent globbing and word splitting. Open
echo $updates > updates.json
- Read upRead up
- Exclude checks
Double quote to prevent globbing and word splitting.
Problematic code:
echo $1
for i in $*; do :; done # this done and the next one also applies to expanding arrays.
for i in $@; do :; done
Correct code:
echo "$1"
for i in "$@"; do :; done # or, 'for i; do'
Rationale
The first code looks like "print the first argument". It's actually "Split the first argument by IFS (spaces, tabs and line feeds). Expand each of them as if it was a glob. Join all the resulting strings and filenames with spaces. Print the result."
The second one looks like "iterate through all arguments". It's actually "join all the arguments by the first character of IFS (space), split them by IFS and expand each of them as globs, and iterate on the resulting list". The third one skips the joining part.
Quoting variables prevents word splitting and glob expansion, and prevents the script from breaking when input contains spaces, line feeds, glob characters and such.
Strictly speaking, only expansions themselves need to be quoted, but for stylistic reasons, entire arguments with multiple variable and literal parts are often quoted as one:
$HOME/$dir/dist/bin/$file # Unquoted (bad)
"$HOME"/"$dir"/dist/bin/"$file" # Minimal quoting (good)
"$HOME/$dir/dist/bin/$file" # Canonical quoting (good)
When quoting composite arguments, make sure to exclude globs and brace expansions, which lose their special meaning in double quotes: "$HOME/$dir/src/*.c"
will not expand, but "$HOME/$dir/src"/*.c
will.
Note that $( )
starts a new context, and variables in it have to be quoted independently:
echo "This $variable is quoted $(but this $variable is not)"
echo "This $variable is quoted $(and now this "$variable" is too)"
Exceptions
Sometimes you want to split on spaces, like when building a command line:
options="-j 5 -B"
make $options file
Just quoting this doesn't work. Instead, you should have used an array (bash, ksh, zsh):
options=(-j 5 -B) # ksh: set -A options -- -j 5 -B
make "${options[@]}" file
or a function (POSIX):
make_with_flags() { make -j 5 -B "$@"; }
make_with_flags file
To split on spaces but not perform glob expansion, Posix has a set -f
to disable globbing. You can disable word splitting by setting IFS=''
.
Similarly, you might want an optional argument:
debug=""
[[ $1 == "--trace-commands" ]] && debug="-x"
bash $debug script
Quoting this doesn't work, since in the default case, "$debug"
would expand to one empty argument while $debug
would expand into zero arguments. In this case, you can use an array with zero or one elements as outlined above, or you can use an unquoted expansion with an alternate value:
debug=""
[[ $1 == "--trace-commands" ]] && debug="yes"
bash ${debug:+"-x"} script
This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}
.
As always, this warning can be [[ignore]]d on a case-by-case basis.
this is especially relevant when BASH many not be available for the array work around. For example, use in eval or in command options where script has total control of the variables...
FLAGS="-av -e 'ssh -x' --delete --delete-excluded"
...
# shellcheck disable=SC2086
eval rsync $FLAGS ~/dir remote_host:dir
Notice
Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.
Quotes/backslashes in this variable will not be respected. Open
echo $updates > updates.json
- Read upRead up
- Exclude checks
Quotes/backslashes in this variable will not be respected.
See companion warning, [[SC2089]].
Notice
Original content from the ShellCheck https://github.com/koalaman/shellcheck/wiki.