[bash] I just assigned a variable, but echo $variable shows something else

Here are a series of cases where echo $var can show a different value than what was just assigned. This happens regardless of whether the assigned value was "double quoted", 'single quoted' or unquoted.

How do I get the shell to set my variable correctly?

Asterisks

The expected output is /* Foobar is free software */, but instead I get a list of filenames:

$ var="/* Foobar is free software */"
$ echo $var 
/bin /boot /dev /etc /home /initrd.img /lib /lib64 /media /mnt /opt /proc ...

Square brackets

The expected value is [a-z], but sometimes I get a single letter instead!

$ var=[a-z]
$ echo $var
c

Line feeds (newlines)

The expected value is a a list of separate lines, but instead all the values are on one line!

$ cat file
foo
bar
baz

$ var=$(cat file)
$ echo $var
foo bar baz

Multiple spaces

I expected a carefully aligned table header, but instead multiple spaces either disappear or are collapsed into one!

$ var="       title     |    count"
$ echo $var
title | count

Tabs

I expected two tab separated values, but instead I get two space separated values!

$ var=$'key\tvalue'
$ echo $var
key value

This question is related to bash shell sh quoting

The answer is


In addition to other issues caused by failing to quote, -n and -e can be consumed by echo as arguments. (Only the former is legal per the POSIX spec for echo, but several common implementations violate the spec and consume -e as well).

To avoid this, use printf instead of echo when details matter.

Thus:

$ vars="-e -n -a"
$ echo $vars      # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a

However, correct quoting won't always save you when using echo:

$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed

...whereas it will save you with printf:

$ vars="-n"
$ printf '%s\n' "$vars"
-n

Additional to putting the variable in quotation, one could also translate the output of the variable using tr and converting spaces to newlines.

$ echo $var | tr " " "\n"
foo
bar
baz

Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.


The answer from ks1322 helped me to identify the issue while using docker-compose exec:

If you omit the -T flag, docker-compose exec add a special character that break output, we see b instead of 1b:

$ test=$(/usr/local/bin/docker-compose exec db bash -c "echo 1")
$ echo "${test}b"
b
echo "${test}" | cat -vte
1^M$

With -T flag, docker-compose exec works as expected:

$ test=$(/usr/local/bin/docker-compose exec -T db bash -c "echo 1")
$ echo "${test}b"
1b

echo $var output highly depends on the value of IFS variable. By default it contains space, tab, and newline characters:

[ks@localhost ~]$ echo -n "$IFS" | cat -vte
 ^I$

This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var) and thus expected output is altered.

One way to prevent word splitting (besides using double quotes) is to set IFS to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :

If the value of IFS is null, no field splitting shall be performed.

Setting to null means setting to empty value:

IFS=

Test:

[ks@localhost ~]$ echo -n "$IFS" | cat -vte
 ^I$
[ks@localhost ~]$ var=$'key\nvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$ 

user double quote to get the exact value. like this:

echo "${var}"

and it will read your value correctly.


You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:

Why do I need to write "$foo"? What happens without the quotes?

$foo does not mean “take the value of the variable foo”. It means something much more complex:

  • First, take the value of the variable.
  • Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable contains foo * bar ? then the result of this step is the 3-element list foo, *, bar.
  • Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this pattern. If the pattern doesn't match any files, it is left unmodified. In our example, this results in the list containing foo, following by the list of files in the current directory, and finally bar. If the current directory is empty, the result is foo, *, bar.

Note that the result is a list of strings. There are two contexts in shell syntax: list context and string context. Field splitting and filename generation only happen in list context, but that's most of the time. Double quotes delimit a string context: the whole double-quoted string is a single string, not to be split. (Exception: "$@" to expand to the list of positional parameters, e.g. "$@" is equivalent to "$1" "$2" "$3" if there are three positional parameters. See What is the difference between $* and $@?)

The same happens to command substitution with $(foo) or with `foo`. On a side note, don't use `foo`: its quoting rules are weird and non-portable, and all modern shells support $(foo) which is absolutely equivalent except for having intuitive quoting rules.

The output of arithmetic substitution also undergoes the same expansions, but that isn't normally a concern as it only contains non-expandable characters (assuming IFS doesn't contain digits or -).

See When is double-quoting necessary? for more details about the cases when you can leave out the quotes.

Unless you mean for all this rigmarole to happen, just remember to always use double quotes around variable and command substitutions. Do take care: leaving out the quotes can lead not just to errors but to security holes.


Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to sh

How to run a cron job inside a docker container? I just assigned a variable, but echo $variable shows something else How to run .sh on Windows Command Prompt? Shell Script: How to write a string to file and to stdout on console? How to cat <<EOF >> a file containing code? Assigning the output of a command to a variable What does set -e mean in a bash script? Get specific line from text file using just shell script Printing PDFs from Windows Command Line Ubuntu says "bash: ./program Permission denied"

Examples related to quoting

I just assigned a variable, but echo $variable shows something else What Vim command(s) can be used to quote/unquote words? How to escape single quotes within single quoted strings