[linux] Read line by line in bash script

I want to do the following, read line by line of a file and use the value per line as params

FILE="cat test"
echo "$FILE" | \
while read CMD; do
echo $CMD
done

but when I do the echo $CMD, it just returns cat :S

xargs is the most flexible solution for splitting output into command arguments.

It is also very human readable and easy to use due to its simple parameterisation.

Format is xargs -n $NUMLINES mycommand.

For example, to echo each individual line in a file /tmp/tmp.txt you'd do:

cat /tmp/tmp.txt | xargs -n 1 echo

Or to diff each successive pair of files listed as lines in a file of the above name you'd do:

cat /tmp/tmp.txt | xargs -n 2 diff

The -n 2 instructs xargs to consume and pass as separate arguments two lines of what you've piped into it at a time.

You can tailor xargs to split on delimiters besides carriage return/newline.

Use man xargs and google to find out more about the power of this versatile utility.

The correct version of your script is as follows;

FILE="cat test"
$FILE | \
while read CMD; do
echo $CMD
done

However this kind of indirection --putting your command in a variable named FILE-- is unnecessary. Use one of the solutions already provided. I just wanted to point out your mistake.

while read CMD; do
    echo $CMD
done  << EOF
data line 1
data line 2
..
EOF

If you want to use each of the lines of the file as command-line params for your application you can use the xargs command.

xargs -a <params_file> <command>

A params file with:

a
b
c
d

and the file tr.py:

import sys
print sys.argv

The execution of

xargs -a params ./tr.py

gives the result:

['./tr.py', 'a', 'b', 'c', 'd']

FILE=test

while read CMD; do
    echo "$CMD"
done < "$FILE"

A redirection with < "$FILE" has a few advantages over cat "$FILE" | while .... It avoids a useless use of cat, saving an unnecessary child process. It also avoids a common pitfall where the loop runs in a subshell. In bash, commands in a | pipeline run in subshells, which means variable assignments are lost after the loop ends. Redirection with < doesn't have that problem, so you could use $CMD after the loop or modify other variables inside the loop. It also, again, avoids unnecessary child processes.

There are some additional improvements that could be made:

• Add IFS= so that read won't trim leading and trailing whitespace from each line.
• Add -r to read to prevent from backslashes from being interpreted as escape sequences.
• Lower case CMD and FILE. The bash convention is only environmental and internal shell variables are uppercase.
• Use printf in place of echo which is safer if $cmd is a string like -n, which echo would interpret as a flag.

file=test

while IFS= read -r cmd; do
    printf '%s\n' "$cmd"
done < "$file"

Do you mean to do:

cat test | \
while read CMD; do
echo $CMD
done