[bash] How do you run a command for each line of a file?

For example, right now I'm using the following to change a couple of files whose Unix paths I wrote to a file:

cat file.txt | while read in; do chmod 755 "$in"; done

Is there a more elegant, safer way?

This question is related to bash loops line

The answer is


I know it's late but still

If by any chance you run into windows saved text file with \r\n instead of \n, you might get confused by the output if your command has sth after read line as argument. So do remove \r, for example:

cat file | tr -d '\r' | xargs -L 1 -i echo do_sth_with_{}_as_line

The logic applies to many other objectives. And how to read .sh_history of each user from /home/ filesystem? What if there are thousand of them?

#!/bin/ksh
last |head -10|awk '{print $1}'|
 while IFS= read -r line
 do
su - "$line" -c 'tail .sh_history'
 done

Here is the script https://github.com/imvieira/SysAdmin_DevOps_Scripts/blob/master/get_and_run.sh


You can also use AWK which can give you more flexibility to handle the file

awk '{ print "chmod 755 "$0"" | "/bin/sh"}' file.txt

if your file has a field separator like:

field1,field2,field3

To get only the first field you do

awk -F, '{ print "chmod 755 "$1"" | "/bin/sh"}' file.txt

You can check more details on GNU Documentation https://www.gnu.org/software/gawk/manual/html_node/Very-Simple.html#Very-Simple


I see that you tagged bash, but Perl would also be a good way to do this:

perl -p -e '`chmod 755 $_`' file.txt

You could also apply a regex to make sure you're getting the right files, e.g. to only process .txt files:

perl -p -e 'if(/\.txt$/) `chmod 755 $_`' file.txt

To "preview" what's happening, just replace the backticks with double quotes and prepend print:

perl -p -e 'if(/\.txt$/) print "chmod 755 $_"' file.txt

If you want to run your command in parallel for each line you can use GNU Parallel

parallel -a <your file> <program>

Each line of your file will be passed to program as an argument. By default parallel runs as many threads as your CPUs count. But you can specify it with -j


if you have a nice selector (for example all .txt files in a dir) you could do:

for i in *.txt; do chmod 755 "$i"; done

bash for loop

or a variant of yours:

while read line; do chmod 755 "$line"; done < file.txt

If you know you don't have any whitespace in the input:

xargs chmod 755 < file.txt

If there might be whitespace in the paths, and if you have GNU xargs:

tr '\n' '\0' < file.txt | xargs -0 chmod 755

Yes.

while read in; do chmod 755 "$in"; done < file.txt

This way you can avoid a cat process.

cat is almost always bad for a purpose such as this. You can read more about Useless Use of Cat.


Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to loops

How to increment a letter N times per iteration and store in an array? Angular 2 Cannot find control with unspecified name attribute on formArrays What is the difference between i = i + 1 and i += 1 in a 'for' loop? Prime numbers between 1 to 100 in C Programming Language Python Loop: List Index Out of Range JavaScript: Difference between .forEach() and .map() Why does using from __future__ import print_function breaks Python2-style print? Creating an array from a text file in Bash Iterate through dictionary values? C# Wait until condition is true

Examples related to line

How do I compute the intersection point of two lines? Java Replace Line In Text File draw diagonal lines in div background with CSS How to make a new line or tab in <string> XML (eclipse/android)? How do you run a command for each line of a file? How can I format a list to print each element on a separate line in python? Remove lines that contain certain string How to paste text to end of every line? Sublime 2 Making PHP var_dump() values display one line per value How to do multiple line editing?