How do you run a command for each line of a file

Question

For example  right now I m using the following to change a couple of files whose Unix paths I wrote to a file   cat file txt   while read in  do chmod 755   in   done   Is there a more elegant  safer way

User · Answer

The logic applies to many other objectives. And how to read .sh_history of each user from /home/ filesystem? What if there are thousand of them?

#!/bin/ksh
last |head -10|awk '{print $1}'|
 while IFS= read -r line
 do
su - "$line" -c 'tail .sh_history'
 done

Here is the script https://github.com/imvieira/SysAdmin_DevOps_Scripts/blob/master/get_and_run.sh

User · Answer

I know it s late but still  If by any chance you run into windows saved text file with  r n instead of  n  you might get confused by the output if your command has sth after read line as argument  So do remove  r  for example   cat file   tr -d   r    xargs -L 1 -i echo do sth with    as line

User · Answer

If you know you don t have any whitespace in the input   xargs chmod 755  lt  file txt   If there might be whitespace in the paths  and if you have GNU xargs   tr   n    0   lt  file txt   xargs -0 chmod 755

User · Answer

if you have a nice selector   for example all  txt files in a dir  you could do  for i in   txt  do chmod 755  quot  i quot   done  bash for loop or a variant of yours  while read line  do chmod 755  quot  line quot   done  lt  file txt

User · Answer

If you want to run your command in parallel for each line you can use GNU Parallel  parallel -a  lt your file gt   lt program gt    Each line of your file will be passed to program as an argument  By default parallel runs as many threads as your CPUs count  But you can specify it with -j

User · Answer

I see that you tagged bash  but Perl would also be a good way to do this   perl -p -e   chmod 755      file txt   You could also apply a regex to make sure you re getting the right files  e g  to only process  txt files   perl -p -e  if    txt     chmod 755      file txt   To  preview  what s happening  just replace the backticks with double quotes and prepend print   perl -p -e  if    txt    print  chmod 755      file txt

User · Answer

Yes   while read in  do chmod 755   in   done  lt  file txt   This way you can avoid a cat process   cat is almost always bad for a purpose such as this  You can read more about Useless Use of Cat

User · Answer

Read a file line by line and execute commands  4 answers  This is because there is not only 1 answer      shell command line expansion xargs dedicated tool while read with some remarks while read -u using dedicated fd  for interactive processing  sample    Regarding the OP request  running chmod on all targets listed in file  xargs is the indicated tool  But for some other applications  small amount of files  etc      Read entire file as command line argument   If your file is not too big and all files are well named  without spaces or other special chars like quotes   you could use shell command line expansion  Simply   chmod 755    lt file txt    For small amount of files  lines   this command is the lighter one  xargs is the right tool  For bigger amount of files  or almost any number of lines in your input file     For many binutils tools  like chown  chmod  rm  cp -t      xargs chmod 755  lt file txt   If you have special chars and or a lot of lines in file txt   xargs -0 chmod 755  lt   lt  tr   n   0  lt file txt    if your command need to be run exactly 1 time by entry   xargs -0 -n 1 chmod 755  lt   lt  tr   n   0  lt file txt    This is not needed for this sample  as chmod accept multiple files as argument  but this match the title of question   For some special case  you could even define location of file argument in commands generateds by xargs   xargs -0 -I      -n 1 myWrapper -arg1 -file      wrapCmd  lt   lt  tr   n   0  lt file txt    Test with seq 1 5 as input  Try this   xargs -n 1 -I   echo Blah    blabla       lt   lt  seq 1 5  Blah 1 blabla 1   Blah 2 blabla 2   Blah 3 blabla 3   Blah 4 blabla 4   Blah 5 blabla 5     Where commande is done once per line  while read and variants   As OP suggest cat file txt   while read in  do chmod 755   in   done will work  but there is 2 issues    cat   is an useless fork  and   while      done will become a subshell where environment will disapear after  done    So this could be better written   while read in  do chmod 755   in   done  lt  file txt   But    You may be warned about  IFS and read flags   help read    read  read  -r       -d delim       name                  Reads a single line from the standard input    The line is split     into fields as with word splitting  and the first word is assigned     to the first NAME  the second word to the second NAME  and so on        Only the characters found in  IFS are recognized as word delimiters              Options                  -d delim   continue until the first character of DELIM is read                    rather than newline                 -r do not allow backslashes to escape any characters             Exit Status      The return code is zero  unless end-of-file is encountered       In some case  you may need to use  while IFS  read -r in do chmod 755   in  done  lt file txt   For avoiding problems with stranges filenames  And maybe if you encouter problems with UTF-8   while LANG C IFS  read -r in   do chmod 755   in  done  lt file txt  While you use STDIN for reading file txt  your script could not be interactive  you cannot use STDIN anymore    while read -u  using dedicated fd   Syntax  while read     done  lt file txt will redirect STDIN to file txt  That mean  you won t be able to deal with process  until they finish   If you plan to create interactive tool  you have to avoid use of STDIN and use some alternative file descriptor   Constants file descriptors are  0 for STDIN  1 for STDOUT and 2 for STDERR  You could see them by   ls -l  dev fd    or  ls -l  proc self fd    From there  you have to choose unused number  between 0 and 63  more  in fact  depending on sysctl superuser tool  as file descriptor   For this demo  I will use fd 7     exec 7 lt file txt        Without spaces between  7  and   lt    ls -l  dev fd      Then you could use read -u 7 this way   while read -u 7 filename do     ans  while   -z   ans    do         read -p  Process file   filename   y n     -sn1 foo             foo    amp  amp    -z    foo  yn      amp  amp  ans  foo    echo          done     if     ans     y     then         echo Yes         echo  Processing   filename        else         echo No     fi done 7 lt file txt     done   To close fd 7   exec 7 lt  amp -              This will close file descriptor 7  ls -l  dev fd      Nota  I let striked version because this syntax could be usefull  when doing many I O with parallels process   mkfifo sshfifo exec 7 gt   gt  ssh -t user host sh  gt sshfifo  exec 6 lt sshfifo

User · Answer

You can also use AWK which can give you more flexibility to handle the file  awk    print  chmod 755   0       bin sh    file txt   if your file has a field separator like      field1 field2 field3   To get only the first field you do  awk -F     print  chmod 755   1       bin sh    file txt   You can check more details on GNU Documentation https   www gnu org software gawk manual html node Very-Simple html Very-Simple

[bash] How do you run a command for each line of a file?

Examples related to bash

Examples related to loops

Examples related to line