A variable modified inside a while loop is not remembered

Question

In the following program  if I set the variable  foo to the value 1 inside the first if statement  it works in the sense that its value is remembered after the if statement  However  when I set the same variable to the value 2 inside an if which is inside a while statement  it s forgotten after the while loop  It s behaving like I m using some sort of copy of the variable  foo inside the while loop and I am modifying only that particular copy  Here s a complete test program      bin bash  set -e set -u  foo 0 bar  hello    if      bar      hello     then     foo 1     echo  Setting   foo to 1   foo  fi  echo  Variable   foo after if statement   foo     lines  first line nsecond line nthird line   echo -e  lines   while read line do     if      line      second line         then     foo 2     echo  Variable   foo updated to  foo inside if inside while loop      fi     echo  Value of   foo in while loop body   foo  done  echo  Variable   foo after while loop   foo     Output        testbash sh   Setting  foo to 1  1   Variable  foo after if statement  1   Value of  foo in while loop body  1   Variable  foo updated to 2 inside if inside while loop   Value of  foo in while loop body  2   Value of  foo in while loop body  2   Variable  foo after while loop  1    bash --version   GNU bash  version 4 1 10 4 -release  i686-pc-cygwin

User · Answer

This is an interesting question and touches on a very basic concept in Bourne shell and subshell. Here I provide a solution that is different from the previous solutions by doing some kind of filtering. I will give an example that may be useful in real life. This is a fragment for checking that downloaded files conform to a known checksum. The checksum file look like the following (Showing just 3 lines):

49174 36326 dna_align_feature.txt.gz
54757     1 dna.txt.gz
55409  9971 exon_transcript.txt.gz

The shell script:

#!/bin/sh

.....

failcnt=0 # this variable is only valid in the parent shell
#variable xx captures all the outputs from the while loop
xx=$(cat ${checkfile} | while read -r line; do
    num1=$(echo $line | awk '{print $1}')
    num2=$(echo $line | awk '{print $2}')
    fname=$(echo $line | awk '{print $3}')
    if [ -f "$fname" ]; then
        res=$(sum $fname)
        filegood=$(sum $fname | awk -v na=$num1 -v nb=$num2 -v fn=$fname '{ if (na == $1 && nb == $2) { print "TRUE"; } else { print "FALSE"; }}')
        if [ "$filegood" = "FALSE" ]; then
            failcnt=$(expr $failcnt + 1) # only in subshell
            echo "$fname BAD $failcnt"
        fi
    fi
done | tail -1) # I am only interested in the final result
# you can capture a whole bunch of texts and do further filtering
failcnt=${xx#* BAD } # I am only interested in the number
# this variable is in the parent shell
echo failcnt $failcnt
if [ $failcnt -gt 0 ]; then
    echo $failcnt files failed
else
    echo download successful
fi

The parent and subshell communicate through the echo command. You can pick some easy to parse text for the parent shell. This method does not break your normal way of thinking, just that you have to do some post processing. You can use grep, sed, awk, and more for doing so.

User · Answer

echo -e  lines   while read line          done   The while loop is executed in a subshell  So any changes you do to the variable will not be available once the subshell exits   Instead you can use a here string to re-write the while loop to be in the main shell process  only echo -e  lines will run in a subshell   while read line do     if      line      second line         then         foo 2         echo  Variable   foo updated to  foo inside if inside while loop      fi     echo  Value of   foo in while loop body   foo  done  lt  lt  lt     echo -e   lines      You can get rid of the rather ugly echo in the here-string above by expanding the backslash sequences immediately when assigning lines  The        form of quoting can be used there   lines   first line nsecond line nthird line  while read line  do         done  lt  lt  lt    lines

User · Answer

How about a very simple method        call your while loop in a function       - set your value inside  nonsense  but shows the example       - return your value inside       capture your value outside      set outside      display outside          bin bash       set -e       set -u       No idea why you need this  not using here      foo 0     bar  hello       if      bar      hello         then         foo 1         echo  Setting    foo to  foo      fi      echo  Variable   foo after if statement   foo       lines  first line nsecond line nthird line       function my while loop            echo -e  lines   while read line     do         if      line      second line             then         foo 2  return 2          echo  Variable   foo updated to  foo inside if inside while loop          fi          echo -e  lines   while read line do     if      line      second line         then     foo 2                echo  Variable   foo updated to  foo inside if inside while loop      return 2      fi        Code below won t be executed since we returned from function in  if  statement       We aready reported the  foo var beint set to 2 anyway     echo  Value of   foo in while loop body   foo   done        my while loop  foo           echo  Variable   foo after while loop   foo        Output      Setting   foo 1     Variable  foo after if statement  1     Value of  foo in while loop body  1     Variable  foo after while loop  2      bash --version      GNU bash  version 3 2 51 1 -release  x86 64-apple-darwin13      Copyright  C  2007 Free Software Foundation  Inc

User · Answer

Though this is an old question and asked several times  here s what I m doing after hours fidgeting with here strings  and the only option that worked for me is to store the value in a file during while loop sub-shells and then retrieve it  Simple  Use echo statement to store and cat statement to retrieve  And the bash user must chown the directory or have read-write chmod access   write to file echo  quot 1 quot   gt  foo txt  while condition  do      if  condition   then          write again to file         echo  quot 2 quot   gt  foo txt           fi done   read from file echo  quot Value of   foo in while loop body    cat foo txt  quot

User · Answer

You are the 742342nd user to ask this bash FAQ  The answer also describes the general case of variables set in subshells created by pipes      E4  If I pipe the output of a command into read variable  why   doesn t the output show up in  variable when the read command finishes       This has to do with the parent-child relationship between Unix   processes   It affects all commands run in pipelines  not just   simple calls to read   For example  piping a command s output   into a while loop that repeatedly calls read will result in   the same behavior       Each element of a pipeline  even a builtin or shell function    runs in a separate process  a child of the shell running the   pipeline   A subprocess cannot affect its parent s environment     When the read command sets the variable to the input  that   variable is set only in the subshell  not the parent shell   When   the subshell exits  the value of the variable is lost        Many pipelines that end with read variable can be converted   into command substitutions  which will capture the output of   a specified command   The output can then be assigned to a   variable   grep  gnu  usr lib news active   wc -l   read ngroup       can be converted into  ngroup   grep  gnu  usr lib news active   wc -l        This does not  unfortunately  work to split the text among   multiple variables  as read does when given multiple variable   arguments   If you need to do this  you can either use the   command substitution above to read the output into a variable   and chop up the variable using the bash pattern removal   expansion operators or use some variant of the following   approach       Say  usr local bin ipaddr is the following shell script       bin sh host  hostname    awk   address   print  NF         Instead of using   usr local bin ipaddr   read A B C D       to break the local machine s IP address into separate octets  use  OIFS   IFS  IFS   set --    usr local bin ipaddr  IFS   OIFS  A   1  B   2  C   3  D   4        Beware  however  that this will change the shell s positional   parameters   If you need them  you should save them before doing   this       This is the general approach -- in most cases you will not need to   set  IFS to a different value       Some other user-supplied alternatives include   read A B C D  lt  lt  HERE       IFS    echo    usr local bin ipaddr   HERE       and  where process substitution is available   read A B C D  lt   lt  IFS    echo    usr local bin ipaddr

User · Answer

UPDATED 2  Explanation is in Blue Moons s answer   Alternative solutions   Eliminate echo  while read line  do     done  lt  lt EOT first line second line third line EOT   Add the echo inside the here-is-the-document  while read line  do     done  lt  lt EOT   echo -e  lines  EOT   Run echo in background   coproc echo -e  lines while read -u   COPROC 0   line  do      done   Redirect to a file handle explicitly  Mind the space in  lt   lt      exec 3 lt   lt  echo -e   lines  while read -u 3 line  do     done   Or just redirect to the stdin   while read line  do     done  lt   lt  echo -e   lines    And one for chepner  eliminating echo    arr   first line   second line   third line    for  i 0 i lt    arr       i     line   arr i             Variable  lines can be converted to an array without starting a new sub-shell  The characters   and n has to be converted to some character  e g  a real new line character  and use the IFS  Internal Field Separator  variable to split the string into array elements  This can be done like   lines  first line nsecond line nthird line  echo   lines  OIFS   IFS  IFS    n  arr    lines    n    n      Conversion IFS   OIFS  echo    arr       Length     arr     set grep  arr   Result is  first line nsecond line nthird line first line second line third line  Length  3 arr   0   first line   1   second line   2   third line

User · Answer

Hmmm    I would almost swear that this worked for the original Bourne shell  but don t have access to a running copy just now to check   There is  however  a very trivial workaround to the problem   Change the first line of the script from      bin bash   to     bin ksh   Et voila  A read at the end of a pipeline works just fine  assuming you have the Korn shell installed

User · Answer

I use stderr to store within a loop  and read from it outside  Here var i is initially set and read inside the loop as 1    reading lines of content from 2 files concatenated   inside loop  write value of var i to stderr  before iteration    outside  read var i from stderr  has last iterative value  f  tmp file1 g  tmp file2 i 1 cat  f  g     while read -r s  do   echo  s  gt   dev null     some work   echo  i  gt  2   let i   done  read -r i  lt  2 echo  i  Or use the heredoc method to reduce the amount of code in a subshell  Note the iterative i value can be read outside the while loop  i 1 while read -r s  do   echo  s  gt   dev null   let i   done  lt  lt EOT   cat  f  g  EOT let i-- echo  i

[bash] A variable modified inside a while loop is not remembered

Examples related to bash

Examples related to while-loop

Examples related to scope

Examples related to sh