Pipe output and capture exit status in Bash

Question

I want to execute a long running command in Bash  and both capture its exit status  and tee its output   So I do this   command   tee out txt ST      The problem is that the variable ST captures the exit status of tee and not of command  How can I solve this   Note that command is long running and redirecting the output to a file to view it later is not a good solution for me

User · Answer

So I wanted to contribute an answer like lesmana's, but I think mine is perhaps a little simpler and slightly more advantageous pure-Bourne-shell solution:

# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; printf $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.

I think this is best explained from the inside out - command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, printf will execute and print icommand1's exit code on its stdout, but that stdout is redirected to file descriptor 3.

While command1 is running, its stdout is being piped to command2 (printf's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor 1 - because we want file descriptor 1 free for a little bit later, because we will bring the printf output on file descriptor 3 back down into file descriptor 1 - because that's what the command substitution (the backticks), will capture and that's what will get placed into the variable.

The final bit of magic is that first exec 4>&1 we did as a separate command - it opens file descriptor 4 as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it - but since command2's output is going to file descriptor 4 as far as the command substitution is concerned, the command substitution doesn't capture it - however once it gets "out" of the command substitution it is effectively still going to the script's overall file descriptor 1.

(The exec 4>&1 has to be a separate command because many common shells don't like it when you try to write to a file descriptor inside a command substitution, that is opened in the "external" command that is using the substitution. So this is the simplest portable way to do it.)

You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the printf's output jumps over command 2 so that command2 doesn't catch it, and then command 2's output jumps over and out of the command substitution just as printf lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its merry way being written to the standard output, just as in a normal pipe.

Also, as I understand it, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out - this, and not having to define an additional function, is why I think this might be a somewhat better solution than the one proposed by lesmana.

Per the caveats lesmana mentions, it's possible that command1 will at some point end up using file descriptors 3 or 4, so to be more robust, you would do:

exec 4>&1
exitstatus=`{ { command1 3>&-; printf $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-

Note that I use compound commands in my example, but subshells (using ( ) instead of { } will also work, though may perhaps be less efficient.)

Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.

I'm not sure how often things use file descriptor three and four directly - I think most of the time programs use syscalls that return not-used-at-the-moment file descriptors, but sometimes code writes to file descriptor 3 directly, I guess (I could imagine a program checking a file descriptor to see if it's open, and using it if it is, or behaving differently accordingly if it's not). So the latter is probably best to keep in mind and use for general-purpose cases.

User · Answer

In Ubuntu and Debian  you can apt-get install moreutils    This contains a utility called mispipe that returns the exit status of the first command in the pipe

User · Answer

There is an internal Bash variable called  PIPESTATUS  it   s an array that holds the exit status of each command in your last foreground pipeline of commands    lt command gt    tee out txt   test   PIPESTATUS 0   -eq 0   Or another alternative which also works with other shells  like zsh  would be to enable pipefail   set -o pipefail       The first option does not work with zsh due to a little bit different syntax

User · Answer

It may sometimes be simpler and clearer to use an external command  rather than digging into the details of bash  pipeline  from the minimal process scripting language execline  exits with the return code of the second command   just like a sh pipeline does  but unlike sh  it allows reversing the direction of the pipe  so that we can capture the return code of the producer process  the below is all on the sh command line  but with execline installed        using the full execline grammar with the execlineb parser    execlineb -c  pipeline   echo  hello world    tee out txt  hello world   cat out txt hello world      for these simple examples  one can forego the parser and just use    as a separator     traditional order   pipeline echo  hello world     tee out txt  hello world       write  order  second command writes rather than reads    pipeline -w tee out txt    echo  hello world  hello world      pipeline execs into the second command  so that s the RC we get   pipeline -w tee out txt    false  echo    1    pipeline -w tee out txt    true  echo    0      output and exit status   pipeline -w tee out txt    sh -c  echo  hello world   exit 42   echo  RC      hello world RC  42   cat out txt hello world   Using pipeline has the same differences to native bash pipelines as the bash process substitution used in answer  43972501     Actually pipeline doesn t exit at all unless there is an error  It executes into the second command  so it s the second command that does the returning

User · Answer

PIPESTATUS    must be copied to an array immediately after the pipe command returns  Any reads of PIPESTATUS    will erase the contents   Copy it to another array if you plan on checking the status of all pipe commands       is the same value as the last element of    PIPESTATUS       and reading it seems to destroy    PIPESTATUS       but I haven t absolutely verified this   declare -a PSA   cmd1   cmd2   cmd3   PSA      PIPESTATUS          This will not work if the pipe is in a sub-shell  For a solution to that problem  see bash pipestatus in backticked command

User · Answer

command   tee out txt  exit   PIPESTATUS 0      Unlike  cODAR s answer this returns the original exit code of the first command and not only 0 for success and 127 for failure  But as  Chaoran pointed out you can just call   PIPESTATUS 0    It is important however that all is put into brackets

User · Answer

There s an array that gives you the exit status of each command in a pipe     cat x  sed  s     cat  x  No such file or directory   echo    0   cat x  sed  s     cat  x  No such file or directory   echo   PIPESTATUS     1 0   touch x   cat x  sed  s  sed  1   s   substitute pattern can not be delimited by newline or backslash   echo   PIPESTATUS     0 1

User · Answer

By combining PIPESTATUS 0  and the result of executing the exit command in a subshell  you can directly access the return value of your initial command   command   tee     exit   PIPESTATUS 0      Here s an example     the  false  shell built-in command returns 1 false   tee     exit   PIPESTATUS 0     echo  return value        will give you   return value  1

User · Answer

using bash s set -o pipefail is helpful     pipefail  the return value of a pipeline is the status of      the last command to exit with a non-zero status       or zero if no command exited with a non-zero status

User · Answer

Dumb solution  Connecting them through a named pipe  mkfifo   Then the command can be run second    mkfifo pipe  tee out txt  lt  pipe  amp   command  gt  pipe  echo

User · Answer

Outside of bash  you can do   bash -o pipefail  -c  command1   tee output    This is useful for example in ninja scripts where the shell is expected to be  bin sh

User · Answer

Pure shell solution     rm -f error flag  echo hello world      cat    echo  First command failed       gt  gt  error flag       cat    echo  Second command failed       gt  gt  error flag       cat    echo  Third command failed       gt  gt  error flag      test -s error flag   amp  amp   echo Some command failed    cat error flag  hello world   And now with the second cat replaced by false     rm -f error flag  echo hello world      cat    echo  First command failed       gt  gt  error flag       false    echo  Second command failed       gt  gt  error flag       cat    echo  Third command failed       gt  gt  error flag      test -s error flag   amp  amp   echo Some command failed    cat error flag  Some command failed  Second command failed  1 First command failed  141   Please note the first cat fails as well  because it s stdout gets closed on it  The order of the failed commands in the log is correct in this example  but don t rely on it   This method allows for capturing stdout and stderr for the individual commands so you can then dump that as well into a log file if an error occurs  or just delete it if no error  like the output of dd

User · Answer

The simplest way to do this in plain bash is to use process substitution instead of a pipeline  There are several differences  but they probably don t matter very much for your use case    When running a pipeline  bash waits until all processes complete  Sending Ctrl-C to bash makes it kill all the processes of a pipeline  not just the main one  The pipefail option and the PIPESTATUS variable are irrelevant to process substitution  Possibly more   With process substitution  bash just starts the process and forgets about it  it s not even visible in jobs   Mentioned differences aside  consumer  lt   lt  producer  and producer   consumer are essentially equivalent   If you want to flip which one is the  main  process  you just flip the commands and the direction of the substitution to producer  gt   gt  consumer   In your case   command  gt   gt  tee out txt    Example       echo  hello world   false     gt   gt  tee out txt  hello world   echo    1   cat out txt hello world    echo  hello world   gt   gt  tee out txt  hello world   echo    0   cat out txt hello world   As I said  there are differences from the pipe expression  The process may never stop running  unless it is sensitive to the pipe closing  In particular  it may keep writing things to your stdout  which may be confusing

User · Answer

Base on  brian-s-wilson  s answer  this bash helper function   pipestatus       local S     PIPESTATUS          if test -n        then test           S        else         S           0         fi     used thus   1  get bad things must succeed  but it should produce no output  but we want to see output that it does produce  get bad things   grep     pipeinfo 0 1    return   2  all pipeline must succeed  thing   something -q   thingy pipeinfo    return

User · Answer

This solution works without using bash specific features or temporary files  Bonus  in the end the exit status is actually an exit status and not some string in a file   Situation   someprog   filter   you want the exit status from someprog and the output from filter   Here is my solution       someprog  echo     gt  amp 3    filter  gt  amp 4  3 gt  amp 1     read xs  exit  xs   4 gt  amp 1  echo      See my answer for the same question on unix stackexchange com for a detailed explanation and an alternative without subshells and some caveats

[bash] Pipe output and capture exit status in Bash

Examples related to bash

Examples related to shell

Examples related to error-handling

Examples related to pipe