How to store standard error in a variable

Question

Let s say I have a script like the following   useless sh  echo  This Is Error  1 gt  amp 2 echo  This Is Output     And I have another shell script   alsoUseless sh    useless sh   sed  s Output Useless     I want to capture  This Is Error   or any other stderr from useless sh  into a variable   Let s call it ERROR   Notice that I am using stdout for something  I want to continue using stdout  so redirecting stderr into stdout is not helpful  in this case   So  basically  I want to do    useless sh 2 gt   ERROR         but that obviously doesn t work   I also know that I could do    useless sh 2 gt   tmp Error ERROR  cat  tmp Error    but that s ugly and unnecessary   Unfortunately  if no answers turn up here that s what I m going to have to do   I m hoping there s another way   Anyone have any better ideas

User · Answer

I ll use find command find   -maxdepth 2 -iname  tmp  -type d  as non superuser for the demo  It should complain  Permission denied  when acessing   dir     bin bash  echo  quot terminal  quot    err  quot   find   -maxdepth 2 -iname  tmp  -type d 2 gt  amp 1 1 gt  amp 3 3 gt  amp -   tee  dev stderr  quot     3 gt  amp 1   tee  dev fd 4 2 gt  amp 1  out   cat  dev fd 4  echo  quot stdout  quot   amp  amp  echo  quot  out quot  echo  quot stderr  quot   amp  amp  echo  quot  err quot   that gives output  terminal  find      root     Permission denied  tmp  var tmp find      lost found     Permission denied stdout   tmp  var tmp stderr  find      root     Permission denied find      lost found     Permission denied  The terminal output has also  dev stderr content the same way as if you were running that find command without any script   out has  dev stdout and  err has  dev stderr content  use     bin bash  echo  quot terminal  quot    err  quot   find   -maxdepth 2 -iname  tmp  -type d 2 gt  amp 1 1 gt  amp 3 3 gt  amp -  quot     3 gt  amp 1   tee  dev fd 4  out   cat  dev fd 4  echo  quot stdout  quot   amp  amp  echo  quot  out quot  echo  quot stderr  quot   amp  amp  echo  quot  err quot   if you don t want to see  dev stderr in the terminal output  terminal   tmp  var tmp stdout   tmp  var tmp stderr  find      root     Permission denied find      lost found     Permission denied

User · Answer

Improving on YellowApple s answer   This is a Bash function to capture stderr into any variable  stderr capture example sh      usr bin env bash    Capture stderr from a command to a variable while maintaining stdout    Args     1  The variable name to store the stderr output    2  Vararg command and arguments    Return    The Command s Returnn-Code or 2 if missing arguments function capture stderr          -lt 2    amp  amp  return 2   local stderr   1    shift         printf -v   stderr    s            1 gt  amp 3    2 gt  amp 1       3 gt  amp 1      Testing with a call to erroring ls LANG C capture stderr my stderr ls   0      printf   nmy stderr contains  n s    my stderr    Testing   bash stderr capture example sh   Output    stderr capture example sh  my stderr contains  ls  cannot access     No such file or directory   This function can be used to capture the returned choice of a dialog command

User · Answer

There are a lot of duplicates for this question  many of which have a slightly simpler usage scenario where you don t want to capture stderr and stdout and the exit code all at the same time   if result   useless sh 2 gt  amp 1   then     stdout  result else     rc        stderr  result fi   works for the common scenario where you expect either proper output in the case of success  or a diagnostic message on stderr in the case of failure   Note that the shell s control statements already examine    under the hood  so anything which looks like  cmd if      -eq 0    then       is just a clumsy  unidiomatic way of saying  if cmd  then

User · Answer

In zsh         useless sh  gt   dev tty   2 gt  amp 1   read ERROR   echo  ERROR   your message

User · Answer

This is an interesting problem to which I hoped there was an elegant solution   Sadly  I end up with a solution similar to Mr  Leffler  but I ll add that you can call useless from inside a Bash function for improved readability       bin bash  function useless        tmp useless sh   sed  s Output Useless      ERROR   useless  echo  ERROR   All other kind of output redirection must be backed by a temporary file

User · Answer

Redirected stderr to stdout  stdout to  dev null  and then use the backticks or     to capture the redirected stderr   ERROR     useless sh 2 gt  amp 1  gt  dev null

User · Answer

Iterating a bit on Tom Hale s answer  I ve found it possible to wrap the redirection yoga into a function for easier reuse   For example     bin sh  capture            captured         quot    quot      1 gt  amp 3     2 gt  amp 1     3 gt  amp 1      Example usage  capturing dialog s output without resorting to temp files   was what motivated me to search for this particular SO question capture dialog --menu  quot Pick one  quot  0 0 0            quot FOO quot   quot Foo quot             quot BAR quot   quot Bar quot             quot BAZ quot   quot Baz quot  choice  captured  clear  echo  choice  It s almost certainly possible to simplify this further   Haven t tested especially-thoroughly  but it does appear to work with both bash and ksh   EDIT  an alternative version of the capture function which stores the captured STDERR output into a user-specified variable  instead of relying on a global  captured   taking inspiration from L  a Gris s answer while preserving the ksh  and zsh  compatibility of the above implementation  capture          if    quot    quot  -lt 2    then         echo  quot Usage  capture varname command  arg      quot          return 1     fi     typeset var captured  captured  quot  1 quot   shift       read  captured  lt  lt  lt         quot    quot      1 gt  amp 3     2 gt  amp 1     3 gt  amp 1    And usage  capture choice dialog --menu  quot Pick one  quot  0 0 0            quot FOO quot   quot Foo quot             quot BAR quot   quot Bar quot             quot BAZ quot   quot Baz quot   clear  echo  choice

User · Answer

alsoUseless sh This will allow you to pipe the output of your useless sh script through a command such as sed and save the stderr in a variable named error  The result of the pipe is sent to stdout for display or to be piped into another command  It sets up a couple of extra file descriptors to manage the redirections needed in order to do this     bin bash  exec 3 gt  amp 1 4 gt  amp 2  set up extra file descriptors  error        useless sh   sed  s Output Useless   2 gt  amp 4 1 gt  amp 3    2 gt  amp 1    echo  quot The message is   quot   error    quot  quot   exec 3 gt  amp - 4 gt  amp -   release the extra file descriptors

User · Answer

For the benefit of the reader  this recipe here   can be re-used as oneliner to catch stderr into a variable still gives access to the return code of the command Sacrifices a temporary file descriptor 3  which can be changed by you of course  And does not expose this temporary file descriptors to the inner command   If you want to catch stderr of some command into var you can do    var       command    2 gt  amp 1 1 gt  amp 3 3 gt  amp -       3 gt  amp 1    Afterwards you have it all   echo  command gives    and stderr   var      If command is simple  not something like a   b  you can leave the inner    away     var    command 2 gt  amp 1 1 gt  amp 3 3 gt  amp -      3 gt  amp 1    Wrapped into an easy reusable bash-function  probably needs version 3 and above for local -n      catch-stderr var cmd  args    catch-stderr     local -n v   1   shift  amp  amp    v         2 gt  amp 1 1 gt  amp 3 3 gt  amp -      3 gt  amp 1      Explained    local -n aliases   1   which is the variable for catch-stderr  3 gt  amp 1 uses file descriptor 3 to save there stdout points   command     or       then executes the command within the output capturing       Please note that the exact order is important here  doing it the wrong way shuffles the file descriptors wrongly     2 gt  amp 1 redirects stderr to the output capturing       1 gt  amp 3 redirects stdout away from the output capturing       back to the  outer  stdout which was saved in file descriptor 3   Note that stderr still refers to where FD 1 pointed before   To the output capturing       3 gt  amp - then closes the file descriptor 3 as it is no more needed  such that command does not suddenly has some unknown open file descriptor showing up   Note that the outer shell still has FD 3 open  but command will not see it  The latter is important  because some programs like lvm complain about unexpected file descriptors   And lvm complains to stderr - just what we are going to capture     You can catch any other file descriptor with this recipe  if you adapt accordingly   Except file descriptor 1 of course  here the redirection logic would be wrong  but for file descriptor 1 you can just use var   command  as usual    Note that this sacrifices file descriptor 3   If you happen to need that file descriptor  feel free to change the number   But be aware  that some shells  from the 1980s  might understand 99 gt  amp 1 as argument 9 followed by 9 gt  amp 1  this is no problem for bash    Also note that it is not particluar easy to make this FD 3 configurable through a variable   This makes things very unreadable     catch-var-from-fd-by-fd variable fd-to-catch fd-to-sacrifice command  args    catch-var-from-fd-by-fd     local -n v   1   local fd1   2  fd2   3   shift 3    return   eval exec   fd2 gt  amp 1   v    eval          fd1 gt  amp 1   1 gt  amp  fd2    fd2 gt  amp -     eval exec   fd2 gt  amp -          Security note  The first 3 arguments to catch-var-from-fd-by-fd must not be taken from a 3rd party   Always give them explicitly in a  static  fashion       So no-no-no catch-var-from-fd-by-fd  var  fda  fdb  command  never do this       If you happen to pass in a variable variable name  at least do it as follows    local -n var   var   catch-var-from-fd-by-fd var 3 5  command      This still will not protect you against every exploit  but at least helps to detect and avoid common scripting errors    Notes    catch-var-from-fd-by-fd var 2 3 cmd   is the same as catch-stderr var cmd   shift    return is just some way to prevent ugly errors in case you forget to give the correct number of arguments   Perhaps terminating the shell would be another way  but this makes it hard to test from commandline   The routine was written such  that it is more easy to understand   One can rewrite the function such that it does not need exec  but then it gets really ugly  This routine can be rewritten for non-bash as well such that there is no need for local -n   However then you cannot use local variables and it gets extremely ugly  Also note that the evals are used in a safe fashion   Usually eval is considerered dangerous   However in this case it is no more evil than using       to execute arbitrary commands    However please be sure to use the exact and correct quoting as shown here  else it becomes very very dangerous

User · Answer

This post helped me come up with a similar solution for my own purposes   MESSAGE    echo  ERROR MESSAGE   format logs py --level ERROR    2 gt  amp 1    Then as long as our MESSAGE is not an empty string  we pass it on to other stuff   This will let us know if our format logs py failed with some kind of python exception

User · Answer

If you want to bypass the use of a temporary file you may be able to use process substitution  I haven t quite gotten it to work yet  This was my first attempt      useless sh 2 gt   gt   ERROR    lt     -bash  command substitution  line 42  syntax error near unexpected token     -bash  command substitution  line 42    lt      Then I tried      useless sh 2 gt   gt   ERROR    cat  lt         This Is Output   echo  ERROR      ERROR is empty   However      useless sh 2 gt   gt   cat  lt     gt  asdf txt   This Is Output   cat asdf txt This Is Error   So the process substitution is doing generally the right thing    unfortunately  whenever I wrap STDIN inside  gt     with something in     in an attempt to capture that to a variable  I lose the contents of      I think that this is because     launches a sub process which no longer has access to the file descriptor in  dev fd which is owned by the parent process   Process substitution has bought me the ability to work with a data stream which is no longer in STDERR  unfortunately I don t seem to be able to manipulate it the way that I want

User · Answer

For error proofing your commands   execute  INVOKING-FUNCTION   COMMAND      execute          function    1       command    2       error   eval    command   2 gt  amp 1  gt   dev null        if        -ne 0    then         echo    function     error           exit 1     fi       Inspired in Lean manufacturing    Make errors impossible by design Make steps the smallest Finish items one by one Make it obvious to anyone

User · Answer

Here s how I did it         1 - name of the  global  variable where the contents of stderr will be stored    2 - command to be executed   captureStderr         local tmpFile   mktemp        2 2 gt   tmpFile      eval   1    lt   tmpFile        rm  tmpFile     Usage example    captureStderr err    useless sh   echo - err-   It does use a temporary file  But at least the ugly stuff is wrapped in a function

User · Answer

A simple solution    ERROR     useless sh 2 gt  amp 1 1 gt  amp  out      out  gt  amp 1 echo  -  echo  ERROR   Will produce   This Is Output - This Is Error

User · Answer

b      a     echo stdout echo stderr  gt  amp 2      2 gt  amp 1     echo  a  gt  a b  gt  b  a  gt stdout b  gt stderr

User · Answer

command receives its input from stdin    command sends its output to stdout  exec 3 gt  amp 1 stderr    command  lt  dev stdin 2 gt  amp 1 1 gt  amp 3   exitcode        echo  STDERR   stderr  exit   exitcode

User · Answer

Capture AND Print stderr  ERROR      useless sh 3 gt  amp 1 1 gt  amp 2 2 gt  amp 3   tee  dev fd 2     Breakdown  You can use     to capture stdout  but you want to capture stderr instead   So you swap stdout and stderr   Using fd 3 as the temporary storage in the standard swap algorithm   If you want to capture AND print use tee to make a duplicate   In this case the output of tee will be captured by     rather than go to the console  but stderr of tee  will still go to the console so we use that as the second output for tee via the special file  dev fd 2 since tee expects a file path rather than a fd number   NOTE  That is an awful lot of redirections in a single line and the order matters       is grabbing the stdout of tee at the end of the pipeline and the pipeline itself routes stdout of   useless sh to the stdin of tee AFTER we swapped stdin and stdout for   useless sh   Using stdout of   useless sh  The OP said he still wanted to use  not just print  stdout  like   useless sh   sed  s Output Useless     No problem just do it BEFORE swapping stdout and stderr   I recommend moving it into a function or file  also-useless sh  and calling that in place of   useless sh in the line above   However  if you want to CAPTURE stdout AND stderr  then I think you have to fall back on temporary files because     will only do one at a time and it makes a subshell from which you cannot return variables

User · Answer

It would be neater to capture the error file thus   ERROR    lt  tmp Error    The shell recognizes this and doesn t have to run  cat  to get the data   The bigger question is hard   I don t think there s an easy way to do it   You d have to build the entire pipeline into the sub-shell  eventually sending its final standard output to a file  so that you can redirect the errors to standard output   ERROR        useless sh   sed s Output Useless   gt  outfile    2 gt  amp 1     Note that the semi-colon is needed  in classic shells - Bourne  Korn - for sure  probably in Bash too    The      does I O redirection over the enclosed commands   As written  it would capture errors from sed too      WARNING  Formally untested code - use at own risk

User · Answer

POSIX STDERR can be captured with some redirection magic      error        ls -ld  XXXX  bin   tr o Z     1 gt  amp 3     2 gt  amp 1     3 gt  amp 1 lrwxrwxrwx 1 rZZt rZZt 7 Aug 22 15 44  bin - gt  usr bin     echo  error ls  cannot access   XXXX   No such file or directory  Note that piping of STDOUT of the command  here ls  is done inside the innermost      If you re executing a simple command  eg  not a pipe   you could remove these inner braces  You can t pipe outside the command as piping makes a subshell in bash and zsh  and the assignment to the variable in the subshell wouldn t be available to the current shell  bash In bash  it would be better not to assume that file descriptor 3 is unused    error        ls -ld  XXXX  bin   tr o Z     1 gt  amp  tmp     2 gt  amp 1      tmp  gt  amp 1   exec  tmp  gt  amp -    With this syntax the FD stays open  Note that this doesn t work in zsh   Thanks to this answer for the general idea

[bash] How to store standard error in a variable

Examples related to bash

Examples related to shell

Examples related to redirect

Examples related to variables

Examples related to stderr