How do I write stderr to a file while using tee with a pipe

Question

I know how to use tee to write the output  STDOUT  of aaa sh to bbb out  while still displaying it in the terminal     aaa sh   tee bbb out   How would I now also write STDERR to a file named ccc out  while still having it displayed

User · Answer

If you re using zsh  you can use multiple redirections  so you don t even need tee     cmd 1 gt  amp 1 2 gt  amp 2 1 gt out file 2 gt err file   Here you re simply redirecting each stream to itself and the target file     Full example     echo  out   echo  err  gt  dev stderr  1 gt  amp 1 2 gt  amp 2 1 gt  tmp out file 2 gt  tmp err file out err   cat  tmp out file out   cat  tmp err file err   Note that this requires the MULTIOS option to be set  which is the default       MULTIOS       Perform implicit tees or cats when multiple redirections are attempted  see Redirection

User · Answer

In my case  a script was running command while redirecting both stdout and stderr to a file  something like   cmd  gt  log 2 gt  amp 1   I needed to update it such that when there is a failure  take some actions based on the error messages  I could of course remove the dup 2 gt  amp 1 and capture the stderr from the script  but then the error messages won t go into the log file for reference  While the accepted answer from  lhunath is supposed to do the same  it redirects stdout and stderr to different files  which is not what I want  but it helped me to come up with the exact solution that I need    cmd 2 gt   gt  tee  dev stderr    gt  log   With the above  log will have a copy of both stdout and stderr and I can capture stderr from my script without having to worry about stdout

User · Answer

The following will work for KornShell ksh  where the process substitution is not available     create a combined stdin and stdout  collector exec 3  lt  gt  combined log    stream stderr instead of stdout to tee  while draining all stdout to the collector   aaa sh 2 gt  amp 1 1 gt  amp 3   tee -a stderr log 1 gt  amp 3    cleanup collector exec 3 gt  amp -   The real trick here  is the sequence of the 2 gt  amp 1 1 gt  amp 3 which in our case redirects the stderr to stdout and redirects the stdout to descriptor 3  At this point the stderr and stdout are not combined yet   In effect  the stderr as stdin  is passed to tee where it logs to stderr log and also redirects to descriptor 3   And descriptor 3 is logging it to combined log all the time  So the combined log contains both stdout and stderr

User · Answer

If using bash     Redirect standard out and standard error separately   cmd  gt stdout-redirect 2 gt stderr-redirect    Redirect standard error and out together   cmd  gt stdout-redirect 2 gt  amp 1    Merge standard error with standard out and pipe   cmd 2 gt  amp 1  cmd2   Credit  not answering from the top of my head  goes here  http   www cygwin com ml cygwin 2003-06 msg00772 html

User · Answer

This may be useful for people finding this via google   Simply uncomment the example you want to try out   Of course  feel free to rename the output files      bin bash  STATUSFILE x out LOGFILE x log      All output to screen     Do nothing  this is the default       All Output to one file  nothing to the screen  exec  gt    LOGFILE  2 gt  amp 1       All output to one file and all output to the screen  exec  gt   gt  tee   LOGFILE   2 gt  amp 1       All output to one file  STDOUT to the screen  exec  gt   gt  tee -a   LOGFILE   2 gt   gt  tee -a   LOGFILE   gt  dev null        All output to one file  STDERR to the screen     Note you need both of these lines for this to work  exec 3 gt  amp 1  exec  gt   gt  tee -a   LOGFILE   gt  dev null  2 gt   gt  tee -a   LOGFILE   gt  amp 3        STDOUT to STATUSFILE  stderr to LOGFILE  nothing to the screen  exec  gt    STATUSFILE  2 gt   LOGFILE        STDOUT to STATUSFILE  stderr to LOGFILE and all output to the screen  exec  gt   gt  tee   STATUSFILE   2 gt   gt  tee   LOGFILE   gt  amp 2        STDOUT to STATUSFILE and screen  STDERR to LOGFILE  exec  gt   gt  tee   STATUSFILE   2 gt   LOGFILE        STDOUT to STATUSFILE  STDERR to LOGFILE and screen  exec  gt    STATUSFILE  2 gt   gt  tee   LOGFILE   gt  amp 2    echo  This is a test  ls -l sdgshgswogswghthb this file will not exist so we get output to stderr aronkjegralhfaff ls -l   0

User · Answer

Like the accepted answer well explained by lhunath  you can use command  gt   gt  tee -a stdout log  2 gt   gt  tee -a stderr log  gt  amp 2   Beware than if you use bash you could have some issue  Let me take the matthew-wilcoxson exemple   And for those who  quot seeing is believing quot   a quick test   echo  quot Test Out quot   gt  amp 2 echo  quot Test Err quot    gt   gt  tee stdout log  2 gt   gt  tee stderr log  gt  amp 2    Personally  when I try  I have this result   user computer     echo  quot Test Out quot   gt  amp 2 echo  quot Test Err quot    gt   gt  tee stdout log  2 gt   gt  tee stderr log  gt  amp 2  user computer    Test Out Test Err   Both message does not appear at the same level  Why Test Out seem to be put like if it is my previous command   Prompt is on a blank line  let me think the process is not finished  and when I press Enter this fix it  When I check the content of the files  it is ok  redirection works  Let take another test  function outerr       echo  quot out quot        stdout   echo  gt  amp 2  quot err quot    stderr    user computer    outerr out err  user computer    outerr  gt  dev null err  user computer    outerr 2 gt  dev null out  Trying again the redirection  but with this function  function test redirect       fout  quot stdout log quot    ferr  quot stderr log quot    echo  quot   outerr quot     outerr   gt   gt  tee  quot  fout quot   2 gt   gt  tee  quot  ferr quot   gt  amp 2    echo  quot    fout content   quot    cat  quot  fout quot    echo  quot    ferr content   quot    cat  quot  ferr quot     Personally  I have this result   user computer    test redirect   outerr   stdout log content   out out err   stderr log content   err user computer     No prompt on a blank line  but I don t see normal output  stdout log content seem to be wrong  only stderr log seem to be ok  If I relaunch it  output can be different    So  why   Because  like explained here    Beware that in bash  this command returns as soon as  first command  finishes  even if the tee commands are still executed  ksh and zsh do wait for the subprocesses   So  if you use bash  prefer use the better exemple given in this other answer       outerr   tee  quot  fout quot     2 gt  amp 1 1 gt  amp 3   tee  quot  ferr quot     3 gt  amp 1 1 gt  amp 2  It will fix the previous issues  Now  the question is  how to retrieve exit status code      does not works  I have no found better solution than switch on pipefail with set -o pipefail  set  o pipefail to switch off  and use   PIPESTATUS 0   like this function outerr       echo  quot out quot    echo  gt  amp 2  quot err quot    return 11    function test outerr       local -   To preserve set option        -o pipefail     amp  amp  set -o pipefail    Or use second part directly   local fout  quot stdout log quot    local ferr  quot stderr log quot    echo  quot   outerr quot        outerr   tee  quot  fout quot     2 gt  amp 1 1 gt  amp 3   tee  quot  ferr quot     3 gt  amp 1 1 gt  amp 2     First save the status or it will be lost   local status  quot   PIPESTATUS 0   quot    Save first  the second is 0  perhaps tee status code    echo  quot     quot    echo  quot    fout content   quot    echo  quot  lt     quot    cat  quot  fout quot    echo  quot     gt  quot    echo  quot    ferr content   quot    echo  quot  lt     quot    cat  quot  ferr quot    echo  quot     gt  quot    if    status  gt  0     then     echo  quot Fail  status  gt  0 quot      return  quot  status quot    or whatever   fi    user computer    test outerr   outerr err out       stdout log content    lt     out     gt    stderr log content    lt     err     gt  Fail 11  gt  0

User · Answer

In other words  you want to pipe stdout into one filter  tee bbb out  and stderr into another filter  tee ccc out   There is no standard way to pipe anything other than stdout into another command  but you can work around that by juggling file descriptors         aaa sh   tee bbb out    2 gt  amp 1 1 gt  amp 3   tee ccc out    3 gt  amp 1 1 gt  amp 2   See also How to grep standard error stream  stderr   and When would you use an additional file descriptor   In bash  and ksh and zsh   but not in other POSIX shells such as dash  you can use process substitution     aaa sh  gt   gt  tee bbb out  2 gt   gt  tee ccc out    Beware that in bash  this command returns as soon as   aaa sh finishes  even if the tee commands are still executed  ksh and zsh do wait for the subprocesses   This may be a problem if you do something like   aaa sh  gt   gt  tee bbb out  2 gt   gt  tee ccc out   process logs bbb out ccc out  In that case  use file descriptor juggling or ksh zsh instead

User · Answer

why not simply     aaa sh 2 gt  amp 1   tee -a log   This simply redirects stderr to stdout  so tee echoes both to log and to screen   Maybe I m missing something  because some of the other solutions seem really complicated   Note  Since bash version 4 you may use   amp  as an abbreviation for 2 gt  amp 1       aaa sh   amp  tee -a log

User · Answer

To redirect stderr to a file  display stdout to screen  and also save stdout to a file     aaa sh 2 ccc out   tee   bbb out  EDIT   To display both stderr and stdout to screen and also save both to a file  you can use bash s I O redirection      bin bash    Create a new file descriptor 4  pointed at the file   which will receive stderr  exec 4 lt  gt ccc out    Also print the contents of this file to screen  tail -f ccc out  amp     Run the command  tee stdout as normal  and send stderr   to our file descriptor 4    aaa sh 2 gt  amp 4   tee bbb out    Clean up  Close file descriptor 4 and kill tail -f  exec 4 gt  amp - kill  1

User · Answer

I m assuming you want to still see STDERR and STDOUT on the terminal   You could go for Josh Kelley s answer  but I find keeping a tail around in the background which outputs your log file very hackish and cludgy   Notice how you need to keep an exra FD and do cleanup afterward by killing it and technically should be doing that in a trap       EXIT   There is a better way to do this  and you ve already discovered it  tee   Only  instead of just using it for your stdout  have a tee for stdout and one for stderr   How will you accomplish this   Process substitution and file redirection   command  gt   gt  tee -a stdout log  2 gt   gt  tee -a stderr log  gt  amp 2    Let s split it up and explain    gt   gt         gt        process substitution  creates a FIFO and lets tee listen on it   Then  it uses  gt   file redirection  to redirect the STDOUT of command to the FIFO that your first tee is listening on   Same thing for the second   2 gt   gt  tee -a stderr log  gt  amp 2    We use process substitution again to make a tee process that reads from STDIN and dumps it into stderr log   tee outputs its input back on STDOUT  but since its input is our STDERR  we want to redirect tee s STDOUT to our STDERR again   Then we use file redirection to redirect command s STDERR to the FIFO s input  tee s STDIN    See http   mywiki wooledge org BashGuide InputAndOutput  Process substitution is one of those really lovely things you get as a bonus of choosing bash as your shell as opposed to sh  POSIX or Bourne      In sh  you d have to do things manually   out    TMPDIR - tmp  out     err    TMPDIR - tmp  err     mkfifo   out    err  trap  rm   out    err   EXIT tee -a stdout log  lt    out   amp  tee -a stderr log  lt    err   gt  amp 2  amp  command  gt   out  2 gt   err

[linux] How do I write stderr to a file while using "tee" with a pipe?

Examples related to linux

Examples related to bash

Examples related to unix

Examples related to tee