How to duplicate sys stdout to a log file

Question

Edit  Since it appears that there s either no solution  or I m doing something so non-standard that nobody knows - I ll revise my question to also ask  What is the best way to accomplish logging when a python app is making a lot of system calls   My app has two modes   In interactive mode  I want all output to go to the screen as well as to a log file  including output from any system calls   In daemon mode  all output goes to the log   Daemon mode works great using os dup2     I can t find a way to  tee  all output to a log in interactive mode  without modifying each and every system call     In other words  I want the functionality of the command line  tee  for any output generated by a python app  including system call output   To clarify    To redirect all output I do something like this  and it works great     open our log file so   se   open   s log    self name   w   0     re-open stdout without buffering sys stdout   os fdopen sys stdout fileno     w   0     redirect stdout and stderr to the log file opened above os dup2 so fileno    sys stdout fileno    os dup2 se fileno    sys stderr fileno      The nice thing about this is that it requires no special print calls from the rest of the code   The code also runs some shell commands  so it s nice not having to deal with each of their output individually as well   Simply  I want to do the same  except duplicating instead of redirecting   At first thought  I thought that simply reversing the dup2 s should work   Why doesn t it   Here s my test    import os  sys      my broken solution  so   se   open  a log    w   0  sys stdout   os fdopen sys stdout fileno     w   0   os dup2 sys stdout fileno    so fileno    os dup2 sys stderr fileno    se fileno         print  foo bar    os spawnve  P WAIT     bin ls      bin ls        os execve   bin ls      bin ls    os environ    The file  a log  should be identical to what was displayed on the screen

User · Answer

The print statement will call the write   method of any object you assign to sys stdout     I would spin up a small class to write to two places at once     import sys  class Logger object       def   init   self           self terminal   sys stdout         self log   open  log dat    a        def write self  message           self terminal write message          self log write message     sys stdout   Logger     Now the print statement will both echo to the screen and append to your log file     prints  1 2  to  lt stdout gt  AND log dat print   d  d     1 2    This is obviously quick-and-dirty   Some notes    You probably ought to parametize the log filename  You should probably revert sys stdout to  lt stdout gt  if you won t be logging for the duration of the program  You may want the ability to write to multiple log files at once  or handle different log levels  etc    These are all straightforward enough that I m comfortable leaving them as exercises for the reader   The key insight here is that print just calls a  file-like object  that s assigned to sys stdout

User · Answer

I wrote a tee   implementation in Python that should work for most cases  and it works on Windows also   https   github com pycontribs tendo  Also  you can use it in combination with logging module from Python if you want

User · Answer

I m writing a script to run cmd-line scripts     Because in some cases  there just is no viable substitute for a Linux command -- such as the case of rsync     What I really wanted was to use the default python logging mechanism in every case where it was possible to do so  but to still capture any error when something went wrong that was unanticipated   This code seems to do the trick   It may not be particularly elegant or efficient   although it doesn t use string  string  so at least it doesn t have that particular potential bottle- neck     I m posting it in case it gives someone else any useful ideas   import logging import os  sys import datetime    Get name of module  use as application name try    ME os path split   file    -1  split      0  except    ME  pyExec    LOG IDENTIFIER  uuu     o O     uuu   LOG IDR LENGTH len LOG IDENTIFIER   class PyExec object        Use this to capture all possible error   output to log   class SuperTee object           Original reference  http   mail python org pipermail python-list 2007-May 442737 html       def   init   self  name  mode             self fl   open name  mode            self fl write   n             self stdout   sys stdout           self stdout write   n             self stderr   sys stderr            sys stdout   self           sys stderr   self        def   del   self             self fl write   n             self fl flush             sys stderr   self stderr           sys stdout   self stdout           self fl close          def write self  data               If the data to write includes the log identifier prefix  then it is already formatted           if data 0 LOG IDR LENGTH   LOG IDENTIFIER              self fl write   s n    data LOG IDR LENGTH                self stdout write data LOG IDR LENGTH                 Otherwise  we can give it a timestamp           else               timestamp str datetime datetime now                if  Traceback     data 0 9                 data   s   s     timestamp  data                self fl write data              else                self fl write data               self stdout write data      def   init   self  aName  aCmd  logFileName     outFileName             Using name for  logger   context    which is separate from the module or the function     baseFormatter logging Formatter    asctime s  t   levelname s  t   name s   module s   lineno d  t   message s       errorFormatter logging Formatter LOG IDENTIFIER      asctime s  t   levelname s  t   name s   module s   lineno d  t   message s        if logFileName          open passed filename as append       fl logging FileHandler   s log    aName      else          otherwise  use log filename as a one-time use file       fl logging FileHandler   s log    aName   w        fl setLevel logging DEBUG      fl setFormatter baseFormatter         This will capture stdout and CRITICAL and beyond errors      if outFileName        teeFile PyExec SuperTee   s out log    aName      else        teeFile PyExec SuperTee   s out log    aName   w        fl out logging StreamHandler  teeFile       fl out setLevel logging CRITICAL      fl out setFormatter errorFormatter         Set up logging     self log logging getLogger  pyExec main       log self log      log addHandler fl      log addHandler fl out       print  Test print statement        log setLevel logging DEBUG       log info  Starting  s   ME      log critical  Critical           Caught exception     try        raise Exception  Exception test        except Exception e        log exception str e          Uncaught exception     a 2 0   PyExec  test pyExec  None    Obviously  if you re not as subject to whimsy as I am  replace LOG IDENTIFIER with another string that you re not like to ever see someone write to a log

User · Answer

I wrote a full replacement for sys stderr and just duplicated the code renaming stderr to stdout to make it also available to replace sys stdout   To do this I create the same object type as the current stderr and stdout  and forward all methods to the original system stderr and stdout   import os import sys import logging  class StdErrReplament object                   How to redirect stdout and stderr to logger in Python         https   stackoverflow com questions 19425736 how-to-redirect-stdout-and-stderr-to-logger-in-python          Set a Read-Only Attribute in Python          https   stackoverflow com questions 24497316 set-a-read-only-attribute-in-python             is active   False       classmethod     def lock cls  logger                           Attach this singleton logger to the  sys stderr  permanently                      global  stderr singleton         global  stderr default         global  stderr default class type            On Sublime Text   sys   stderr    is set to None  because they already replaced  sys stderr            by some   LogWriter    class  then just save the current one over there          if not sys   stderr                sys   stderr     sys stderr          try               stderr default              stderr default class type          except NameError               stderr default   sys stderr              stderr default class type   type   stderr default              Recreate the sys stderr logger when it was reset by  unlock            if not cls is active              cls is active   True              stderr write    stderr default write              logger call   logger debug             clean formatter   logger clean formatter              global  sys stderr write             global  sys stderr write hidden              if sys version info  lt    3 2                   logger file handler terminator     n                 Always recreate override the internal write function used by   sys stderr write              def  sys stderr write hidden  args    kwargs                                           Suppress newline in Python logging module                     https   stackoverflow com questions 7168790 suppress-newline-in-python-logging-module                                      try                       stderr write   args    kwargs                       file handler   logger file handler                      formatter   file handler formatter                     terminator   file handler terminator                      file handler formatter   clean formatter                     file handler terminator                           kwargs  extra        duplicated from file   True                      logger call   args    kwargs                        file handler formatter   formatter                     file handler terminator   terminator                  except Exception                      logger exception   Could not write to the file handler   s  s    file handler  logger                       cls unlock                  Only create one   sys stderr write  function pointer ever             try                   sys stderr write              except NameError                   def  sys stderr write  args    kwargs                                                   Hides the actual function pointer  This allow the external function pointer to                         be cached while the internal written can be exchanged between the standard                          sys stderr write  and our custom wrapper around it                                               sys stderr write hidden   args    kwargs            try                Only create one singleton instance ever              stderr singleton          except NameError               class StdErrReplamentHidden  stderr default class type                                           Which special methods bypasses   getattribute   in Python                      https   stackoverflow com questions 12872695 which-special-methods-bypasses-getattribute-in-python                                      if hasattr   stderr default     abstractmethods                             abstractmethods      stderr default   abstractmethods                    if hasattr   stderr default     base                             base      stderr default   base                    if hasattr   stderr default     bases                             bases      stderr default   bases                    if hasattr   stderr default     basicsize                             basicsize      stderr default   basicsize                    if hasattr   stderr default     call                             call      stderr default   call                    if hasattr   stderr default     class                             class      stderr default   class                    if hasattr   stderr default     delattr                             delattr      stderr default   delattr                    if hasattr   stderr default     dict                             dict      stderr default   dict                    if hasattr   stderr default     dictoffset                             dictoffset      stderr default   dictoffset                    if hasattr   stderr default     dir                             dir      stderr default   dir                    if hasattr   stderr default     doc                             doc      stderr default   doc                    if hasattr   stderr default     eq                             eq      stderr default   eq                    if hasattr   stderr default     flags                             flags      stderr default   flags                    if hasattr   stderr default     format                             format      stderr default   format                    if hasattr   stderr default     ge                             ge      stderr default   ge                    if hasattr   stderr default     getattribute                             getattribute      stderr default   getattribute                    if hasattr   stderr default     gt                             gt      stderr default   gt                    if hasattr   stderr default     hash                             hash      stderr default   hash                    if hasattr   stderr default     init                             init      stderr default   init                    if hasattr   stderr default     init subclass                             init subclass      stderr default   init subclass                    if hasattr   stderr default     instancecheck                             instancecheck      stderr default   instancecheck                    if hasattr   stderr default     itemsize                             itemsize      stderr default   itemsize                    if hasattr   stderr default     le                             le      stderr default   le                    if hasattr   stderr default     lt                             lt      stderr default   lt                    if hasattr   stderr default     module                             module      stderr default   module                    if hasattr   stderr default     mro                             mro      stderr default   mro                    if hasattr   stderr default     name                             name      stderr default   name                    if hasattr   stderr default     ne                             ne      stderr default   ne                    if hasattr   stderr default     new                             new      stderr default   new                    if hasattr   stderr default     prepare                             prepare      stderr default   prepare                    if hasattr   stderr default     qualname                             qualname      stderr default   qualname                    if hasattr   stderr default     reduce                             reduce      stderr default   reduce                    if hasattr   stderr default     reduce ex                             reduce ex      stderr default   reduce ex                    if hasattr   stderr default     repr                             repr      stderr default   repr                    if hasattr   stderr default     setattr                             setattr      stderr default   setattr                    if hasattr   stderr default     sizeof                             sizeof      stderr default   sizeof                    if hasattr   stderr default     str                             str      stderr default   str                    if hasattr   stderr default     subclasscheck                             subclasscheck      stderr default   subclasscheck                    if hasattr   stderr default     subclasses                             subclasses      stderr default   subclasses                    if hasattr   stderr default     subclasshook                             subclasshook      stderr default   subclasshook                    if hasattr   stderr default     text signature                             text signature      stderr default   text signature                    if hasattr   stderr default     weakrefoffset                             weakrefoffset      stderr default   weakrefoffset                    if hasattr   stderr default   mro                         mro    stderr default mro                  def   init   self                                                   Override any super class  type   stderr default    constructor  so we can                          instantiate any kind of  sys stderr  replacement object  in case it was already                          replaced by something else like on Sublime Text with   LogWriter                              Assures all attributes were statically replaced just above  This should happen in case                         some new attribute is added to the python language                           This also ignores the only two methods which are not equal     init      and    getattribute                                                   different methods       init        getattribute                         attributes to check   set  dir  object     dir  type                          for attribute in attributes to check                           if attribute not in different methods                                   and hasattr   stderr default  attribute                                 base class attribute   super   stderr default class type  self     getattribute    attribute                               target class attribute    stderr default   getattribute    attribute                                if base class attribute    target class attribute                                  sys stderr write       The base class attribute   s  is different from the target class  n s n s n n                                              attribute  base class attribute  target class attribute                      def   getattribute   self  item                        if item     write                           return  sys stderr write                      try                          return  stderr default   getattribute    item                        except AttributeError                          return super   stderr default class type   stderr default     getattribute    item                 stderr singleton   StdErrReplamentHidden               sys stderr    stderr singleton          return cls       classmethod     def unlock cls                           Detach this  stderr  writer from  sys stderr  and allow the next call to  lock    create             a new writer for the stderr                       if cls is active              global  sys stderr write hidden              cls is active   False              sys stderr write hidden    stderr default write    class StdOutReplament object                   How to redirect stdout and stderr to logger in Python         https   stackoverflow com questions 19425736 how-to-redirect-stdout-and-stderr-to-logger-in-python          Set a Read-Only Attribute in Python          https   stackoverflow com questions 24497316 set-a-read-only-attribute-in-python             is active   False       classmethod     def lock cls  logger                           Attach this singleton logger to the  sys stdout  permanently                      global  stdout singleton         global  stdout default         global  stdout default class type            On Sublime Text   sys   stdout    is set to None  because they already replaced  sys stdout            by some   LogWriter    class  then just save the current one over there          if not sys   stdout                sys   stdout     sys stdout          try               stdout default              stdout default class type          except NameError               stdout default   sys stdout              stdout default class type   type   stdout default              Recreate the sys stdout logger when it was reset by  unlock            if not cls is active              cls is active   True              stdout write    stdout default write              logger call   logger debug             clean formatter   logger clean formatter              global  sys stdout write             global  sys stdout write hidden              if sys version info  lt    3 2                   logger file handler terminator     n                 Always recreate override the internal write function used by   sys stdout write              def  sys stdout write hidden  args    kwargs                                           Suppress newline in Python logging module                     https   stackoverflow com questions 7168790 suppress-newline-in-python-logging-module                                      try                       stdout write   args    kwargs                       file handler   logger file handler                      formatter   file handler formatter                     terminator   file handler terminator                      file handler formatter   clean formatter                     file handler terminator                           kwargs  extra        duplicated from file   True                      logger call   args    kwargs                        file handler formatter   formatter                     file handler terminator   terminator                  except Exception                      logger exception   Could not write to the file handler   s  s    file handler  logger                       cls unlock                  Only create one   sys stdout write  function pointer ever             try                   sys stdout write              except NameError                   def  sys stdout write  args    kwargs                                                   Hides the actual function pointer  This allow the external function pointer to                         be cached while the internal written can be exchanged between the standard                          sys stdout write  and our custom wrapper around it                                               sys stdout write hidden   args    kwargs            try                Only create one singleton instance ever              stdout singleton          except NameError               class StdOutReplamentHidden  stdout default class type                                           Which special methods bypasses   getattribute   in Python                      https   stackoverflow com questions 12872695 which-special-methods-bypasses-getattribute-in-python                                      if hasattr   stdout default     abstractmethods                             abstractmethods      stdout default   abstractmethods                    if hasattr   stdout default     base                             base      stdout default   base                    if hasattr   stdout default     bases                             bases      stdout default   bases                    if hasattr   stdout default     basicsize                             basicsize      stdout default   basicsize                    if hasattr   stdout default     call                             call      stdout default   call                    if hasattr   stdout default     class                             class      stdout default   class                    if hasattr   stdout default     delattr                             delattr      stdout default   delattr                    if hasattr   stdout default     dict                             dict      stdout default   dict                    if hasattr   stdout default     dictoffset                             dictoffset      stdout default   dictoffset                    if hasattr   stdout default     dir                             dir      stdout default   dir                    if hasattr   stdout default     doc                             doc      stdout default   doc                    if hasattr   stdout default     eq                             eq      stdout default   eq                    if hasattr   stdout default     flags                             flags      stdout default   flags                    if hasattr   stdout default     format                             format      stdout default   format                    if hasattr   stdout default     ge                             ge      stdout default   ge                    if hasattr   stdout default     getattribute                             getattribute      stdout default   getattribute                    if hasattr   stdout default     gt                             gt      stdout default   gt                    if hasattr   stdout default     hash                             hash      stdout default   hash                    if hasattr   stdout default     init                             init      stdout default   init                    if hasattr   stdout default     init subclass                             init subclass      stdout default   init subclass                    if hasattr   stdout default     instancecheck                             instancecheck      stdout default   instancecheck                    if hasattr   stdout default     itemsize                             itemsize      stdout default   itemsize                    if hasattr   stdout default     le                             le      stdout default   le                    if hasattr   stdout default     lt                             lt      stdout default   lt                    if hasattr   stdout default     module                             module      stdout default   module                    if hasattr   stdout default     mro                             mro      stdout default   mro                    if hasattr   stdout default     name                             name      stdout default   name                    if hasattr   stdout default     ne                             ne      stdout default   ne                    if hasattr   stdout default     new                             new      stdout default   new                    if hasattr   stdout default     prepare                             prepare      stdout default   prepare                    if hasattr   stdout default     qualname                             qualname      stdout default   qualname                    if hasattr   stdout default     reduce                             reduce      stdout default   reduce                    if hasattr   stdout default     reduce ex                             reduce ex      stdout default   reduce ex                    if hasattr   stdout default     repr                             repr      stdout default   repr                    if hasattr   stdout default     setattr                             setattr      stdout default   setattr                    if hasattr   stdout default     sizeof                             sizeof      stdout default   sizeof                    if hasattr   stdout default     str                             str      stdout default   str                    if hasattr   stdout default     subclasscheck                             subclasscheck      stdout default   subclasscheck                    if hasattr   stdout default     subclasses                             subclasses      stdout default   subclasses                    if hasattr   stdout default     subclasshook                             subclasshook      stdout default   subclasshook                    if hasattr   stdout default     text signature                             text signature      stdout default   text signature                    if hasattr   stdout default     weakrefoffset                             weakrefoffset      stdout default   weakrefoffset                    if hasattr   stdout default   mro                         mro    stdout default mro                  def   init   self                                                   Override any super class  type   stdout default    constructor  so we can                          instantiate any kind of  sys stdout  replacement object  in case it was already                          replaced by something else like on Sublime Text with   LogWriter                              Assures all attributes were statically replaced just above  This should happen in case                         some new attribute is added to the python language                           This also ignores the only two methods which are not equal     init      and    getattribute                                                   different methods       init        getattribute                         attributes to check   set  dir  object     dir  type                          for attribute in attributes to check                           if attribute not in different methods                                   and hasattr   stdout default  attribute                                 base class attribute   super   stdout default class type  self     getattribute    attribute                               target class attribute    stdout default   getattribute    attribute                                if base class attribute    target class attribute                                  sys stdout write       The base class attribute   s  is different from the target class  n s n s n n                                              attribute  base class attribute  target class attribute                      def   getattribute   self  item                        if item     write                           return  sys stdout write                      try                          return  stdout default   getattribute    item                        except AttributeError                          return super   stdout default class type   stdout default     getattribute    item                 stdout singleton   StdOutReplamentHidden               sys stdout    stdout singleton          return cls       classmethod     def unlock cls                           Detach this  stdout  writer from  sys stdout  and allow the next call to  lock    create             a new writer for the stdout                       if cls is active              global  sys stdout write hidden              cls is active   False              sys stdout write hidden    stdout default write   To use this you can just call StdErrReplament  lock logger  and StdOutReplament  lock logger  passing the logger you want to use to send the output text  For example   import os import sys import logging  current folder   os path dirname  os path realpath    file       log file path   os path join  current folder   my log file txt     file handler   logging FileHandler  log file path   a    file handler formatter   logging Formatter     asctime s   name s   levelname s -   message s     Y- m- d  H  M  S     log   logging getLogger    name     log setLevel   DEBUG    log addHandler  file handler    log file handler   file handler log clean formatter   logging Formatter            StdOutReplament lock  log   StdErrReplament lock  log    log debug   I am doing usual logging debug       sys stderr write   Tests 1    n    sys stdout write   Tests 2    n      Running this code  you will see on the screen      And on the file contents     If you would like to also see the contents of the log debug calls on the screen  you will need to add a stream handler to your logger  On this case it would be like this   import os import sys import logging  class ContextFilter logging Filter           This filter avoids duplicated information to be displayed to the StreamHandler log          def filter self  record           return not   duplicated from file  in record   dict    current folder   os path dirname  os path realpath    file       log file path   os path join  current folder   my log file txt     stream handler   logging StreamHandler   file handler   logging FileHandler  log file path   a     formatter   logging Formatter     asctime s   name s   levelname s -   message s     Y- m- d  H  M  S    file handler formatter   formatter stream handler formatter   formatter stream handler addFilter  ContextFilter      log   logging getLogger    name     log setLevel   DEBUG    log addHandler  file handler   log addHandler  stream handler    log file handler   file handler log stream handler   stream handler log clean formatter   logging Formatter            StdOutReplament lock  log   StdErrReplament lock  log    log debug   I am doing usual logging debug       sys stderr write   Tests 1    n    sys stdout write   Tests 2    n      Which would output like this when running     While it would still saving this to the file my log file txt     When disabling this with StdErrReplament unlock    it will only restore the standard behavior of the stderr stream  as the attached logger cannot be never detached because someone else can have a reference to its older version  This is why it is a global singleton which can never dies  Therefore  in case of reloading this module with imp or something else  it will never recapture the current sys stderr as it was already injected on it and have it saved internally

User · Answer

another solution using logging module   import logging import sys  log   logging getLogger  stdxxx    class StreamLogger object        def   init   self  stream  prefix              self stream   stream         self prefix   prefix         self data           def write self  data           self stream write data          self stream flush            self data    data         tmp   str self data          if   x0a  in tmp or   x0d  in tmp              tmp   tmp rstrip   x0a x0d               log info   s s     self prefix  tmp               self data        logging basicConfig level logging INFO                      filename  text log                       filemode  a    sys stdout   StreamLogger sys stdout    stdout      print  test for stdout

User · Answer

As per a request by  user5359531 in the comments under  John T s answer  here s a copy of the referenced post to the revised version of the linked discussion in that answer   Issue of redirecting the stdout to both file and screen Gabriel Genellina gagsl-py2 at yahoo com ar Mon May 28 12 45 51 CEST 2007      Previous message  Issue of redirecting the stdout to both file and screen     Next message  Formal interfaces with Python     Messages sorted by    date     thread     subject     author    En Mon  28 May 2007 06 17 39 -0300                   lt kelvin you at gmail com gt  escribi      gt  I wanna print the log to both the screen and file  so I simulatered a  gt   tee   gt   gt  class Tee file    gt   gt      def   init   self  name  mode    gt          file   init   self  name  mode   gt          self stdout   sys stdout  gt          sys stdout   self  gt   gt      def   del   self    gt          sys stdout   self stdout  gt          self close    gt   gt      def write self  data    gt          file write self  data   gt          self stdout write data   gt   gt  Tee  logfile    w    gt  print  gt  gt sys stdout   abcdefg   gt   gt  I found that it only output to the file  nothing to screen  Why   gt  It seems the  write  function was not called when I  print  something   You create a Tee instance and it is immediately garbage collected  I d restore sys stdout on Tee close  not   del    you forgot to call the inherited   del   method  btw   Mmm  doesn t work  I think there is an optimization somewhere  if it looks like a real file object  it uses the original file write method  not yours  The trick would be to use an object that does NOT inherit from file   import sys class TeeNoFile object        def   init   self  name  mode            self file   open name  mode           self stdout   sys stdout          sys stdout   self      def close self            if self stdout is not None               sys stdout   self stdout              self stdout   None          if self file is not None               self file close                self file   None      def write self  data            self file write data           self stdout write data       def flush self            self file flush            self stdout flush        def   del   self            self close    tee TeeNoFile  logfile    w   print  abcdefg  print  another line  tee close   print  screen only  del tee   should do nothing  -- Gabriel Genellina

User · Answer

What you really want is logging module from standard library   Create a logger and attach two handlers  one would be writing to a file and the other to stdout or stderr   See  Logging to multiple destinations for details

User · Answer

You can also add stderr as well  based on  shx2 s answer above  using class multifile   class Log object        def   init   self  path log  mode  quot w quot   encoding  quot utf-8 quot            h   open path log  mode  encoding encoding          sys stdout   multifile   sys stdout  h            sys stderr   multifile   sys stderr  h         def   enter   self            quot  quot  quot  Necessary if called by with  or with    as   quot  quot  quot          return self       only necessary if  quot as quot       def   exit   self  type  value  tb            quot  quot  quot  Necessary if call by with  quot  quot  quot          pass      def   del   self           if sys is not None                restoring             sys stdout   sys   stdout               sys stderr   sys   stderr    log   Log  quot test txt quot   print  quot line 1 quot   print  quot line 2 quot   file sys stderr  del log print  quot line 3 only on screen quot

User · Answer

Since you re comfortable spawning external processes from your code  you could use tee itself   I don t know of any Unix system calls that do exactly what tee does     Note this version was written circa Python 2 6  see below for   an updated 3 3 -compatible version  import subprocess  os  sys    Unbuffer output  this ensures the output is in the correct order  sys stdout   os fdopen sys stdout fileno     w   0   tee   subprocess Popen   tee    log txt    stdin subprocess PIPE  os dup2 tee stdin fileno    sys stdout fileno    os dup2 tee stdin fileno    sys stderr fileno     print   nstdout  print  gt  gt sys stderr   stderr  os spawnve  P WAIT     bin ls      bin ls        os execve   bin ls      bin ls    os environ    You could also emulate tee using the multiprocessing package  or use processing if you re using Python 2 5 or earlier    Update  Here is a Python 3 3 -compatible version   import subprocess  os  sys  tee   subprocess Popen   tee    log txt    stdin subprocess PIPE    Cause tee s stdin to get a copy of our stdin stdout  as well as that   of any child processes we spawn  os dup2 tee stdin fileno    sys stdout fileno    os dup2 tee stdin fileno    sys stderr fileno       The flush flag is needed to guarantee these lines are written before   the two spawned  bin ls processes emit any output print   nstdout   flush True  print  stderr   file sys stderr  flush True     These child processes  stdin stdout are  os spawnve  P WAIT     bin ls      bin ls        os execve   bin ls      bin ls    os environ

User · Answer

I know this question has been answered repeatedly  but for this I ve taken the main answer from John T s answer and modified it so it contains the suggested flush and followed its linked revised version  I ve also added the enter and exit as mentioned in cladmi s answer for use with the with statement  In addition  the documentation mentions to flush files using os fsync   so I ve added that as well  I don t know if you really need that but its there   import sys  os  class Logger object        Lumberjack class - duplicates sys stdout to a log file and it s okay       source  https   stackoverflow com q 616645     def   init   self  filename  Red Wood   mode  a   buff 0           self stdout   sys stdout         self file   open filename  mode  buff          sys stdout   self      def   del   self           self close        def   enter   self           pass      def   exit   self   args           self close        def write self  message           self stdout write message          self file write message       def flush self           self stdout flush           self file flush           os fsync self file fileno         def close self           if self stdout    None              sys stdout   self stdout             self stdout   None          if self file    None              self file close               self file   None   You can then use it  with Logger  My best girlie by my side        print  we d sing sing sing     or   Log Logger  Sleeps all night   print  works all day   Log close

User · Answer

I had this same issue before and found this snippet very useful   class Tee object       def   init   self  name  mode           self file   open name  mode          self stdout   sys stdout         sys stdout   self     def   del   self           sys stdout   self stdout         self file close       def write self  data           self file write data          self stdout write data      def flush self           self file flush     from  http   mail python org pipermail python-list 2007-May 438106 html

User · Answer

Ah  just re-read your question and see that this doesn t quite apply    Here is a sample program that makes uses the python logging module   This logging module has been in all versions since 2 3   In this sample the logging is configurable by command line options     In quite mode it will only log to a file  in normal mode it will log to both a file and the console   import os import sys import logging from optparse import OptionParser  def initialize logging options           Log information based upon users options         logger   logging getLogger  project       formatter   logging Formatter    asctime s   levelname s t  message s       level   logging   dict   get options loglevel upper   logging DEBUG      logger setLevel level         Output logging information to screen     if not options quiet          hdlr   logging StreamHandler sys stderr          hdlr setFormatter formatter          logger addHandler hdlr         Output logging information to file     logfile   os path join options logdir   project log       if options clean and os path isfile logfile           os remove logfile      hdlr2   logging FileHandler logfile      hdlr2 setFormatter formatter      logger addHandler hdlr2       return logger  def main argv None       if argv is None          argv   sys argv 1          Setup command line options     parser   OptionParser  usage   prog  options        parser add option  -l    --logdir   dest  logdir   default      help  log DIRECTORY  default           parser add option  -v    --loglevel   dest  loglevel   default  debug   help  logging level  debug  info  error        parser add option  -q    --quiet   action  store true   dest  quiet   help  do not log to console       parser add option  -c    --clean   dest  clean   action  store true   default False  help  remove old log file          Process command line options      options  args    parser parse args argv         Setup logger format and output locations     logger   initialize logging options         Examples     logger error  This is an error message        logger info  This is an info message        logger debug  This is a debug message     if   name         main         sys exit main

User · Answer

Here is another solution  which is more general than the others -- it supports splitting output  written to sys stdout  to any number of file-like objects  There s no requirement that   stdout   itself is included   import sys  class multifile object       def   init   self  files           self  files   files     def   getattr   self  attr   args           return self  wrap attr   args      def  wrap self  attr   args           def g  a    kw               for f in self  files                  res   getattr f  attr   args   a    kw              return res         return g    for a tee-like behavior  use like this  sys stdout   multifile   sys stdout  open  myfile txt    w         all these forms work  print  abc  print  gt  gt sys stdout   line2  sys stdout write  line3 n     NOTE  This is a proof-of-concept  The implementation here is not complete  as it only wraps methods of the file-like objects  e g  write   leaving out members properties setattr  etc   However  it is probably good enough for most people as it currently stands   What I like about it  other than its generality  is that it is clean in the sense it doesn t make any direct calls to write  flush  os dup2  etc

User · Answer

To complete John T answer  https   stackoverflow com a 616686 395687  I added   enter   and   exit   methods to use it as a context manager with the with keyword  which gives this code  class Tee object       def   init   self  name  mode           self file   open name  mode          self stdout   sys stdout         sys stdout   self      def   del   self           sys stdout   self stdout         self file close        def write self  data           self file write data          self stdout write data       def   enter   self           pass      def   exit   self   type   value   traceback           pass   It can then be used as  with Tee  outfile log    w        print  I am written to both stdout and outfile log

User · Answer

As described elsewhere  perhaps the best solution is to use the logging module directly   import logging  logging basicConfig level logging DEBUG  filename  mylog log   logging info  this should to write to the log file     However  there are some  rare  occasions where you really want to redirect stdout  I had this situation when I was extending django s runserver command which uses print  I didn t want to hack the django source but needed the print statements to go to a file   This is a way of redirecting stdout and stderr away from the shell using the logging module   import logging  sys  class LogFile object          File-like object to log text using the  logging  module          def   init   self  name None           self logger   logging getLogger name       def write self  msg  level logging INFO           self logger log level  msg       def flush self           for handler in self logger handlers              handler flush    logging basicConfig level logging DEBUG  filename  mylog log      Redirect stdout and stderr sys stdout   LogFile  stdout   sys stderr   LogFile  stderr    print  this should to write to the log file    You should only use this LogFile implementation if you really cannot use the logging module directly

User · Answer

If you wish to log all output to a file AND output it to a text file then you can do the following  It s a bit hacky but it works   import logging debug   input  Debug or not   if debug     1       logging basicConfig level logging DEBUG  filename    OUT txt       old print   print     def print string           old print string          logging info string  print  OMG it works      EDIT  Note that this does not log errors unless you redirect sys stderr to sys stdout  EDIT2  A second issue is that you have to pass 1 argument unlike with the builtin function   EDIT3  See the code before to write stdin and stdout to console and file with stderr only going to file  import logging  sys debug   input  Debug or not   if debug     1       old input   input     sys stderr write   logging info     def input string              string in   old input string          logging info  STRING IN     string in          return string in     logging basicConfig level logging DEBUG  filename    OUT txt       old print   print     def print string     string2              old print string  string2          logging info string          logging info string2  print  OMG   b   input   print a     Deliberate error for testing

User · Answer

None of the answers above really seems to answer the problem posed   I know this is an old thread  but I think this problem is a lot simpler than everyone is making it   class tee err object     def   init   self       self errout   sys stderr      sys stderr   self      self log    logfile log      log   open self log  w       log close     def write self  line        log   open self log  a       log write line      log close           self errout write line    Now this will repeat everything to the normal sys stderr handler and your file   Create another class tee out for sys stdout

[python] How to duplicate sys.stdout to a log file?

Examples related to python

Examples related to tee