logger configuration to log to file and print to stdout

Question

I m using Python s logging module to log some debug strings to a file which works pretty well  Now in addition  I d like to use this module to also print the strings out to stdout  How do I do this  In order to log my strings to a file I use following code   import logging import logging handlers logger   logging getLogger     logger setLevel logging DEBUG  handler   logging handlers RotatingFileHandler      LOGFILE  maxBytes  1048576 5   backupCount 7   formatter   logging Formatter    asctime s -   name s -   levelname s -   message s   handler setFormatter formatter  logger addHandler handler    and then call a logger function like  logger debug  I am written to the file     Thank you for some help here

User · Answer

For 2 7  try the following     fh   logging handlers RotatingFileHandler LOGFILE  maxBytes  1048576 5   backupCount 7

User · Answer

Logging to stdout and rotating file with different levels and formats   import logging import logging handlers import sys  if   name         main            Change root logger level from WARNING  default  to NOTSET in order for all messages to be delegated      logging getLogger   setLevel logging NOTSET         Add stdout handler  with level INFO     console   logging StreamHandler sys stdout      console setLevel logging INFO      formater   logging Formatter    name -13s    levelname -8s   message s       console setFormatter formater      logging getLogger   addHandler console         Add file rotating handler  with level DEBUG     rotatingHandler   logging handlers RotatingFileHandler filename  rotating log   maxBytes 1000  backupCount 5      rotatingHandler setLevel logging DEBUG      formatter   logging Formatter    asctime s -   name s -   levelname s -   message s       rotatingHandler setFormatter formatter      logging getLogger   addHandler rotatingHandler       log   logging getLogger  app       name         log debug  Debug message  should only appear in the file        log info  Info message  should appear in file and stdout        log warning  Warning message  should appear in file and stdout        log error  Error message  should appear in file and stdout

User · Answer

Here is a complete  nicely wrapped solution based on Waterboy s answer and various other sources  It supports logging to both console and log file  allows for different log level settings  provides colorized output and is easily configurable  also available as Gist      usr bin env python3   - - coding  utf-8 - -    -------------------------------------------------------------------------------                                                                                 -    Python dual-logging setup  console and log file                              -    supporting different log levels and colorized output                         -                                                                                 -    Created by Fonic  lt https   github com fonic gt                                   -    Date  04 05 20                                                               -                                                                                 -    Based on                                                                     -    https   stackoverflow com a 13733863 1976617                                 -    https   uran198 github io en python 2016 07 12 colorful-python-logging html  -    https   en wikipedia org wiki ANSI escape code Colors                        -                                                                                 -   -------------------------------------------------------------------------------    Imports import os import sys import logging    Logging formatter supporting colored output class LogFormatter logging Formatter        COLOR CODES             logging CRITICAL   quot  033 1 35m quot     bright bold magenta         logging ERROR      quot  033 1 31m quot     bright bold red         logging WARNING    quot  033 1 33m quot     bright bold yellow         logging INFO       quot  033 0 37m quot     white   light gray         logging DEBUG      quot  033 1 30m quot     bright bold black   dark gray            RESET CODE    quot  033 0m quot       def   init   self  color   args    kwargs           super LogFormatter  self    init    args    kwargs          self color   color      def format self  record   args    kwargs           if  self color    True and record levelno in self COLOR CODES               record color on    self COLOR CODES record levelno              record color off   self RESET CODE         else              record color on     quot  quot              record color off    quot  quot          return super LogFormatter  self  format record   args    kwargs     Setup logging def setup logging console log output  console log level  console log color  logfile file  logfile log level  logfile log color  log line template          Create logger       For simplicity  we use the root logger  i e  call  logging getLogger          without name argument  This way we can simply use module methods for       for logging throughout the script  An alternative would be exporting       the logger  i e   global logger  logger   logging getLogger  quot  lt name gt  quot        logger   logging getLogger          Set global log level to  debug   required for handler levels to work      logger setLevel logging DEBUG         Create console handler     console log output   console log output lower       if  console log output     quot stdout quot            console log output   sys stdout     elif  console log output     quot stderr quot            console log output   sys stderr     else          print  quot Failed to set console output  invalid output    s  quot    console log output          return False     console handler   logging StreamHandler console log output         Set console log level     try          console handler setLevel console log level upper      only accepts uppercase level names     except          print  quot Failed to set console log level  invalid level    s  quot    console log level          return False        Create and set formatter  add console handler to logger     console formatter   LogFormatter fmt log line template  color console log color      console handler setFormatter console formatter      logger addHandler console handler         Create log file handler     try          logfile handler   logging FileHandler logfile file      except Exception as exception          print  quot Failed to set up log file   s quot    str exception           return False        Set log file log level     try          logfile handler setLevel logfile log level upper      only accepts uppercase level names     except          print  quot Failed to set log file log level  invalid level    s  quot    logfile log level          return False        Create and set formatter  add log file handler to logger     logfile formatter   LogFormatter fmt log line template  color logfile log color      logfile handler setFormatter logfile formatter      logger addHandler logfile handler         Success     return True    Main function def main           Setup logging     script name   os path splitext os path basename sys argv 0    0      if  not setup logging console log output  quot stdout quot   console log level  quot warning quot   console log color True                          logfile file script name    quot  log quot   logfile log level  quot debug quot   logfile log color False                          log line template  quot   color on s   created d     threadName s     levelname -8s    message s  color off s quot             print  quot Failed to setup logging  aborting  quot           return 1        Log some messages     logging debug  quot Debug message quot       logging info  quot Info message quot       logging warning  quot Warning message quot       logging error  quot Error message quot       logging critical  quot Critical message quot      Call main function if    name       quot   main   quot        sys exit main      NOTE regarding Microsoft Windows 10  For colors to actually appear on Microsoft Windows 10  ANSI terminal mode has to be enabled first  Here is a function to do just that    Enable ANSI terminal on Microsoft Windows 10   https   stackoverflow com a 36760881 1976617   https   docs microsoft com en-us windows console setconsolemode def windows enable ansi terminal        if  sys platform     quot win32 quot            return None     try          import ctypes         kernel32   ctypes windll kernel32         result   kernel32 SetConsoleMode kernel32 GetStdHandle -11   7          if  result    0   raise Exception         return True     except          return False

User · Answer

Either run basicConfig with stream sys stdout as the argument prior to setting up any other handlers or logging any messages  or manually add a StreamHandler that pushes messages to stdout to the root logger  or any other logger you want  for that matter

User · Answer

logging basicConfig   can take a keyword argument handlers since Python 3 3  which simplifies logging setup a lot  especially when setting up multiple handlers with the same formatter      handlers  ndash  If specified  this should be an iterable of already created handlers to add to the root logger  Any handlers which don   t already have a formatter set will be assigned the default formatter created in this function    The whole setup can therefore be done with a single call like this   import logging  logging basicConfig      level logging INFO      format    asctime s    levelname s    message s       handlers           logging FileHandler  debug log            logging StreamHandler              Or with import sys   StreamHandler sys stdout  per original question s requirements     the default for StreamHandler is to write to stderr  Look at LogRecord attributes if you want to customize the log format and add things like filename line  thread info etc    The setup above needs to be done only once near the beginning of the script  You can use the logging from all other places in the codebase later like this   logging info  Useful message   logging error  Something bad happened         Note  If it doesn t work  someone else has probably already initialized the logging system differently  Comments suggest doing logging root handlers      before the call to basicConfig

User · Answer

After having used Waterboy s code over and over in multiple Python packages  I finally cast it into a tiny standalone Python package  which you can find here   https   github com acschaefer duallog  The code is well documented and easy to use  Simply download the  py file and include it in your project  or install the whole package via pip install duallog

User · Answer

Just get a handle to the root logger and add the StreamHandler  The StreamHandler writes to stderr  Not sure if you really need stdout over stderr  but this is what I use when I setup the Python logger and I also add the FileHandler as well  Then all my logs go to both places  which is what it sounds like you want    import logging logging getLogger   addHandler logging StreamHandler      If you want to output to stdout instead of stderr  you just need to specify it to the StreamHandler constructor   import sys       logging getLogger   addHandler logging StreamHandler sys stdout     You could also add a Formatter to it so all your log lines have a common header   ie   import logging logFormatter   logging Formatter    asctime s    threadName -12 12s     levelname -5 5s     message s   rootLogger   logging getLogger    fileHandler   logging FileHandler   0   1  log  format logPath  fileName   fileHandler setFormatter logFormatter  rootLogger addHandler fileHandler   consoleHandler   logging StreamHandler   consoleHandler setFormatter logFormatter  rootLogger addHandler consoleHandler    Prints to the format of   2012-12-05 16 58 26 618  MainThread     INFO    my message

User · Answer

Adding a StreamHandler without arguments goes to stderr instead of stdout  If some other process has a dependency on the stdout dump  i e  when writing an NRPE plugin   then make sure to specify stdout explicitly or you might run into some unexpected troubles   Here s a quick example reusing the assumed values and LOGFILE from the question   import logging from logging handlers import RotatingFileHandler from logging import handlers import sys  log   logging getLogger     log setLevel logging DEBUG  format   logging Formatter    asctime s -   name s -   levelname s -   message s    ch   logging StreamHandler sys stdout  ch setFormatter format  log addHandler ch   fh   handlers RotatingFileHandler LOGFILE  maxBytes  1048576 5   backupCount 7  fh setFormatter format  log addHandler fh

[python] logger configuration to log to file and print to stdout

Examples related to python

Examples related to file

Examples related to logging

Examples related to stdout