Log exception with traceback

Question

How can I log my Python errors   try      do something   except        How can I log my exception here  complete with its traceback

User · Answer

Here is a version that uses sys excepthook  import traceback import sys  logger   logging getLogger    def handle excepthook type  message  stack        logger error f An unhandled exception occured   message   Traceback   traceback format tb stack      sys excepthook   handle excepthook

User · Answer

This is how I do it  try      do something   except        How can I log my exception here  complete with its traceback      import traceback     traceback format exc     this will print a complete trace to stout

User · Answer

Uncaught exception messages go to STDERR  so instead of implementing your logging in Python itself you could send STDERR to a file using whatever shell you re using to run your Python script  In a Bash script  you can do this with output redirection  as described in the BASH guide   Examples  Append errors to file  other output to the terminal     test py 2 gt  gt  mylog log   Overwrite file with interleaved STDOUT and STDERR output     test py  amp  gt  mylog log

User · Answer

Heres a simple example taken from the python 2 6 documentation   import logging LOG FILENAME     tmp logging example out  logging basicConfig filename LOG FILENAME level logging DEBUG    logging debug  This message should go to the log file

User · Answer

You can log all uncaught exceptions on the main thread by assigning a handler to sys excepthook  perhaps using the exc info parameter of Python s logging functions   import sys import logging  logging basicConfig filename   tmp foobar log    def exception hook exc type  exc value  exc traceback       logging error           Uncaught exception           exc info  exc type  exc value  exc traceback         sys excepthook   exception hook  raise Exception  Boom     If your program uses threads  however  then note that threads created using threading Thread will not trigger sys excepthook when an uncaught exception occurs inside them  as noted in Issue 1230540 on Python s issue tracker  Some hacks have been suggested there to work around this limitation  like monkey-patching Thread   init   to overwrite self run with an alternative run method that wraps the original in a try block and calls sys excepthook from inside the except block  Alternatively  you could just manually wrap the entry point for each of your threads in try except yourself

User · Answer

My job recently tasked me with logging all the tracebacks exceptions from our application  I tried numerous techniques that others had posted online such as the one above but settled on a different approach  Overriding traceback print exception    I have a write up at http   www bbarrows com  That would be much easier to read but Ill paste it in here as well   When tasked with logging all the exceptions that our software might encounter in the wild I tried a number of different techniques to log our python exception tracebacks  At first I thought that the python system exception hook  sys excepthook would be the perfect place to insert the logging code  I was trying something similar to   import traceback import StringIO import logging import os  sys  def my excepthook excType  excValue  traceback  logger logger       logger error  Logging an uncaught exception                    exc info  excType  excValue  traceback    sys excepthook   my excepthook     This worked for the main thread but I soon found that the my sys excepthook would not exist across any new threads my process started  This is a huge issue because most everything happens in threads in this project   After googling and reading plenty of documentation the most helpful information I found was from the Python Issue tracker   The first post on the thread shows a working example of the sys excepthook NOT persisting across threads  as shown below   Apparently this is expected behavior   import sys  threading  def log exception  args       print  got exception  s     args   sys excepthook   log exception  def foo        a   1   0  threading Thread target foo  start     The messages on this Python Issue thread really result in 2 suggested hacks  Either subclass Thread and wrap the run method in our own try except block in order to catch and log exceptions or monkey patch threading Thread run to run in your own try except block and log the exceptions   The first method of subclassing Thread seems to me to be less elegant in your code as you would have to import and use your custom Thread class EVERYWHERE you wanted to have a logging thread  This ended up being a hassle because I had to search our entire code base and replace all normal Threads with this custom Thread  However  it was clear as to what this Thread was doing and would be easier for someone to diagnose and debug if something went wrong with the custom logging code  A custome logging thread might look like this   class TracebackLoggingThread threading Thread       def run self           try              super TracebackLoggingThread  self  run           except  KeyboardInterrupt  SystemExit               raise         except Exception  e              logger   logging getLogger                 logger exception  Logging an uncaught exception     The second method of monkey patching threading Thread run is nice because I could just run it once right after   main   and instrument my logging code in all exceptions  Monkey patching can be annoying to debug though as it changes the expected functionality of something  The suggested patch from the Python Issue tracker was   def installThreadExcepthook                Workaround for sys excepthook thread bug     From http   spyced blogspot com 2007 06 workaround-for-sysexcepthook-bug html   https   sourceforge net tracker  func detail amp atid 105470 amp aid 1230540 amp group id 5470       Call once from   main   before creating any threads      If using psyco  call psyco cannotcompile threading Thread run      since this replaces a new-style class method              init old   threading Thread   init       def init self   args    kwargs           init old self   args    kwargs          run old   self run         def run with except hook  args    kw               try                  run old  args    kw              except  KeyboardInterrupt  SystemExit                   raise             except                  sys excepthook  sys exc info            self run   run with except hook     threading Thread   init     init   It was not until I started testing my exception logging I realized that I was going about it all wrong   To test I had placed a  raise Exception  Test     somewhere in my code  However  wrapping a a method that called this method was a try except block that printed out the traceback and swallowed the exception  This was very frustrating because I saw the traceback bring printed to STDOUT but not being logged  It was I then decided that a much easier method of logging the tracebacks was just to monkey patch the method that all python code uses to print the tracebacks themselves  traceback print exception  I ended up with something similar to the following   def add custom print exception        old print exception   traceback print exception     def custom print exception etype  value  tb  limit None  file None           tb output   StringIO StringIO           traceback print tb tb  limit  tb output          logger   logging getLogger  customLogger           logger error tb output getvalue            tb output close           old print exception etype  value  tb  limit None  file None      traceback print exception   custom print exception   This code writes the traceback to a String Buffer and logs it to logging ERROR  I have a custom logging handler set up the  customLogger  logger which takes the ERROR level logs and send them home for analysis

User · Answer

Use logging exception from within the except  handler block to log the current exception along with the trace information  prepended with a message   import logging LOG FILENAME     tmp logging example out  logging basicConfig filename LOG FILENAME  level logging DEBUG   logging debug  This message should go to the log file    try      run my stuff   except      logging exception  Got exception on main handler       raise   Now looking at the log file   tmp logging example out   DEBUG root This message should go to the log file ERROR root Got exception on main handler Traceback  most recent call last     File   tmp teste py   line 9  in  lt module gt      run my stuff   NameError  name  run my stuff  is not defined

User · Answer

You can get the traceback using a logger  at any level  DEBUG  INFO        Note that using logging exception  the level is ERROR     test app py import sys import logging  logging basicConfig level  DEBUG    def do something        raise ValueError        try      do something   except Exception      logging debug  Something went wrong   exc info sys exc info      DEBUG root Something went wrong Traceback  most recent call last     File  test app py   line 10  in  lt module gt      do something     File  test app py   line 7  in do something     raise ValueError       ValueError       EDIT   This works too  using python 3 6   logging debug  Something went wrong   exc info True

User · Answer

What I was looking for   import sys import traceback  exc type  exc value  exc traceback   sys exc info   traceback in var   traceback format tb exc traceback    See     https   docs python org 3 library traceback html

User · Answer

Use exc info options may be better  remains warning or error title    try        coode in here except Exception as e      logging error e  exc info True

User · Answer

maybe not as stylish  but easier      bin bash log   var log yourlog   path to your script py 2 gt  amp 1    while read  do echo   REPLY   gt  gt   log  done

[python] Log exception with traceback

Examples related to python

Examples related to exception

Examples related to logging

Examples related to error-handling