How do I log a Python error with debug information

Question

I am printing Python exception messages to a log file with logging error   import logging try      1 0 except ZeroDivisionError as e      logging error e     ERROR root division by zero   Is it possible to print more detailed information about the exception and the code that generated it than just the exception string  Things like line numbers or stack traces would be great

User · Answer

If  quot debugging information quot  means the values present when exception was raised  then logging exception      won t help  So you ll need a tool that logs all variable values along with the traceback lines automatically  Out of the box you ll get log like 2020-03-30 18 24 31 main ERROR   File  quot   temp py quot   line 13  in get ratio 2020-03-30 18 24 31 main ERROR     return height   width 2020-03-30 18 24 31 main ERROR       height   300 2020-03-30 18 24 31 main ERROR       width   0 2020-03-30 18 24 31 main ERROR builtins ZeroDivisionError  division by zero  Have a look at some pypi tools  I d name   tbvaccine traceback-with-variables better-exceptions  Some of them give you pretty crash messages   But you might find some more on pypi

User · Answer

If you can cope with the extra dependency then use twisted log  you don t have to explicitly log errors and also it returns the entire traceback and time to the file or stream

User · Answer

This answer builds up from the above excellent ones   In most applications  you won t be calling logging exception e  directly  Most likely you have defined a custom logger specific for your application or module like this     Set the name of the app or module my logger   logging getLogger  NEM Sequencer     Set the log level my logger setLevel logging INFO     Let s say we want to be fancy and log to a graylog2 log server graylog handler   graypy GELFHandler  some server ip   12201  graylog handler setLevel logging INFO  my logger addHandler graylog handler    In this case  just use the logger to call the exception e  like this   try      1 0 except ZeroDivisionError  e      my logger exception e

User · Answer

If you use plain logs - all your log records should correspond this rule  one record   one line  Following this rule you can use grep and other tools to process your log files   But traceback information is multi-line  So my answer is an extended version of solution proposed by zangw above in this thread  The problem is that traceback lines could have  n inside  so we need to do an extra work to get rid of this line endings   import logging   logger   logging getLogger  your logger here    def log app error e  BaseException  level logging ERROR  - gt  None      e traceback   traceback format exception e   class    e  e   traceback        traceback lines          for line in  line rstrip   n   for line in e traceback           traceback lines extend line splitlines        logger log level  traceback lines   str        After that  when you ll be analyzing your logs  you could copy   paste required traceback lines from your log file and do this   ex traceback     line 1    line 2        for line in ex traceback      print line    Profit

User · Answer

In your logging module if custom module  just enable stack info   api logger exceptionLog   Input your Custom error message   stack info True

User · Answer

Quoting     What if your application does logging some other way     not using the logging module    Now  traceback could be used here   import traceback  def log traceback ex  ex traceback None       if ex traceback is None          ex traceback   ex   traceback       tb lines     line rstrip   n   for line in                  traceback format exception ex   class    ex  ex traceback       exception logger log tb lines     Use it in Python 2   try        your function call is here except Exception as ex            ex traceback   sys exc info       log traceback ex  ex traceback   Use it in Python 3   try      x   get number   except Exception as ex      log traceback ex

User · Answer

A clean way to do it is using format exc   and then parse the output to get the relevant part   from traceback import format exc  try      1 0 except Exception      print  the relevant part is    format exc   split   n   -2    Regards

User · Answer

logger exception will output a stack trace alongside the error message  For example  import logging try      1 0 except ZeroDivisionError      logging exception  quot message quot    Output  ERROR root message Traceback  most recent call last     File  quot  lt stdin gt  quot   line 2  in  lt module gt  ZeroDivisionError  integer division or modulo by zero   Paulo Cheque notes   quot be aware that in Python 3 you must call the logging exception method just inside the except part  If you call this method in an arbitrary place you may get a bizarre exception  The docs alert about that  quot

User · Answer

If you look at the this code example  which works for Python 2 and 3  you ll see the function definition below which can extract  method line number code context file path  for an entire stack trace  whether or not there has been an exception  def sentry friendly trace get last exception True       try          current call   list map frame trans  traceback extract stack             alert frame   current call -4          before call   current call  -4           err type  err  tb   sys exc info   if get last exception else  None  None  None          after call    alert frame  if err type is None else extract all sentry frames from exception tb           return before call   after call  err  alert frame     except          return None  None  None  Of course  this function depends on the entire gist linked above  and in particular extract all sentry frames from exception   and frame trans   but the exception info extraction totals less than around 60 lines  Hope that helps

User · Answer

One nice thing about logging exception that SiggyF s answer doesn t show is that you can pass in an arbitrary message  and logging will still show the full traceback with all the exception details   import logging try      1 0 except ZeroDivisionError      logging exception  Deliberate divide by zero traceback     With the default  in recent versions  logging behaviour of just printing errors to sys stderr  it looks like this    gt  gt  gt  import logging  gt  gt  gt  try          1 0     except ZeroDivisionError          logging exception  Deliberate divide by zero traceback        ERROR root Deliberate divide by zero traceback Traceback  most recent call last     File   lt stdin gt    line 2  in  lt module gt  ZeroDivisionError  integer division or modulo by zero

User · Answer

A little bit of decorator treatment  very loosely inspired by the Maybe monad and lifting   You can safely remove Python 3 6 type annotations and use an older message formatting style   fallible py  from functools import wraps from typing import Callable  TypeVar  Optional import logging   A   TypeVar  A     def fallible  exceptions  logger None            - gt  Callable  Callable      A    Callable      Optional A                  param exceptions  a list of exceptions to catch      param logger  pass a custom logger  None means the default logger                      False disables logging altogether              def fwrap f  Callable      A   - gt  Callable      Optional A              wraps f          def wrapped  args    kwargs               try                  return f  args    kwargs              except exceptions                  message   f called  f  with  args  args  and   kwargs  kwargs                   if logger                      logger exception message                  if logger is None                      logging exception message                  return None          return wrapped      return fwrap   Demo   In  1  from fallible import fallible  In  2    fallible ArithmeticError           def div a  b                return a   b                      In  3   div 1  2  Out 3   0 5  In  4   res   div 1  0  ERROR root called  lt function div at 0x10d3c6ae8 gt  with  args  1  0  and   kwargs    Traceback  most recent call last     File   Users user fallible py   line 17  in wrapped     return f  args    kwargs    File   lt ipython-input-17-e056bd886b5c gt    line 3  in div     return a   b  In  5   repr res   None    You can also modify this solution to return something a bit more meaningful than None from the except part  or even make the solution generic  by specifying this return value in fallible s arguments

User · Answer

Using exc info options may be better  to allow you to choose the error level  if you use exception  it will always be at the error level     try        do something here except Exception as e      logging critical e  exc info True     log exception info at CRITICAL log level

User · Answer

You can log the stack trace without an exception   https   docs python org 3 library logging html logging Logger debug     The second optional keyword argument is stack info  which defaults to False  If true  stack information is added to the logging message  including the actual logging call  Note that this is not the same stack information as that displayed through specifying exc info  The former is stack frames from the bottom of the stack up to the logging call in the current thread  whereas the latter is information about stack frames which have been unwound  following an exception  while searching for exception handlers    Example    gt  gt  gt  import logging  gt  gt  gt  logging basicConfig level logging DEBUG   gt  gt  gt  logging getLogger   info  This prints the stack   stack info True  INFO root This prints the stack Stack  most recent call last     File   lt stdin gt    line 1  in  lt module gt   gt  gt  gt

[python] How do I log a Python error with debug information?

Examples related to python

Examples related to exception

Examples related to logging

Examples related to exception-handling