[python] Catching an exception while using a Python 'with' statement

Differentiating between the possible origins of exceptions raised from a compound with statement

Differentiating between exceptions that occur in a with statement is tricky because they can originate in different places. Exceptions can be raised from either of the following places (or functions called therein):

• ContextManager.__init__
• ContextManager.__enter__
• the body of the with
• ContextManager.__exit__

For more details see the documentation about Context Manager Types.

If we want to distinguish between these different cases, just wrapping the with into a try .. except is not sufficient. Consider the following example (using ValueError as an example but of course it could be substituted with any other exception type):

try:
    with ContextManager():
        BLOCK
except ValueError as err:
    print(err)

Here the except will catch exceptions originating in all of the four different places and thus does not allow to distinguish between them. If we move the instantiation of the context manager object outside the with, we can distinguish between __init__ and BLOCK / __enter__ / __exit__:

try:
    mgr = ContextManager()
except ValueError as err:
    print('__init__ raised:', err)
else:
    try:
        with mgr:
            try:
                BLOCK
            except TypeError:  # catching another type (which we want to handle here)
                pass
    except ValueError as err:
        # At this point we still cannot distinguish between exceptions raised from
        # __enter__, BLOCK, __exit__ (also BLOCK since we didn't catch ValueError in the body)
        pass

Effectively this just helped with the __init__ part but we can add an extra sentinel variable to check whether the body of the with started to execute (i.e. differentiating between __enter__ and the others):

try:
    mgr = ContextManager()  # __init__ could raise
except ValueError as err:
    print('__init__ raised:', err)
else:
    try:
        entered_body = False
        with mgr:
            entered_body = True  # __enter__ did not raise at this point
            try:
                BLOCK
            except TypeError:  # catching another type (which we want to handle here)
                pass
    except ValueError as err:
        if not entered_body:
            print('__enter__ raised:', err)
        else:
            # At this point we know the exception came either from BLOCK or from __exit__
            pass

The tricky part is to differentiate between exceptions originating from BLOCK and __exit__ because an exception that escapes the body of the with will be passed to __exit__ which can decide how to handle it (see the docs). If however __exit__ raises itself, the original exception will be replaced by the new one. To deal with these cases we can add a general except clause in the body of the with to store any potential exception that would have otherwise escaped unnoticed and compare it with the one caught in the outermost except later on - if they are the same this means the origin was BLOCK or otherwise it was __exit__ (in case __exit__ suppresses the exception by returning a true value the outermost except will simply not be executed).

try:
    mgr = ContextManager()  # __init__ could raise
except ValueError as err:
    print('__init__ raised:', err)
else:
    entered_body = exc_escaped_from_body = False
    try:
        with mgr:
            entered_body = True  # __enter__ did not raise at this point
            try:
                BLOCK
            except TypeError:  # catching another type (which we want to handle here)
                pass
            except Exception as err:  # this exception would normally escape without notice
                # we store this exception to check in the outer `except` clause
                # whether it is the same (otherwise it comes from __exit__)
                exc_escaped_from_body = err
                raise  # re-raise since we didn't intend to handle it, just needed to store it
    except ValueError as err:
        if not entered_body:
            print('__enter__ raised:', err)
        elif err is exc_escaped_from_body:
            print('BLOCK raised:', err)
        else:
            print('__exit__ raised:', err)

Alternative approach using the equivalent form mentioned in PEP 343

PEP 343 -- The "with" Statement specifies an equivalent "non-with" version of the with statement. Here we can readily wrap the various parts with try ... except and thus differentiate between the different potential error sources:

import sys

try:
    mgr = ContextManager()
except ValueError as err:
    print('__init__ raised:', err)
else:
    try:
        value = type(mgr).__enter__(mgr)
    except ValueError as err:
        print('__enter__ raised:', err)
    else:
        exit = type(mgr).__exit__
        exc = True
        try:
            try:
                BLOCK
            except TypeError:
                pass
            except:
                exc = False
                try:
                    exit_val = exit(mgr, *sys.exc_info())
                except ValueError as err:
                    print('__exit__ raised:', err)
                else:
                    if not exit_val:
                        raise
        except ValueError as err:
            print('BLOCK raised:', err)
        finally:
            if exc:
                try:
                    exit(mgr, None, None, None)
                except ValueError as err:
                    print('__exit__ raised:', err)

Usually a simpler approach will do just fine

The need for such special exception handling should be quite rare and normally wrapping the whole with in a try ... except block will be sufficient. Especially if the various error sources are indicated by different (custom) exception types (the context managers need to be designed accordingly) we can readily distinguish between them. For example:

try:
    with ContextManager():
        BLOCK
except InitError:  # raised from __init__
    ...
except AcquireResourceError:  # raised from __enter__
    ...
except ValueError:  # raised from BLOCK
    ...
except ReleaseResourceError:  # raised from __exit__
    ...