python-peps/pep-0377.txt

PEP: 377
Title: Allow __enter__() methods to skip the statement body
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Mar-2009
Python-Version: 2.7, 3.1
Post-History: 8-Mar-2009


Abstract
========

This PEP proposes a backwards compatible mechanism that allows ``__enter__()``
methods to skip the body of the associated ``with`` statment. The lack of
this ability currently means the ``contextlib.nested`` context manager
is unable to fulfil its specification of being equivalent to writing out
multiple nested ``with`` statements [1].

The proposed change is to introduce a new flow control exception
``SkipStatement``, and skip the execution of the ``with``
statement body if ``__enter__()`` raises this exception.


Proposed Change
===============

The semantics of the ``with`` statement will be changed to include a
new ``try``/``except``/``else`` block around the call to ``__enter__()``.
If ``SkipStatement`` is raised by the ``__enter__()`` method, then
the main section of the ``with`` statement (now located in the ``else``
clause) will not be executed. To avoid leaving the names in any ``as``
clause unbound in this case, a new ``StatementSkipped`` singleton
(similar to the existing ``NotImplemented`` singleton) will be
assigned to all names that appear in the ``as`` clause.

The components of the ``with`` statement remain as described in PEP 343 [2]::

    with EXPR as VAR:
        BLOCK

After the modification, the ``with`` statement semantics would
be as follows::

    mgr = (EXPR)
    exit = mgr.__exit__  # Not calling it yet
    try:
        value = mgr.__enter__()
    except SkipStatement:
        VAR = StatementSkipped
        # Only if "as VAR" is present and
        # VAR is a single name
        # If VAR is a tuple of names, then StatementSkipped
        # will be assigned to each name in the tuple
    else:
        exc = True
        try:
            try:
                VAR = value  # Only if "as VAR" is present
                BLOCK
            except:
                # The exceptional case is handled here
                exc = False
                if not exit(*sys.exc_info()):
                    raise
                # The exception is swallowed if exit() returns true
        finally:
            # The normal and non-local-goto cases are handled here
            if exc:
                exit(None, None, None)

With the above change in place for the ``with`` statement semantics,
``contextlib.contextmanager()`` will then be modified to raise
``SkipStatement`` instead of ``RuntimeError`` when the underlying
generator doesn't yield.


Rationale for Change
====================

Currently, some apparently innocuous context managers may raise
``RuntimeError`` when executed. This occurs when the context
manager's ``__enter__()`` method encounters a situation where
the written out version of the code corresponding to the
context manager would skip the code that is now the body
of the ``with`` statement. Since the ``__enter__()`` method
has no mechanism available to signal this to the interpreter,
it is instead forced to raise an exception that not only
skips the body of the ``with`` statement, but also jumps over
all code until the nearest exception handler. This goes against
one of the design goals of the ``with`` statement, which was to
be able to factor out arbitrary common exception handling code
into a single context manager by putting into a generator
function and replacing the variant part of the code with a
``yield`` statement.

Specifically, the following examples behave differently if
``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
then handles and suppresses::

  with cmA():
    with cmB():
      do_stuff()
  # This will resume here without executing "do_stuff()"

  @contextlib.contextmanager
  def combined():
    with cmA():
      with cmB():
        yield

  with combined():
    do_stuff()
  # This will raise a RuntimeError complaining that the context
  # manager's underlying generator didn't yield

  with contextlib.nested(cmA(), cmB()):
    do_stuff()
  # This will raise the same RuntimeError as the contextmanager()
  # example (unsurprising, given that the nested() implementation
  # uses contextmanager())

  # The following class based version shows that the issue isn't
  # specific to contextlib.contextmanager() (it also shows how
  # much simpler it is to write context managers as generators
  # instead of as classes!)
  class CM(object):
    def __init__(self):
        self.cmA = None
        self.cmB = None

    def __enter__(self):
        if self.cmA is not None:
           raise RuntimeError("Can't re-use this CM")
        self.cmA = cmA()
        self.cmA.__enter__()
        try:
          self.cmB = cmB()
          self.cmB.__enter__()
        except:
          self.cmA.__exit__(*sys.exc_info())
          # Can't suppress in __enter__(), so must raise
          raise

    def __exit__(self, *args):
        suppress = False
        try:
          if self.cmB is not None:
            suppress = self.cmB.__exit__(*args)
        except:
          suppress = self.cmA.__exit__(*sys.exc_info()):
          if not suppress:
            # Exception has changed, so reraise explicitly
            raise
        else:
          if suppress:
             # cmB already suppressed the exception,
             # so don't pass it to cmA
            suppress = self.cmA.__exit__(None, None, None):
          else:
            suppress = self.cmA.__exit__(*args):
        return suppress


Reference Implementation
========================

In work.


Acknowledgements
================

James William Pye both raised the issue and suggested the solution
described in this PEP.

References
==========

.. [1] Issue 5251: contextlib.nested inconsistent with nested with statements
   (http://bugs.python.org/issue5251)

.. [2] PEP 343: The "with" Statement
   (http://www.python.org/dev/peps/pep-0343/)

Copyright
=========

This document has been placed in the public domain.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: