New PEP to cover problems with being to implement contextlib.nested() properly

pep-0377.txt

@@ -0,0 +1,199 @@
+PEP: 377
+Title: Allow __enter__() methods to skip the statement body
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan@gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 8-Mar-2009
+Python-Version: 2.7, 3.1
+Post-History: 8-Mar-2009
+
+
+Abstract
+========
+
+This PEP proposes a backwards compatible mechanism that allows ``__enter__()``
+methods to skip the body of the associated ``with`` statment. The lack of
+this ability currently means the ``contextlib.nested`` context manager
+is unable to fulfil its specification of being equivalent to writing out
+multiple nested ``with`` statements [1].
+
+The proposed change is to introduce a new flow control exception
+``SkipStatement``, and skip the execution of the ``with``
+statement body if ``__enter__()`` raises this exception.
+
+
+Proposed Change
+===============
+
+The semantics of the ``with`` statement will be changed to include a
+new ``try``/``except``/``else`` block around the call to ``__enter__()``.
+If ``SkipStatement`` is raised by the ``__enter__()`` method, then
+the main section of the ``with`` statement (now located in the ``else``
+clause) will not be executed. To avoid leaving the names in any ``as``
+clause unbound in this case, a new ``StatementSkipped`` singleton
+(similar to the existing ``NotImplemented`` singleton) will be
+assigned to all names that appear in the ``as`` clause.
+
+The components of the ``with`` statement remain as described in PEP 343 [2]::
+
+    with EXPR as VAR:
+        BLOCK
+
+After the modification, the ``with`` statement semantics would
+be as follows::
+
+    mgr = (EXPR)
+    exit = mgr.__exit__  # Not calling it yet
+    try:
+        value = mgr.__enter__()
+    except SkipStatement:
+        VAR = StatementSkipped
+        # Only if "as VAR" is present and
+        # VAR is a single name
+        # If VAR is a tuple of names, then StatementSkipped
+        # will be assigned to each name in the tuple
+    else:
+        exc = True
+        try:
+            try:
+                VAR = value  # Only if "as VAR" is present
+                BLOCK
+            except:
+                # The exceptional case is handled here
+                exc = False
+                if not exit(*sys.exc_info()):
+                    raise
+                # The exception is swallowed if exit() returns true
+        finally:
+            # The normal and non-local-goto cases are handled here
+            if exc:
+                exit(None, None, None)
+
+With the above change in place for the ``with`` statement semantics,
+``contextlib.contextmanager()`` will then be modified to raise
+``SkipStatement`` instead of ``RuntimeError`` when the underlying
+generator doesn't yield.
+
+Rationale for Change
+====================
+
+Currently, some apparently innocuous context managers may raise
+``RuntimeError`` when executed. This occurs when the context
+manager's ``__enter__()`` method encounters a situation where
+the written out version of the code corresponding to the
+context manager would skip the code that is now the body
+of the ``with`` statement. Since the ``__enter__()`` method
+has no mechanism available to signal this to the interpreter,
+it is instead forced to raise an exception that not only
+skips the body of the ``with`` statement, but also jumps over
+all code until the nearest exception handler. This goes against
+one of the design goals of the ``with`` statement, which was to
+be able to factor out arbitrary common exception handling code
+into a single context manager by putting into a generator
+function and replacing the variant part of the code with a
+``yield`` statement.
+
+Specifically, the following examples behave differently if
+``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
+then handles and suppresses::
+
+  with cmA():
+    with cmB():
+      do_stuff()
+  # This will resume here without executing "do_stuff()"
+
+  @contextlib.contextmanager
+  def combined():
+    with cmA():
+      with cmB():
+        yield
+
+  with combined():
+    do_stuff()
+  # This will raise a RuntimeError complaining that the context
+  # manager's underlying generator didn't yield
+
+  with contextlib.nested(cmA(), cmB()):
+    do_stuff()
+  # This will raise the same RuntimeError as the contextmanager()
+  # example (unsurprising, given that the nested() implementation
+  # uses contextmanager())
+
+  # The following class based version shows that the issue isn't
+  # specific to contextlib.contextmanager() (it also shows how
+  # much simpler it is to write context managers as generators
+  # instead of as classes!)
+  class CM(object):
+    def __init__(self):
+        self.cmA = None
+        self.cmB = None
+
+    def __enter__(self):
+        if self.cmA is not None:
+           raise RuntimeError("Can't re-use this CM")
+        self.cmA = cmA()
+        self.cmA.__enter__()
+        try:
+          self.cmB = cmB()
+          self.cmB.__enter__()
+        except:
+          self.cmA.__exit__(*sys.exc_info())
+          # Can't suppress in __enter__(), so must raise
+          raise
+
+    def __exit__(self, *args):
+        suppress = False
+        try:
+          if self.cmB is not None:
+            suppress = self.cmB.__exit__(*args)
+        except:
+          suppress = self.cmA.__exit__(*sys.exc_info()):
+          if not suppress:
+            # Exception has changed, so reraise explicitly
+            raise
+        else:
+          if suppress:
+             # cmB already suppressed the exception,
+             # so don't pass it to cmA
+            suppress = self.cmA.__exit__(None, None, None):
+          else:
+            suppress = self.cmA.__exit__(*args):
+        return suppress
+
+
+Reference Implementation
+========================
+
+In work.
+
+
+Acknowledgements
+================
+
+James William Pye both raised the issue and suggested the solution
+described in this PEP.
+
+References
+==========
+
+.. [1] Issue 5251: contextlib.nested inconsistent with nested with statements
+   (http://bugs.python.org/issue5251)
+
+.. [2] PEP 343: The "with" Statement
+   (http://www.python.org/dev/peps/pep-0343/)
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   End: