Establish the "alternative" version. The exception API is called

__exit__(), and its signature is the same as that of the
raise-statement.  Still some loose ends.

pep-0340.txt

@@ -26,30 +26,6 @@ Introduction
     this on python-dev recently [1], and I figured it would be time to
     write up a precise spec in PEP form.
 
-Proposal Evolution
-
-    The discussion on python-dev has changed my mind slightly on how
-    exceptions should be handled, but I don't have the time to do a
-    full update of the PEP right now.  Basically, I'm now in favor of
-    a variation on the exception handling proposed in the section
-    "Alternative __next__() and Generator Exception Handling" below.
-
-    The added twist is that instead of adding a flag argument to
-    next() and __next__() to indicate whether the previous argument is
-    a value or an exception, we use a separate API (an __exit__()
-    method taking an exception and perhaps a traceback) for the
-    exception.  If an iterator doesn't implement __exit__(), the
-    exception is just re-raised.  It is expected that, apart from
-    generators, very few iterators will implement __exit__(); one use
-    case would be a fast implementation of synchronized() written in
-    C.
-
-    The built-in next() function only interfaces to the next() and
-    __next__() methods; there is no user-friendly API to call
-    __exit__().  (Or perhaps calling next(itr, exc, traceback) would
-    call itr.__exit__(exc, traceback) if itr has an __exit__ method
-    and otherwise raise exc.__class__, exc, traceback?)
-
 Motivation and Summary
 
     (Thanks to Shane Hathaway -- Hi Shane!)
@@ -96,40 +72,25 @@ Use Cases
 
     TBD.  For now, see the Examples section near the end.
 
-Specification: the Iteration Exception Hierarchy
-
-    Two new built-in exceptions are defined, and StopIteration is
-    moved in the exception hierarchy:
-
-        class Iteration(Exception):
-            pass
-
-        class StopIteration(Iteration):
-            pass
-
-        class ContinueIteration(Iteration):
-            def __init__(self, value=None):
-                self.value = None
-
 Specification: the __next__() Method
 
     A new method for iterators is proposed, called __next__().  It
-    takes one optional argument, which defaults to None.  If not None,
+    takes one optional argument, which defaults to None.  Calling the
-    the argument must be an Iteration instance.  Calling the
     __next__() method without argument or with None is equivalent to
     using the old iterator API, next().  For backwards compatibility,
     it is recommended that iterators also implement a next() method as
     an alias for calling the __next__() method without an argument.
 
-    Calling the __next__() method with a StopIteration instance
+    The argument to the __next__() method may be used by the iterator
-    signals the iterator that the caller wants to abort the iteration
+    as a hint on what to do next.
-    sequence; the iterator should respond by doing any necessary
+
-    cleanup and raising StopIteration.  Calling it with a
+Specification: the __exit__() Method
-    ContinueIteration instance signals the iterator that the caller
+
-    wants to continue the iteration; the ContinueIteration exception
+    An optional new method for iterators is proposed, called
-    has a 'value' attribute which may be used by the iterator as a
+    __exit__().  It takes up to three arguments which correspond to
-    hint on what to do next.  Calling it with a (base class) Iteration
+    the three "arguments" to the raise-statement: type, value, and
-    instance is the same as calling it with None.
+    traceback.  If all three arguments are None, sys.exc_info() may be
+    consulted to provide suitable default values.
 
 Specification: the next() Built-in Function
 
@@ -143,34 +104,43 @@ Specification: the next() Built-in Function
                 return itr.next()
             raise TypeError("next() with arg for old-style iterator")
 
-Specification: the 'for' Loop
+    This function is proposed because there is often a need to call
+    the next() method outside a for-loop; the new API, and the
+    backwards compatibility code, is too ugly to have to repeat in
+    user code.
+
+    Note that I'm not proposing a built-in function to call the
+    __exit__() method of an iterator.  I don't expect that this will
+    be called much outside the block-statement.
+
+Specification: a Change to the 'for' Loop
 
     A small change in the translation of the for-loop is proposed.
     The statement
 
         for VAR1 in EXPR1:
             BLOCK1
+        else:
+            BLOCK2
 
     will be translated as follows:
 
         itr = iter(EXPR1)
         arg = None
+        brk = False
         while True:
             try:
                 VAR1 = next(itr, arg)
             except StopIteration:
+                brk = True
                 break
             arg = None
             BLOCK1
+        if brk:
+            BLOCK2
 
-    (However, 'itr' and 'arg' are hidden from the user, their scope
+    (However, the variables 'itr' etc. are not user-visible and the
-    ends when the while-loop is exited, and they are not shared with
+    built-in names used cannot be overridden by the user.)
-    nested or outer for-loops, and the user cannot override the
-    built-ins referenced.)
-
-    I'm leaving the translation of an else-clause up to the reader;
-    note that you can't simply affix the else-clause to the while-loop
-    since it is always broken out.
 
 Specification: the Extended 'continue' Statement
 
@@ -180,7 +150,7 @@ Specification: the Extended 'continue' Statement
 
     is legal and is translated into
 
-        arg = ContinueIteration(EXPR2)
+        arg = EXPR2
         continue
 
     (Where 'arg' references the corresponding hidden variable from the
@@ -195,21 +165,28 @@ Specification: the Anonymous Block Statement
 
         block EXPR1 as VAR1:
             BLOCK1
+        else:
+            BLOCK2
 
     Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary
     expression (but not an expression-list) and VAR1 is an arbitrary
     assignment target (which may be a comma-separated list).
 
-    The "as VAR1" part is optional; if omitted, the assignment to VAR1
+    The "as VAR1" part is optional; if omitted, the assignments to
-    in the translation below is omitted (but the next() call is not!).
+    VAR1 in the translation below are omitted (but the expressions
+    assigned are still evaluated!).
 
-    The choice of the 'block' keyword is contentious; it has even been
+    The choice of the 'block' keyword is contentious; many
-    proposed not to use a keyword at all.  PEP 310 uses 'with' for
+    alternatives have been proposed, including not to use a keyword at
-    similar semantics, but I would like to reserve that for a
+    all (which I actually like).  PEP 310 uses 'with' for similar
-    with-statement similar to the one found in Pascal and VB.  To
+    semantics, but I would like to reserve that for a with-statement
-    sidestep this issue momentarily I'm using 'block' until we can
+    similar to the one found in Pascal and VB.  (Though I just found
-    agree on a keyword.  (I just found that the C# designers don't
+    that the C# designers don't like 'with' [2], and I have to agree
-    like 'with' [2].)
+    with their reasoning.)  To sidestep this issue momentarily I'm
+    using 'block' until we can agree on the right keyword, if any.
+
+    Note that the 'as' keyword is not contentious (it will finally be
+    elevated to proper keyword status).
 
     Note that it is left in the middle whether a block-statement
     represents a loop or not; this is up to the iterator, but in the
@@ -218,61 +195,66 @@ Specification: the Anonymous Block Statement
     The translation is subtly different from the translation of a
     for-loop: iter() is not called, so EXPR1 should already be an
     iterator (not just an iterable); and the iterator is guaranteed to
-    be exhausted when the block-statement is left:
+    be notified when the block-statement is left, regardless if this
+    is due to a break, return or exception:
 
-        itr = EXPR1
+        itr = EXPR1  # The iterator
-        val = arg = None
+        ret = False  # True if a return statement is active
-        ret = False
+        val = None   # Return value, if ret == True
+        arg = None   # Argument to __next__() (value from continue)
+        exc = None   # sys.exc_info() tuple if an exception is active
         while True:
             try:
-                VAR1 = next(itr, arg)
+                if exc:
+                    ext = getattr(itr, "__exit__", None)
+                    if ext is not None:
+                        VAR1 = ext(*exc)   # May re-raise *exc
+                    else:
+                        raise *exc         # Well, the moral equivalent :-)
+                else:
+                    VAR1 = next(itr, arg)  # May raise StopIteration
             except StopIteration:
                 if ret:
                     return val
-                if val is not None:
-                    raise val
                 break
             try:
-                val = arg = None
                 ret = False
+                val = arg = exc = None
                 BLOCK1
-            except Exception, val:
+            except:
-                arg = StopIteration()
+                exc = sys.exc_info()
 
-    (Again, 'itr' etc. are hidden, and the user cannot override the
+    (Again, the variables and built-ins are hidden from the user.)
-    built-ins.)
-
-    The "raise val" translation is inexact; this is supposed to
-    re-raise the exact exception that was raised inside BLOCK1, with
-    the same traceback.  We can't use a bare raise-statement because
-    we've just caught StopIteration.
 
     Inside BLOCK1, the following special translations apply:
 
     - "continue" and "continue EXPR2" are always legal; the latter is
       translated as shown earlier:
 
-        arg = ContinueIteration(EXPR2)
+        arg = EXPR2
         continue
 
     - "break" is always legal; it is translated into:
 
-        arg = StopIteration()
+        exc = (StopIteration,)
         continue
 
     - "return EXPR3" is only legal when the block-statement is
       contained in a function definition; it is translated into:
 
-        val = EXPR3
+        exc = (StopIteration,)
         ret = True
-        arg = StopIteration()
+        val = EXPR3
         continue
 
     The net effect is that break, continue and return behave much the
     same as if the block-statement were a for-loop, except that the
     iterator gets a chance at resource cleanup before the
-    block-statement is left.  The iterator also gets a chance if the
+    block-statement is left, through the optional __exit__() method.
-    block-statement is left through raising an exception.
+    The iterator also gets a chance if the block-statement is left
+    through raising an exception.  If the iterator doesn't have an
+    __exit__() method, there is no difference with a for-loop (except
+    that a for-loop calls iter() on EXPR1).
 
     Note that a yield-statement (or a yield-expression, see below) in
     a block-statement is not treated differently.  It suspends the
@@ -287,15 +269,18 @@ Specification: the Anonymous Block Statement
     be resumed eventually.
 
     I haven't decided yet whether the block-statement should also
-    allow an optional else-clause, like the for-loop.  I think it
+    allow an optional else-clause, like the for-loop, but I'm leaning
-    would be confusing, and emphasize the "loopiness" of the
+    against it.  I think it would be confusing, and emphasize the
-    block-statement, while I want to emphasize its *difference* from a
+    "loopiness" of the block-statement, while I want to emphasize its
-    for-loop.
+    *difference* from a for-loop.  In addition, there are several
+    possible semantics for an else-clause.
 
 Specification: Generator Exception Handling
 
     Generators will implement the new __next__() method API, as well
-    as the old argument-less next() method.
+    as the old argument-less next() method which becomes an alias for
+    calling __next__() without an argument.  They will also implement
+    the new __exit__() method API.
 
     Generators will be allowed to have a yield statement inside a
     try-finally statement.
@@ -330,32 +315,41 @@ Specification: Generator Exception Handling
     are all illegal.  (Some of the edge cases are motivated by the
     current legality of "yield 12, 42".)
 
-    When __next__() is called with a StopIteration instance argument,
+    When __exit__() is called, the generator is resumed but at the
-    the yield statement that is resumed by the __next__() call will
+    point of the yield-statement or -expression the exception
-    raise this StopIteration exception.  The generator should re-raise
+    represented by the __exit__ argument(s) is raised.  The generator
-    this exception; it should not yield another value.  When the
+    may re-raise this exception, raise another exception, or yield
-    *initial* call to __next__() receives a StopIteration instance
+    another value, execpt that if the exception passed in to
-    argument, the generator's execution is aborted and the exception
+    __exit__() was StopIteration, it ought to raise StopIteration
-    is re-raised without passing control to the generator's body.
+    (otherwise the effect would be that a break is turned into
+    continue, which is unexpected at least).  When the *initial* call
+    resuming the generator is an __exit__() call instead of a
+    __next__() call, the generator's execution is aborted and the
+    exception is re-raised without passing control to the generator's
+    body.
 
-    When __next__() is called with a ContinueIteration instance
+    When __next__() is called with an argument that is not None, the
-    argument, the yield-expression that it resumes will return the
+    yield-expression that it resumes will return the value attribute
-    value attribute of the argument.  If it resumes a yield-statement,
+    of the argument.  If it resumes a yield-statement, the value is
-    the value is ignored.  When the *initial* call to __next__()
+    ignored (or should this be considered an error?).  When the
-    receives a ContinueIteration instance argument, the generator's
+    *initial* call to __next__() receives an argument that is not
-    execution is started normally; the argument's value attribute is
+    None, the generator's execution is started normally; the
-    ignored.
+    argument's value attribute is ignored (or should this be
+    considered an error?).  When __next__() is called without an
+    argument or with None as argument, and a yield-expression is
+    resumed, the yield-expression returns None.
 
     When a generator that has not yet terminated is garbage-collected
     (either through reference counting or by the cyclical garbage
-    collector), its __next__() method is called once with a
+    collector), its __exit__() method is called once with
-    StopIteration instance argument.  Together with the requirement
+    StopIteration as its first argument.  Together with the
-    that __next__() should always re-raise a StopIteration argument,
+    requirement that a generator ought to raise StopIteration when
-    this guarantees the eventual activation of any finally-clauses
+    __exit__() is called with StopIteration, this guarantees the
-    that were active when the generator was last suspended.  Of
+    eventual activation of any finally-clauses that were active when
-    course, under certain circumstances the generator may never be
+    the generator was last suspended.  Of course, under certain
-    garbage-collected.  This is no different than the guarantees that
+    circumstances the generator may never be garbage-collected.  This
-    are made about finalizers (__del__() methods) of other objects.
+    is no different than the guarantees that are made about finalizers
+    (__del__() methods) of other objects.
 
     Note: the syntactic extensions to yield make its use very similar
     to that in Ruby.  This is intentional.  Do note that in Python the
@@ -367,71 +361,6 @@ Specification: Generator Exception Handling
     cases work differently; in Python, you cannot save the block for
     later use, and you cannot test whether there is a block or not.
 
-Specification: Alternative __next__() and Generator Exception Handling
-
-    The above specification doesn't let the generator handle general
-    exceptions.  If we want that, we could modify the __next__() API
-    to take either a value or an exception argument, with an
-    additional flag argument to distinguish between the two.  When the
-    second argument is True, the first must be an Exception instance,
-    which raised at the point of the resuming yield; otherwise the
-    first argument is the value that is returned from the
-    yield-expression (or ignored by a yield-statement).  Wrapping a
-    regular value in a ContinueIteration is then no longer necessary.
-
-    The next() built-in would be modified likewise:
-
-        def next(itr, arg=None, exc=False):
-            nxt = getattr(itr, "__next__", None)
-            if nxt is not None:
-                return nxt(arg, exc)
-            if arg is None and not exc:
-                return itr.next()
-            raise TypeError("next() with args for old-style iterator")
-
-    The translation of a block-statement would become:
-
-        itr = EXPR1
-        arg = val = None
-        ret = exc = False
-        while True:
-            try:
-                VAR1 = next(itr, arg, exc)
-            except StopIteration:
-                if ret:
-                    return val
-                break
-            try:
-                arg = val = None
-                ret = exc = False
-                BLOCK1
-            except Exception, arg:
-                exc = True
-
-    The translation of "continue EXPR2" would become:
-
-        arg = EXPR2
-        continue
-
-    The translation of "break" inside a block-statement would become:
-
-        arg = StopIteration()
-        exc = True
-        continue
-
-    The translation of "return EXPR3" inside a block-statement would
-    become:
-
-        val = EXPR3
-        arg = StopIteration()
-        ret = exc = True
-        continue
-
-    The translation of a for-loop would be the same as indicated
-    earlier (inside a for-loop only the translation of "continue
-    EXPR2" is changed; break and return translate to themselves in
-    that case).
-
 Loose Ends
 
     These are things that need to be resolved before accepting the
@@ -447,17 +376,8 @@ Loose Ends
     - Decide on the keyword ('block', 'with', '@', nothing, or
       something else?).
 
-    - Phillip Eby wants a way to pass tracebacks along with
-      exceptions.
-
-    - The translation for the for-loop's else-clause.
-
     - Whether a block-statement should allow an else-clause.
 
-    - Which API to use to pass in an exception: itr.__next__(exc),
-      itr.__next__(exc, True) or itr.__exit__(exc[, traceback]).
-      Hmm..., perhaps itr.__next__(exc, traceback)?
-
 Comparison to Thunks
 
     Alternative semantics proposed for the block-statement turn the
@@ -626,7 +546,7 @@ Examples
 
         block auto_retry(3, IOError):
             f = urllib.urlopen("http://python.org/peps/pep-0340.html")
-	    print f.read()
+            print f.read()
 
     5. It is possible to nest blocks and combine templates: