From a7de7b62af2bd72303fbe2dfbafba674aa8c6ab2 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 2 May 2005 03:30:07 +0000 Subject: [PATCH] Establish the "alternative" version. The exception API is called __exit__(), and its signature is the same as that of the raise-statement. Still some loose ends. --- pep-0340.txt | 308 +++++++++++++++++++-------------------------------- 1 file changed, 114 insertions(+), 194 deletions(-) diff --git a/pep-0340.txt b/pep-0340.txt index 87aee3a38..2234896ce 100644 --- a/pep-0340.txt +++ b/pep-0340.txt @@ -26,30 +26,6 @@ Introduction this on python-dev recently [1], and I figured it would be time to write up a precise spec in PEP form. -Proposal Evolution - - The discussion on python-dev has changed my mind slightly on how - exceptions should be handled, but I don't have the time to do a - full update of the PEP right now. Basically, I'm now in favor of - a variation on the exception handling proposed in the section - "Alternative __next__() and Generator Exception Handling" below. - - The added twist is that instead of adding a flag argument to - next() and __next__() to indicate whether the previous argument is - a value or an exception, we use a separate API (an __exit__() - method taking an exception and perhaps a traceback) for the - exception. If an iterator doesn't implement __exit__(), the - exception is just re-raised. It is expected that, apart from - generators, very few iterators will implement __exit__(); one use - case would be a fast implementation of synchronized() written in - C. - - The built-in next() function only interfaces to the next() and - __next__() methods; there is no user-friendly API to call - __exit__(). (Or perhaps calling next(itr, exc, traceback) would - call itr.__exit__(exc, traceback) if itr has an __exit__ method - and otherwise raise exc.__class__, exc, traceback?) - Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) @@ -96,40 +72,25 @@ Use Cases TBD. For now, see the Examples section near the end. -Specification: the Iteration Exception Hierarchy - - Two new built-in exceptions are defined, and StopIteration is - moved in the exception hierarchy: - - class Iteration(Exception): - pass - - class StopIteration(Iteration): - pass - - class ContinueIteration(Iteration): - def __init__(self, value=None): - self.value = None - Specification: the __next__() Method A new method for iterators is proposed, called __next__(). It - takes one optional argument, which defaults to None. If not None, - the argument must be an Iteration instance. Calling the + takes one optional argument, which defaults to None. Calling the __next__() method without argument or with None is equivalent to using the old iterator API, next(). For backwards compatibility, it is recommended that iterators also implement a next() method as an alias for calling the __next__() method without an argument. - Calling the __next__() method with a StopIteration instance - signals the iterator that the caller wants to abort the iteration - sequence; the iterator should respond by doing any necessary - cleanup and raising StopIteration. Calling it with a - ContinueIteration instance signals the iterator that the caller - wants to continue the iteration; the ContinueIteration exception - has a 'value' attribute which may be used by the iterator as a - hint on what to do next. Calling it with a (base class) Iteration - instance is the same as calling it with None. + The argument to the __next__() method may be used by the iterator + as a hint on what to do next. + +Specification: the __exit__() Method + + An optional new method for iterators is proposed, called + __exit__(). It takes up to three arguments which correspond to + the three "arguments" to the raise-statement: type, value, and + traceback. If all three arguments are None, sys.exc_info() may be + consulted to provide suitable default values. Specification: the next() Built-in Function @@ -143,34 +104,43 @@ Specification: the next() Built-in Function return itr.next() raise TypeError("next() with arg for old-style iterator") -Specification: the 'for' Loop + This function is proposed because there is often a need to call + the next() method outside a for-loop; the new API, and the + backwards compatibility code, is too ugly to have to repeat in + user code. + + Note that I'm not proposing a built-in function to call the + __exit__() method of an iterator. I don't expect that this will + be called much outside the block-statement. + +Specification: a Change to the 'for' Loop A small change in the translation of the for-loop is proposed. The statement for VAR1 in EXPR1: BLOCK1 + else: + BLOCK2 will be translated as follows: itr = iter(EXPR1) arg = None + brk = False while True: try: VAR1 = next(itr, arg) except StopIteration: + brk = True break arg = None BLOCK1 + if brk: + BLOCK2 - (However, 'itr' and 'arg' are hidden from the user, their scope - ends when the while-loop is exited, and they are not shared with - nested or outer for-loops, and the user cannot override the - built-ins referenced.) - - I'm leaving the translation of an else-clause up to the reader; - note that you can't simply affix the else-clause to the while-loop - since it is always broken out. + (However, the variables 'itr' etc. are not user-visible and the + built-in names used cannot be overridden by the user.) Specification: the Extended 'continue' Statement @@ -180,7 +150,7 @@ Specification: the Extended 'continue' Statement is legal and is translated into - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue (Where 'arg' references the corresponding hidden variable from the @@ -195,21 +165,28 @@ Specification: the Anonymous Block Statement block EXPR1 as VAR1: BLOCK1 + else: + BLOCK2 Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary expression (but not an expression-list) and VAR1 is an arbitrary assignment target (which may be a comma-separated list). - The "as VAR1" part is optional; if omitted, the assignment to VAR1 - in the translation below is omitted (but the next() call is not!). + The "as VAR1" part is optional; if omitted, the assignments to + VAR1 in the translation below are omitted (but the expressions + assigned are still evaluated!). - The choice of the 'block' keyword is contentious; it has even been - proposed not to use a keyword at all. PEP 310 uses 'with' for - similar semantics, but I would like to reserve that for a - with-statement similar to the one found in Pascal and VB. To - sidestep this issue momentarily I'm using 'block' until we can - agree on a keyword. (I just found that the C# designers don't - like 'with' [2].) + The choice of the 'block' keyword is contentious; many + alternatives have been proposed, including not to use a keyword at + all (which I actually like). PEP 310 uses 'with' for similar + semantics, but I would like to reserve that for a with-statement + similar to the one found in Pascal and VB. (Though I just found + that the C# designers don't like 'with' [2], and I have to agree + with their reasoning.) To sidestep this issue momentarily I'm + using 'block' until we can agree on the right keyword, if any. + + Note that the 'as' keyword is not contentious (it will finally be + elevated to proper keyword status). Note that it is left in the middle whether a block-statement represents a loop or not; this is up to the iterator, but in the @@ -218,61 +195,66 @@ Specification: the Anonymous Block Statement The translation is subtly different from the translation of a for-loop: iter() is not called, so EXPR1 should already be an iterator (not just an iterable); and the iterator is guaranteed to - be exhausted when the block-statement is left: + be notified when the block-statement is left, regardless if this + is due to a break, return or exception: - itr = EXPR1 - val = arg = None - ret = False + itr = EXPR1 # The iterator + ret = False # True if a return statement is active + val = None # Return value, if ret == True + arg = None # Argument to __next__() (value from continue) + exc = None # sys.exc_info() tuple if an exception is active while True: try: - VAR1 = next(itr, arg) + if exc: + ext = getattr(itr, "__exit__", None) + if ext is not None: + VAR1 = ext(*exc) # May re-raise *exc + else: + raise *exc # Well, the moral equivalent :-) + else: + VAR1 = next(itr, arg) # May raise StopIteration except StopIteration: if ret: return val - if val is not None: - raise val break try: - val = arg = None ret = False + val = arg = exc = None BLOCK1 - except Exception, val: - arg = StopIteration() + except: + exc = sys.exc_info() - (Again, 'itr' etc. are hidden, and the user cannot override the - built-ins.) - - The "raise val" translation is inexact; this is supposed to - re-raise the exact exception that was raised inside BLOCK1, with - the same traceback. We can't use a bare raise-statement because - we've just caught StopIteration. + (Again, the variables and built-ins are hidden from the user.) Inside BLOCK1, the following special translations apply: - "continue" and "continue EXPR2" are always legal; the latter is translated as shown earlier: - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue - "break" is always legal; it is translated into: - arg = StopIteration() + exc = (StopIteration,) continue - "return EXPR3" is only legal when the block-statement is contained in a function definition; it is translated into: - val = EXPR3 + exc = (StopIteration,) ret = True - arg = StopIteration() + val = EXPR3 continue The net effect is that break, continue and return behave much the same as if the block-statement were a for-loop, except that the iterator gets a chance at resource cleanup before the - block-statement is left. The iterator also gets a chance if the - block-statement is left through raising an exception. + block-statement is left, through the optional __exit__() method. + The iterator also gets a chance if the block-statement is left + through raising an exception. If the iterator doesn't have an + __exit__() method, there is no difference with a for-loop (except + that a for-loop calls iter() on EXPR1). Note that a yield-statement (or a yield-expression, see below) in a block-statement is not treated differently. It suspends the @@ -287,15 +269,18 @@ Specification: the Anonymous Block Statement be resumed eventually. I haven't decided yet whether the block-statement should also - allow an optional else-clause, like the for-loop. I think it - would be confusing, and emphasize the "loopiness" of the - block-statement, while I want to emphasize its *difference* from a - for-loop. + allow an optional else-clause, like the for-loop, but I'm leaning + against it. I think it would be confusing, and emphasize the + "loopiness" of the block-statement, while I want to emphasize its + *difference* from a for-loop. In addition, there are several + possible semantics for an else-clause. Specification: Generator Exception Handling Generators will implement the new __next__() method API, as well - as the old argument-less next() method. + as the old argument-less next() method which becomes an alias for + calling __next__() without an argument. They will also implement + the new __exit__() method API. Generators will be allowed to have a yield statement inside a try-finally statement. @@ -330,32 +315,41 @@ Specification: Generator Exception Handling are all illegal. (Some of the edge cases are motivated by the current legality of "yield 12, 42".) - When __next__() is called with a StopIteration instance argument, - the yield statement that is resumed by the __next__() call will - raise this StopIteration exception. The generator should re-raise - this exception; it should not yield another value. When the - *initial* call to __next__() receives a StopIteration instance - argument, the generator's execution is aborted and the exception - is re-raised without passing control to the generator's body. + When __exit__() is called, the generator is resumed but at the + point of the yield-statement or -expression the exception + represented by the __exit__ argument(s) is raised. The generator + may re-raise this exception, raise another exception, or yield + another value, execpt that if the exception passed in to + __exit__() was StopIteration, it ought to raise StopIteration + (otherwise the effect would be that a break is turned into + continue, which is unexpected at least). When the *initial* call + resuming the generator is an __exit__() call instead of a + __next__() call, the generator's execution is aborted and the + exception is re-raised without passing control to the generator's + body. - When __next__() is called with a ContinueIteration instance - argument, the yield-expression that it resumes will return the - value attribute of the argument. If it resumes a yield-statement, - the value is ignored. When the *initial* call to __next__() - receives a ContinueIteration instance argument, the generator's - execution is started normally; the argument's value attribute is - ignored. + When __next__() is called with an argument that is not None, the + yield-expression that it resumes will return the value attribute + of the argument. If it resumes a yield-statement, the value is + ignored (or should this be considered an error?). When the + *initial* call to __next__() receives an argument that is not + None, the generator's execution is started normally; the + argument's value attribute is ignored (or should this be + considered an error?). When __next__() is called without an + argument or with None as argument, and a yield-expression is + resumed, the yield-expression returns None. When a generator that has not yet terminated is garbage-collected (either through reference counting or by the cyclical garbage - collector), its __next__() method is called once with a - StopIteration instance argument. Together with the requirement - that __next__() should always re-raise a StopIteration argument, - this guarantees the eventual activation of any finally-clauses - that were active when the generator was last suspended. Of - course, under certain circumstances the generator may never be - garbage-collected. This is no different than the guarantees that - are made about finalizers (__del__() methods) of other objects. + collector), its __exit__() method is called once with + StopIteration as its first argument. Together with the + requirement that a generator ought to raise StopIteration when + __exit__() is called with StopIteration, this guarantees the + eventual activation of any finally-clauses that were active when + the generator was last suspended. Of course, under certain + circumstances the generator may never be garbage-collected. This + is no different than the guarantees that are made about finalizers + (__del__() methods) of other objects. Note: the syntactic extensions to yield make its use very similar to that in Ruby. This is intentional. Do note that in Python the @@ -367,71 +361,6 @@ Specification: Generator Exception Handling cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Specification: Alternative __next__() and Generator Exception Handling - - The above specification doesn't let the generator handle general - exceptions. If we want that, we could modify the __next__() API - to take either a value or an exception argument, with an - additional flag argument to distinguish between the two. When the - second argument is True, the first must be an Exception instance, - which raised at the point of the resuming yield; otherwise the - first argument is the value that is returned from the - yield-expression (or ignored by a yield-statement). Wrapping a - regular value in a ContinueIteration is then no longer necessary. - - The next() built-in would be modified likewise: - - def next(itr, arg=None, exc=False): - nxt = getattr(itr, "__next__", None) - if nxt is not None: - return nxt(arg, exc) - if arg is None and not exc: - return itr.next() - raise TypeError("next() with args for old-style iterator") - - The translation of a block-statement would become: - - itr = EXPR1 - arg = val = None - ret = exc = False - while True: - try: - VAR1 = next(itr, arg, exc) - except StopIteration: - if ret: - return val - break - try: - arg = val = None - ret = exc = False - BLOCK1 - except Exception, arg: - exc = True - - The translation of "continue EXPR2" would become: - - arg = EXPR2 - continue - - The translation of "break" inside a block-statement would become: - - arg = StopIteration() - exc = True - continue - - The translation of "return EXPR3" inside a block-statement would - become: - - val = EXPR3 - arg = StopIteration() - ret = exc = True - continue - - The translation of a for-loop would be the same as indicated - earlier (inside a for-loop only the translation of "continue - EXPR2" is changed; break and return translate to themselves in - that case). - Loose Ends These are things that need to be resolved before accepting the @@ -447,17 +376,8 @@ Loose Ends - Decide on the keyword ('block', 'with', '@', nothing, or something else?). - - Phillip Eby wants a way to pass tracebacks along with - exceptions. - - - The translation for the for-loop's else-clause. - - Whether a block-statement should allow an else-clause. - - Which API to use to pass in an exception: itr.__next__(exc), - itr.__next__(exc, True) or itr.__exit__(exc[, traceback]). - Hmm..., perhaps itr.__next__(exc, traceback)? - Comparison to Thunks Alternative semantics proposed for the block-statement turn the @@ -626,7 +546,7 @@ Examples block auto_retry(3, IOError): f = urllib.urlopen("http://python.org/peps/pep-0340.html") - print f.read() + print f.read() 5. It is possible to nest blocks and combine templates: