From 1eee9e2d60181846ce8e1215ee5a92d83b8bff1e Mon Sep 17 00:00:00 2001 From: Tim Peters Date: Sat, 23 Jun 2001 08:53:21 +0000 Subject: [PATCH] Lotsa changes: more details and examples covering recent decisions wrt return stmts, try/except, try/finally; more Q&A. I expect this is the last major revision of this PEP, so I'm going to post it again too. --- pep-0255.txt | 154 ++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 115 insertions(+), 39 deletions(-) diff --git a/pep-0255.txt b/pep-0255.txt index d8249e4e0..661bfff6b 100644 --- a/pep-0255.txt +++ b/pep-0255.txt @@ -10,7 +10,7 @@ Type: Standards Track Requires: 234 Created: 18-May-2001 Python-Version: 2.2 -Post-History: 14-Jun-2001 +Post-History: 14-Jun-2001, 23-Jun-2001 Abstract @@ -118,17 +118,22 @@ Motivation of Python iterator[1], but of an especially powerful kind. -Specification +Specification: Yield A new statement is introduced: yield_stmt: "yield" expression_list "yield" is a new keyword, so a future statement[8] is needed to phase - this in. [XXX spell this out] + this in. [XXX spell this out -- but new keywords have ripple effects + across tools too, and it's not clear this can be forced into the future + framework at all -- it's not even clear that Python's parser alone can + be taught to swing both ways based on a future stmt] The yield statement may only be used inside functions. A function that - contains a yield statement is called a generator function. + contains a yield statement is called a generator function. A generator + function is an ordinary function object in all respects, but has the + new CO_GENERATOR flag set in the code object's co_flags member. When a generator function is called, the actual arguments are bound to function-local formal argument names in the usual way, but no code in @@ -153,6 +158,15 @@ Specification proceed exactly as if the yield statement were just another external call. + Restriction: A yield statement is not allowed in the try clause of a + try/finally construct. The difficulty is that there's no guarantee + the generator will ever be resumed, hence no guarantee that the finally + block will ever get executed; that's too much a violation of finally's + purpose to bear. + + +Specification: Return + A generator function can also contain return statements of the form: "return" @@ -161,15 +175,42 @@ Specification in the body of a generator (although, of course, they may appear in the bodies of non-generator functions nested within the generator). - When a return statement is encountered, nothing is returned, but a - StopIteration exception is raised, signalling that the iterator is - exhausted. The same is true if control flows off the end of the - function. Note that return means "I'm done, and have nothing - interesting to return", for both generator functions and non-generator - functions. + When a return statement is encountered, control proceeds as in any + function return, executing the appropriate finally clauses (if any + exist). Then a StopIteration exception is raised, signalling that the + iterator is exhausted. A StopIteration exception is also raised if + control flows off the end of the generator without an explict return. + + Note that return means "I'm done, and have nothing interesting to + return", for both generator functions and non-generator functions. + + Note that return isn't always equivalent to raising StopIteration: the + difference lies in how enclosing try/except constructs are treated. + For example, + + >>> def f1(): + ... try: + ... return + ... except: + ... yield 1 + >>> print list(f1()) + [] + + because, as in any function, return simply exits, but + + >>> def f2(): + ... try: + ... raise StopIteration + ... except: + ... yield 42 + >>> print list(f2()) + [42] + + because StopIteration is captured by a bare "except", as is any + exception. -Generators and Exception Propagation +Specification: Generators and Exception Propagation If an unhandled exception-- including, but not limited to, StopIteration --is raised by, or passes through, a generator function, @@ -192,44 +233,45 @@ Generators and Exception Propagation File "", line 2, in g File "", line 2, in f ZeroDivisionError: integer division or modulo by zero - >>> k.next() # and the generator function cannot be resumed + >>> k.next() # and the generator cannot be resumed Traceback (most recent call last): File "", line 1, in ? StopIteration >>> -Yield and Try/Except/Finally +Specification: Try/Except/Finally - While "yield" is a control-flow statement, and in most respects acts - like a "return" statement from the caller's point of view, within a - generator it acts more like a callback function. In particular, it has - no special semantics with respect to try/except/finally. This is best - illustrated by a contrived example; the primary lesson to take from - this is that using yield in a finally block is a dubious idea! + As noted earlier, yield is not allowed in the try clause of a try/ + finally construct. A consequence is that generators should allocate + critical resources with great care. There is no restriction on yield + otherwise appearing in finally clauses, except clauses, or in the try + clause of a try/except construct: - >>> def g(): + >>> def f(): ... try: ... yield 1 - ... 1/0 # raises exception - ... yield 2 # we never get here + ... try: + ... yield 2 + ... 1/0 + ... yield 3 # never get here + ... except ZeroDivisionError: + ... yield 4 + ... yield 5 + ... raise + ... except: + ... yield 6 + ... yield 7 # the "raise" above stops this + ... except: + ... yield 8 + ... yield 9 + ... try: + ... x = 12 ... finally: - ... yield 3 # yields, and we raise the exception *next* time - ... yield 4 # we never get here - >>> k = g() - >>> k.next() - 1 - >>> k.next() - 3 - >>> k.next() # as if "yield 3" were a callback, exception raised now - Traceback (most recent call last): - File "", line 1, in ? - File "", line 4, in g - ZeroDivisionError: integer division or modulo by zero - >>> k.next() # unhandled exception terminated the generator - Traceback (most recent call last): - File "", line 1, in ? - StopIteration + ... yield 10 + ... yield 11 + >>> print list(f()) + [1, 2, 4, 5, 8, 9, 10, 11] >>> @@ -299,6 +341,10 @@ Example print x, print + Both output blocks display: + + A B C D E F G H I J K L M N O P Q R S T U V W X Y Z + Q & A @@ -312,7 +358,34 @@ Q & A yield is a control construct. It's also believed that efficient implementation in Jython requires that the compiler be able to determine potential suspension points at compile-time, and a new - keyword makes that easy. + keyword makes that easy. The CPython referrence implementation also + exploits it heavily, to detect which functions *are* generator- + functions (although a new keyword in place of "def" would solve that + for CPython -- but people asking the "why a new keyword?" question + don't want any new keyword). + + Q: Then why not some other special syntax without a new keyword? For + example, one of these instead of "yield 3": + + return 3 and continue + return and continue 3 + return generating 3 + continue return 3 + return >> , 3 + from generator return 3 + return >> 3 + return << 3 + >> 3 + << 3 + + A: Did I miss one ? Out of hundreds of messages, I counted two + suggesting such an alternative, and extracted the above from them. + It would be nice not to need a new keyword, but nicer to make yield + very clear -- I don't want to have to *deduce* that a yield is + occurring from making sense of a previous senseless sequence of + keywords or operators. Still, if this attracts enough interest, + proponents should settle on a single consensus suggestion, and Guido + will Pronounce on it. Q. Why allow "return" at all? Why not force termination to be spelled "raise StopIteration"? @@ -323,6 +396,9 @@ Q & A these mechanisms for advanced users. That's not an argument for forcing everyone to work at that level, though. "return" means "I'm done" in any kind of function, and that's easy to explain and to use. + Note that "return" isn't always equivalent to "raise StopIteration" + in try/except construct, either (see the "Specification: Return" + section). Q. Then why not allow an expression on "return" too?