208 lines
6.7 KiB
Plaintext
208 lines
6.7 KiB
Plaintext
PEP: 342
|
|
Title: Enhanced Iterators
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Guido van Rossum
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Content-Type: text/plain
|
|
Created: 10-May-2005
|
|
Post-History:
|
|
|
|
Introduction
|
|
|
|
This PEP proposes a new iterator API that allows values to be
|
|
passed into an iterator using "continue EXPR". These values are
|
|
received in the iterator as an argument to the new __next__
|
|
method, and can be accessed in a generator with a
|
|
yield-expression.
|
|
|
|
The content of this PEP is derived from the original content of
|
|
PEP 340, broken off into its own PEP as the new iterator API is
|
|
pretty much orthogonal from the anonymous block statement
|
|
discussion. Thanks to Steven Bethard for doing the editing.
|
|
|
|
Motivation and Summary
|
|
|
|
TBD.
|
|
|
|
Use Cases
|
|
|
|
See the Examples section near the end.
|
|
|
|
Specification: the __next__() Method
|
|
|
|
A new method for iterators is proposed, called __next__(). It
|
|
takes one optional argument, which defaults to None. Calling the
|
|
__next__() method without argument or with None is equivalent to
|
|
using the old iterator API, next(). For backwards compatibility,
|
|
it is recommended that iterators also implement a next() method as
|
|
an alias for calling the __next__() method without an argument.
|
|
|
|
The argument to the __next__() method may be used by the iterator
|
|
as a hint on what to do next.
|
|
|
|
Specification: the next() Built-in Function
|
|
|
|
This is a built-in function defined as follows:
|
|
|
|
def next(itr, arg=None):
|
|
nxt = getattr(itr, "__next__", None)
|
|
if nxt is not None:
|
|
return nxt(arg)
|
|
if arg is None:
|
|
return itr.next()
|
|
raise TypeError("next() with arg for old-style iterator")
|
|
|
|
This function is proposed because there is often a need to call
|
|
the next() method outside a for-loop; the new API, and the
|
|
backwards compatibility code, is too ugly to have to repeat in
|
|
user code.
|
|
|
|
Specification: a Change to the 'for' Loop
|
|
|
|
A small change in the translation of the for-loop is proposed.
|
|
The statement
|
|
|
|
for VAR1 in EXPR1:
|
|
BLOCK1
|
|
else:
|
|
BLOCK2
|
|
|
|
will be translated as follows:
|
|
|
|
itr = iter(EXPR1)
|
|
arg = None # Set by "continue EXPR2", see below
|
|
brk = False
|
|
while True:
|
|
try:
|
|
VAR1 = next(itr, arg)
|
|
except StopIteration:
|
|
brk = True
|
|
break
|
|
arg = None
|
|
BLOCK1
|
|
if brk:
|
|
BLOCK2
|
|
|
|
(However, the variables 'itr' etc. are not user-visible and the
|
|
built-in names used cannot be overridden by the user.)
|
|
|
|
Specification: the Extended 'continue' Statement
|
|
|
|
In the translation of the for-loop, inside BLOCK1, the new syntax
|
|
|
|
continue EXPR2
|
|
|
|
is legal and is translated into
|
|
|
|
arg = EXPR2
|
|
continue
|
|
|
|
(Where 'arg' references the corresponding hidden variable from the
|
|
previous section.)
|
|
|
|
This is also the case in the body of the block-statement proposed
|
|
below.
|
|
|
|
EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to
|
|
"continue (1, 2, 3)".
|
|
|
|
Specification: Generators and Yield-Expressions
|
|
|
|
Generators will implement the new __next__() method API, as well
|
|
as the old argument-less next() method which becomes an alias for
|
|
calling __next__() without an argument.
|
|
|
|
The yield-statement will be allowed to be used on the right-hand
|
|
side of an assignment; in that case it is referred to as
|
|
yield-expression. The value of this yield-expression is None
|
|
unless __next__() was called with an argument; see below.
|
|
|
|
A yield-expression must always be parenthesized except when it
|
|
occurs at the top-level expression on the right-hand side of an
|
|
assignment. So
|
|
|
|
x = yield 42
|
|
x = yield
|
|
x = 12 + (yield 42)
|
|
x = 12 + (yield)
|
|
foo(yield 42)
|
|
foo(yield)
|
|
|
|
are all legal, but
|
|
|
|
x = 12 + yield 42
|
|
x = 12 + yield
|
|
foo(yield 42, 12)
|
|
foo(yield, 12)
|
|
|
|
are all illegal. (Some of the edge cases are motivated by the
|
|
current legality of "yield 12, 42".)
|
|
|
|
Note that a yield-statement or yield-expression without an
|
|
expression is now legal. This makes sense: when the information
|
|
flow in the next() call is reversed, it should be possible to
|
|
yield without passing an explicit value ("yield" is of course
|
|
equivalent to "yield None").
|
|
|
|
When __next__() is called with an argument that is not None, the
|
|
yield-expression that it resumes will return the argument. If it
|
|
resumes a yield-statement, the value is ignored (this is similar
|
|
to ignoring the value returned by a function call). When the
|
|
*initial* call to __next__() receives an argument that is not
|
|
None, TypeError is raised; this is likely caused by some logic
|
|
error. When __next__() is called without an argument or with None
|
|
as argument, and a yield-expression is resumed, the
|
|
yield-expression returns None.
|
|
|
|
Note: the syntactic extensions to yield make its use very similar
|
|
to that in Ruby. This is intentional. Do note that in Python the
|
|
block passes a value to the generator using "continue EXPR" rather
|
|
than "return EXPR", and the underlying mechanism whereby control
|
|
is passed between the generator and the block is completely
|
|
different. Blocks in Python are not compiled into thunks; rather,
|
|
yield suspends execution of the generator's frame. Some edge
|
|
cases work differently; in Python, you cannot save the block for
|
|
later use, and you cannot test whether there is a block or not.
|
|
|
|
Alternative
|
|
|
|
An alternative proposal is still under consideration, where
|
|
instead of adding a __next__() method, the existing next() method
|
|
is given an optional argument. The next() built-in function is
|
|
then unnecessary. The only line that changes in the translation is
|
|
the line
|
|
|
|
VAR1 = next(itr, arg)
|
|
|
|
which will be replaced by this
|
|
|
|
if arg is None:
|
|
VAR1 = itr.next()
|
|
else:
|
|
VAR1 = itr.next(arg)
|
|
|
|
If "continue EXPR2" is used and EXPR2 does not evaluate to None,
|
|
and the iterator's next() method does not support the optional
|
|
argument, a TypeError exception will be raised, which is the same
|
|
behavior as above.
|
|
|
|
This proposal is more compatible (no new method name, no new
|
|
built-in needed) but less future-proof; in some sense it was a
|
|
mistake to call this method next() instead of __next__(), since
|
|
*all* other operations corresponding to function pointers in the C
|
|
type structure have names with leading and trailing underscores.
|
|
|
|
Acknowledgements
|
|
|
|
See Acknowledgements of PEP 340.
|
|
|
|
References
|
|
|
|
TBD.
|
|
|
|
Copyright
|
|
|
|
This document has been placed in the public domain.
|