New section with contextlib example (Isaac Schwabacher).
Other additions by Chris, including renumbered references.
This commit is contained in:
Guido van Rossum
parent
1c2c1c0a7a
commit
a6b3c8b1f7
105
pep-0479.txt
105
pep-0479.txt
|
|
@ -37,12 +37,39 @@ raises ``StopIteration`` and causes the iteration controlled by the
|
|||
generator to terminate silently. (When another exception is raised, a
|
||||
traceback is printed pinpointing the cause of the problem.)
|
||||
|
||||
The proposal also clears up the confusion about how to terminate a
|
||||
generator: the proper way is ``return``, not ``raise StopIteration``.
|
||||
This is particularly pernicious in combination with the ``yield from``
|
||||
construct of PEP 380 [1]_, as it breaks the abstraction that a
|
||||
subgenerator may be factored out of a generator. That PEP notes this
|
||||
limitation, but notes that "use cases for these [are] rare to non-
|
||||
existent". Unfortunately while intentional use is rare, it is easy to
|
||||
stumble on these cases by accident::
|
||||
|
||||
Finally, the proposal reduces the difference between list
|
||||
@contextlib.contextmanager
|
||||
def transaction():
|
||||
begin()
|
||||
try:
|
||||
yield from do_it()
|
||||
except:
|
||||
rollback()
|
||||
raise
|
||||
else:
|
||||
commit()
|
||||
|
||||
def do_it():
|
||||
initial_preparations()
|
||||
yield
|
||||
finishing_touches()
|
||||
|
||||
Here factoring out ``do_it`` into a subgenerator has introduced a
|
||||
subtle bug: if the wrapped block raises ``StopIteration``, under the
|
||||
current behavior ``do_it`` will fail but report success by returning
|
||||
normally, causing the failed transaction to be committed! Similarly
|
||||
problematic behavior occurs when an ``asyncio`` coroutine raises
|
||||
``StopIteration``, causing it to terminate silently.
|
||||
|
||||
Additionally, the proposal reduces the difference between list
|
||||
comprehensions and generator expressions, preventing surprises such as
|
||||
the one that started this discussion [1]_. Henceforth, the following
|
||||
the one that started this discussion [2]_. Henceforth, the following
|
||||
statements will produce the same result if either produces a result at
|
||||
all::
|
||||
|
||||
|
|
@ -56,6 +83,10 @@ a (truncated) result, while the second form raises an exception
|
|||
will raise an exception at this point (albeit ``RuntimeError`` in the
|
||||
first case and ``StopIteration`` in the second).
|
||||
|
||||
Finally, the proposal also clears up the confusion about how to
|
||||
terminate a generator: the proper way is ``return``, not
|
||||
``raise StopIteration``.
|
||||
|
||||
|
||||
Background information
|
||||
======================
|
||||
|
|
@ -77,7 +108,7 @@ Proposal
|
|||
If a ``StopIteration`` is about to bubble out of a generator frame, it
|
||||
is replaced with ``RuntimeError``, which causes the ``next()`` call
|
||||
(which invoked the generator) to fail, passing that exception out.
|
||||
From then on it's just like any old exception. [3]_
|
||||
From then on it's just like any old exception. [4]_
|
||||
|
||||
This affects the third outcome listed above, without altering any
|
||||
other effects. Furthermore, it only affects this outcome when the
|
||||
|
|
@ -119,10 +150,10 @@ Consequences for existing code
|
|||
|
||||
This change will affect existing code that depends on
|
||||
``StopIteration`` bubbling up. The pure Python reference
|
||||
implementation of ``groupby`` [2]_ currently has comments "Exit on
|
||||
implementation of ``groupby`` [3]_ currently has comments "Exit on
|
||||
``StopIteration``" where it is expected that the exception will
|
||||
propagate and then be handled. This will be unusual, but not unknown,
|
||||
and such constructs will fail. Other examples abound, e.g. [5]_, [6]_.
|
||||
and such constructs will fail. Other examples abound, e.g. [6]_, [7]_.
|
||||
|
||||
(Nick Coghlan comments: """If you wanted to factor out a helper
|
||||
function that terminated the generator you'd have to do "return
|
||||
|
|
@ -235,6 +266,31 @@ instead of the initial line) and an "abnormal" termination (failing
|
|||
to find the end marker in the inner loop, which will now raise
|
||||
``RuntimeError``).
|
||||
|
||||
This effect of ``StopIteration`` has been used to cut a generator
|
||||
expression short, creating a form of ``takewhile``::
|
||||
|
||||
def stop():
|
||||
raise StopIteration
|
||||
print(list(x for x in range(10) if x < 5 or stop()))
|
||||
# prints [0, 1, 2, 3, 4]
|
||||
|
||||
Under the current proposal, this form of non-local flow control is
|
||||
not supported, and would have to be rewritten in statement form::
|
||||
|
||||
def gen():
|
||||
for x in range(10):
|
||||
if x >= 5: return
|
||||
yield x
|
||||
print(list(gen()))
|
||||
# prints [0, 1, 2, 3, 4]
|
||||
|
||||
While this is a small loss of functionality, it is functionality that
|
||||
often comes at the cost of readability, and just as ``lambda`` has
|
||||
restrictions compared to ``def``, so does a generator expression have
|
||||
restrictions compared to a generator function. In many cases, the
|
||||
transformation to full generator function will be trivially easy, and
|
||||
may improve structural clarity.
|
||||
|
||||
|
||||
Explanation of generators, iterators, and StopIteration
|
||||
=======================================================
|
||||
|
|
@ -330,7 +386,7 @@ If it is not that subclass, it is an escaping exception rather than a
|
|||
return statement.
|
||||
|
||||
The inspiration for this alternative proposal was Nick's observation
|
||||
[7]_ that if an ``asyncio`` coroutine [8]_ accidentally raises
|
||||
[8]_ that if an ``asyncio`` coroutine [9]_ accidentally raises
|
||||
``StopIteration``, it currently terminates silently, which may present
|
||||
a hard-to-debug mystery to the developer. The main proposal turns
|
||||
such accidents into clearly distinguishable ``RuntimeError`` exceptions,
|
||||
|
|
@ -367,7 +423,7 @@ of the distinction between the two exception types.
|
|||
Converting the exception inside next()
|
||||
--------------------------------------
|
||||
|
||||
Mark Shannon suggested [11]_ that the problem could be solved in
|
||||
Mark Shannon suggested [12]_ that the problem could be solved in
|
||||
``next()`` rather than at the boundary of generator functions. By
|
||||
having ``next()`` catch ``StopIteration`` and raise instead
|
||||
``ValueError``, all unexpected ``StopIteration`` bubbling would be
|
||||
|
|
@ -384,11 +440,11 @@ Criticism
|
|||
=========
|
||||
|
||||
Unofficial and apocryphal statistics suggest that this is seldom, if
|
||||
ever, a problem. [4]_ Code does exist which relies on the current
|
||||
behaviour (e.g. [2]_, [5]_, [6]_), and there is the concern that this
|
||||
ever, a problem. [5]_ Code does exist which relies on the current
|
||||
behaviour (e.g. [3]_, [6]_, [7]_), and there is the concern that this
|
||||
would be unnecessary code churn to achieve little or no gain.
|
||||
|
||||
Steven D'Aprano started an informal survey on comp.lang.python [9]_;
|
||||
Steven D'Aprano started an informal survey on comp.lang.python [10]_;
|
||||
at the time of writing only two responses have been received: one was
|
||||
in favor of changing list comprehensions to match generator
|
||||
expressions (!), the other was in favor of this PEP's main proposal.
|
||||
|
|
@ -411,37 +467,40 @@ generators from doing so.
|
|||
References
|
||||
==========
|
||||
|
||||
.. [1] Initial mailing list comment
|
||||
.. [1] PEP 380 - Syntax for Delegating to a Subgenerator
|
||||
(https://www.python.org/dev/peps/pep-0380)
|
||||
|
||||
.. [2] Initial mailing list comment
|
||||
(https://mail.python.org/pipermail/python-ideas/2014-November/029906.html)
|
||||
|
||||
.. [2] Pure Python implementation of groupby
|
||||
.. [3] Pure Python implementation of groupby
|
||||
(https://docs.python.org/3/library/itertools.html#itertools.groupby)
|
||||
|
||||
.. [3] Proposal by GvR
|
||||
.. [4] Proposal by GvR
|
||||
(https://mail.python.org/pipermail/python-ideas/2014-November/029953.html)
|
||||
|
||||
.. [4] Response by Steven D'Aprano
|
||||
.. [5] Response by Steven D'Aprano
|
||||
(https://mail.python.org/pipermail/python-ideas/2014-November/029994.html)
|
||||
|
||||
.. [5] Split a sequence or generator using a predicate
|
||||
.. [6] Split a sequence or generator using a predicate
|
||||
(http://code.activestate.com/recipes/578416-split-a-sequence-or-generator-using-a-predicate/)
|
||||
|
||||
.. [6] wrap unbounded generator to restrict its output
|
||||
.. [7] wrap unbounded generator to restrict its output
|
||||
(http://code.activestate.com/recipes/66427-wrap-unbounded-generator-to-restrict-its-output/)
|
||||
|
||||
.. [7] Post from Nick Coghlan mentioning asyncio
|
||||
.. [8] Post from Nick Coghlan mentioning asyncio
|
||||
(https://mail.python.org/pipermail/python-ideas/2014-November/029961.html)
|
||||
|
||||
.. [8] Coroutines in asyncio
|
||||
.. [9] Coroutines in asyncio
|
||||
(https://docs.python.org/3/library/asyncio-task.html#coroutines)
|
||||
|
||||
.. [9] Thread on comp.lang.python started by Steven D'Aprano
|
||||
.. [10] Thread on comp.lang.python started by Steven D'Aprano
|
||||
(https://mail.python.org/pipermail/python-list/2014-November/680757.html)
|
||||
|
||||
.. [10] Tracker issue with Proof-of-Concept patch
|
||||
.. [11] Tracker issue with Proof-of-Concept patch
|
||||
(http://bugs.python.org/issue22906)
|
||||
|
||||
.. [11] Post from Mark Shannon with alternate proposal
|
||||
.. [12] Post from Mark Shannon with alternate proposal
|
||||
(https://mail.python.org/pipermail/python-dev/2014-November/137129.html)
|
||||
|
||||
Copyright
|
||||
|
|
|
|||
Loading…
Reference in New Issue