From a2bb199d1d024f5c08c60e8642518e1e80ad737b Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Tue, 25 Nov 2014 22:05:29 -0800 Subject: [PATCH] Another major editing pass. Move __future__ into spec. Add section about compatible code. --- pep-0479.txt | 75 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 28 deletions(-) diff --git a/pep-0479.txt b/pep-0479.txt index 1fa544606..442b7f88a 100644 --- a/pep-0479.txt +++ b/pep-0479.txt @@ -14,10 +14,12 @@ Post-History: 15-Nov-2014, 19-Nov-2014 Abstract ======== -This PEP proposes a backwards incompatible change to generators: when -``StopIteration`` is raised inside a generator, it is replaced it with -``RuntimeError``. (More precisely, this happens when the exception is -about to bubble out of the generator's stack frame.) +This PEP proposes a change to generators: when ``StopIteration`` is +raised inside a generator, it is replaced it with ``RuntimeError``. +(More precisely, this happens when the exception is about to bubble +out of the generator's stack frame.) Because the change is backwards +incompatible, the feature is initially introduced using a +``__future__`` statement. Rationale @@ -100,6 +102,17 @@ a generator, if the generator doesn't catch it (which it could do using a ``try/except`` around the ``yield``), it will be transformed into ``RuntimeError``. +During the transition phase, the new feature must be enabled +per-module using:: + + from __future__ import generator_stop + +Any generator function constructed under the influence of this +directive will have the ``REPLACE_STOPITERATION`` flag set on its code +object, and generators with the flag set will behave according to this +proposal. Once the feature becomes standard, the flag may be dropped; +code should not inspect generators for it. + Consequences for existing code ============================== @@ -116,28 +129,34 @@ function that terminated the generator you'd have to do "return yield from helper()" rather than just "helper()".""") There are also examples of generator expressions floating around that -rely on a StopIteration raised by the expression, the target or the -predicate (rather than by the __next__() call implied in the ``for`` +rely on a ``StopIteration`` raised by the expression, the target or the +predicate (rather than by the ``__next__()`` call implied in the ``for`` loop proper). -As this can break code, it is proposed to utilize the ``__future__`` -mechanism to introduce this in Python 3.5, finally making it standard -in Python 3.6 or 3.7. The proposed syntax is:: +Writing backwards and forwards compatible code +---------------------------------------------- - from __future__ import generator_stop +With the exception of hacks that raise ``StopIteration`` to exit a +generator expression, it is easy to write code that works equally well +under older Python versions as under the new semantics. -Any generator function constructed under the influence of this -directive will have the REPLACE_STOPITERATION flag set on its code -object, and generators with the flag set will behave according to this -proposal. Once the feature becomes standard, the flag may be dropped; -code should not inspect generators for it. +This is done by enclosing those places in the generator body where a +``StopIteration`` is expected (e.g. bare ``next()`` calls or in some +cases helper functions that are expected to raise ``StopIteration``) +in a ``try/except`` construct that returns when ``StopIteration`` +returns. The ``try/except`` construct should appear directly in the +generator function; doing this in a helper function that is not itself +a generator does not work. If ``raise StopIteration`` occurs directly +in a generator, simply replace it with ``return``. -Examples --------- -Generators which explicitly raise StopIteration can generally be +Examples of breakage +-------------------- + +Generators which explicitly raise ``StopIteration`` can generally be changed to simply return instead. This will be compatible with all -existing Python versions, and will not be affected by __future__. +existing Python versions, and will not be affected by ``__future__``. +Here are some illustrations from the standard library. Lib/ipaddress.py:: @@ -164,7 +183,7 @@ Becomes:: (The ``return`` is necessary for a strictly-equivalent translation, though in this particular file, there is no further code, and the -``return`` can be elided.) For compatibility with pre-3.3 versions +``return`` can be omitted.) For compatibility with pre-3.3 versions of Python, this could be written with an explicit ``for`` loop:: if context is None: @@ -172,10 +191,10 @@ of Python, this could be written with an explicit ``for`` loop:: yield line return -More complicated iteration patterns will need explicit try/catch -constructs. For example, a parser construct like this:: +More complicated iteration patterns will need explicit ``try/except`` +constructs. For example, a hypothetical parser like this:: - def unwrap(f): + def parser(f): while True: data = next(f) while True: @@ -227,7 +246,7 @@ perceived conveniences, but proper separation will make bugs more visible. An iterator is an object with a ``__next__`` method. Like many other -dunder methods, it may either return a value, or raise a specific +special methods, it may either return a value, or raise a specific exception - in this case, ``StopIteration`` - to signal that it has no value to return. In this, it is similar to ``__getattr__`` (can raise ``AttributeError``), ``__getitem__`` (can raise ``KeyError``), @@ -271,9 +290,9 @@ Transition plan deprecation warning if ``StopIteration`` bubbles out of a generator not under ``__future__`` import. -- Python 3.6: non-silent deprecation warning. +- Python 3.6: Non-silent deprecation warning. -- Python 3.7: enable new semantics everywhere. +- Python 3.7: Enable new semantics everywhere. Alternate proposals @@ -363,12 +382,12 @@ The existing model has been compared to the perfectly-acceptable issues inherent to every other case where an exception has special meaning. For instance, an unexpected ``KeyError`` inside a ``__getitem__`` method will be interpreted as failure, rather than -permitted to bubble up. However, there is a difference. Dunder +permitted to bubble up. However, there is a difference. Special methods use ``return`` to indicate normality, and ``raise`` to signal abnormality; generators ``yield`` to indicate data, and ``return`` to signal the abnormal state. This makes explicitly raising ``StopIteration`` entirely redundant, and potentially surprising. If -other dunder methods had dedicated keywords to distinguish between +other special methods had dedicated keywords to distinguish between their return paths, they too could turn unexpected exceptions into ``RuntimeError``; the fact that they cannot should not preclude generators from doing so.