214 lines
7.9 KiB
ReStructuredText
214 lines
7.9 KiB
ReStructuredText
PEP: 3114
|
|
Title: Renaming iterator.next() to iterator.__next__()
|
|
Author: Ka-Ping Yee <ping@zesty.ca>
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 04-Mar-2007
|
|
Python-Version: 3.0
|
|
Post-History:
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
The iterator protocol in Python 2.x consists of two methods:
|
|
``__iter__()`` called on an iterable object to yield an iterator, and
|
|
``next()`` called on an iterator object to yield the next item in the
|
|
sequence. Using a ``for`` loop to iterate over an iterable object
|
|
implicitly calls both of these methods. This PEP proposes that the
|
|
``next`` method be renamed to ``__next__``, consistent with all the
|
|
other protocols in Python in which a method is implicitly called as
|
|
part of a language-level protocol, and that a built-in function named
|
|
``next`` be introduced to invoke ``__next__`` method, consistent with
|
|
the manner in which other protocols are explicitly invoked.
|
|
|
|
|
|
Names With Double Underscores
|
|
=============================
|
|
|
|
In Python, double underscores before and after a name are used to
|
|
distinguish names that belong to the language itself. Attributes and
|
|
methods that are implicitly used or created by the interpreter employ
|
|
this naming convention; some examples are:
|
|
|
|
* ``__file__`` - an attribute automatically created by the interpreter
|
|
|
|
* ``__dict__`` - an attribute with special meaning to the interpreter
|
|
|
|
* ``__init__`` - a method implicitly called by the interpreter
|
|
|
|
Note that this convention applies to methods such as ``__init__`` that
|
|
are explicitly defined by the programmer, as well as attributes such as
|
|
``__file__`` that can only be accessed by naming them explicitly, so it
|
|
includes names that are used *or* created by the interpreter.
|
|
|
|
(Not all things that are called "protocols" are made of methods with
|
|
double-underscore names. For example, the ``__contains__`` method has
|
|
double underscores because the language construct ``x in y`` implicitly
|
|
calls ``__contains__``. But even though the ``read`` method is part of
|
|
the file protocol, it does not have double underscores because there is
|
|
no language construct that implicitly invokes ``x.read()``.)
|
|
|
|
The use of double underscores creates a separate namespace for names
|
|
that are part of the Python language definition, so that programmers
|
|
are free to create variables, attributes, and methods that start with
|
|
letters, without fear of silently colliding with names that have a
|
|
language-defined purpose. (Colliding with reserved keywords is still
|
|
a concern, but at least this will immediately yield a syntax error.)
|
|
|
|
The naming of the ``next`` method on iterators is an exception to
|
|
this convention. Code that nowhere contains an explicit call to a
|
|
``next`` method can nonetheless be silently affected by the presence
|
|
of such a method. Therefore, this PEP proposes that iterators should
|
|
have a ``__next__`` method instead of a ``next`` method (with no
|
|
change in semantics).
|
|
|
|
|
|
Double-Underscore Methods and Built-In Functions
|
|
================================================
|
|
|
|
The Python language defines several protocols that are implemented or
|
|
customized by defining methods with double-underscore names. In each
|
|
case, the protocol is provided by an internal method implemented as a
|
|
C function in the interpreter. For objects defined in Python, this
|
|
C function supports customization by implicitly invoking a Python method
|
|
with a double-underscore name (it often does a little bit of additional
|
|
work beyond just calling the Python method.)
|
|
|
|
Sometimes the protocol is invoked by a syntactic construct:
|
|
|
|
* ``x[y]`` --> internal ``tp_getitem`` --> ``x.__getitem__(y)``
|
|
|
|
* ``x + y`` --> internal ``nb_add`` --> ``x.__add__(y)``
|
|
|
|
* ``-x`` --> internal ``nb_negative`` --> ``x.__neg__()``
|
|
|
|
Sometimes there is no syntactic construct, but it is still useful to be
|
|
able to explicitly invoke the protocol. For such cases Python offers a
|
|
built-in function of the same name but without the double underscores.
|
|
|
|
* ``len(x)`` --> internal ``sq_length`` --> ``x.__len__()``
|
|
|
|
* ``hash(x)`` --> internal ``tp_hash`` --> ``x.__hash__()``
|
|
|
|
* ``iter(x)`` --> internal ``tp_iter`` --> ``x.__iter__()``
|
|
|
|
Following this pattern, the natural way to handle ``next`` is to add a
|
|
``next`` built-in function that behaves in exactly the same fashion.
|
|
|
|
* ``next(x)`` --> internal ``tp_iternext`` --> ``x.__next__()``
|
|
|
|
Further, it is proposed that the ``next`` built-in function accept a
|
|
sentinel value as an optional second argument, following the style of
|
|
the ``getattr`` and ``iter`` built-in functions. When called with two
|
|
arguments, ``next`` catches the StopIteration exception and returns
|
|
the sentinel value instead of propagating the exception. This creates
|
|
a nice duality between ``iter`` and ``next``:
|
|
|
|
iter(function, sentinel) <--> next(iterator, sentinel)
|
|
|
|
|
|
Previous Proposals
|
|
==================
|
|
|
|
This proposal is not a new idea. The idea proposed here was supported
|
|
by the BDFL on python-dev [1]_ and is even mentioned in the original
|
|
iterator PEP, :pep:`234`::
|
|
|
|
(In retrospect, it might have been better to go for __next__()
|
|
and have a new built-in, next(it), which calls it.__next__().
|
|
But alas, it's too late; this has been deployed in Python 2.2
|
|
since December 2001.)
|
|
|
|
|
|
Objections
|
|
==========
|
|
|
|
There have been a few objections to the addition of more built-ins.
|
|
In particular, Martin von Loewis writes [2]_::
|
|
|
|
I dislike the introduction of more builtins unless they have a true
|
|
generality (i.e. are likely to be needed in many programs). For this
|
|
one, I think the normal usage of __next__ will be with a for loop, so
|
|
I don't think one would often need an explicit next() invocation.
|
|
|
|
It is also not true that most protocols are explicitly invoked through
|
|
builtin functions. Instead, most protocols are can be explicitly invoked
|
|
through methods in the operator module. So following tradition, it
|
|
should be operator.next.
|
|
|
|
...
|
|
|
|
As an alternative, I propose that object grows a .next() method,
|
|
which calls __next__ by default.
|
|
|
|
|
|
Transition Plan
|
|
===============
|
|
|
|
Two additional transformations will be added to the 2to3 translation
|
|
tool [3]_:
|
|
|
|
* Method definitions named ``next`` will be renamed to ``__next__``.
|
|
|
|
* Explicit calls to the ``next`` method will be replaced with calls
|
|
to the built-in ``next`` function. For example, ``x.next()`` will
|
|
become ``next(x)``.
|
|
|
|
Collin Winter looked into the possibility of automatically deciding
|
|
whether to perform the second transformation depending on the presence
|
|
of a module-level binding to ``next`` [4]_ and found that it would be
|
|
"ugly and slow". Instead, the translation tool will emit warnings
|
|
upon detecting such a binding. Collin has proposed warnings for the
|
|
following conditions [5]_:
|
|
|
|
* Module-level assignments to ``next``.
|
|
|
|
* Module-level definitions of a function named ``next``.
|
|
|
|
* Module-level imports of the name ``next``.
|
|
|
|
* Assignments to ``__builtin__.next``.
|
|
|
|
|
|
Approval
|
|
========
|
|
|
|
This PEP was accepted by Guido on March 6, 2007 [6]_.
|
|
|
|
|
|
Implementation
|
|
==============
|
|
|
|
A patch with the necessary changes (except the 2to3 tool) was written
|
|
by Georg Brandl and committed as revision 54910.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] Single- vs. Multi-pass iterability (Guido van Rossum)
|
|
https://mail.python.org/pipermail/python-dev/2002-July/026814.html
|
|
|
|
.. [2] PEP: rename it.next() to it.__next__()... (Martin von Loewis)
|
|
https://mail.python.org/pipermail/python-3000/2007-March/005965.html
|
|
|
|
.. [3] 2to3 refactoring tool
|
|
https://github.com/python/cpython/tree/ef04c44e29a8276a484f58d03a75a2dec516302d/Lib/lib2to3
|
|
|
|
.. [4] PEP: rename it.next() to it.__next__()... (Collin Winter)
|
|
https://mail.python.org/pipermail/python-3000/2007-March/006020.html
|
|
|
|
.. [5] PEP 3113 transition plan
|
|
https://mail.python.org/pipermail/python-3000/2007-March/006044.html
|
|
|
|
.. [6] PEP: rename it.next() to it.__next__()... (Guido van Rossum)
|
|
https://mail.python.org/pipermail/python-3000/2007-March/006027.html
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|