227 lines
8.0 KiB
Plaintext
227 lines
8.0 KiB
Plaintext
PEP: 3114
|
||
Title: Renaming iterator.next() to iterator.__next__()
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Ka-Ping Yee <ping@zesty.ca>
|
||
Status: Final
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 04-Mar-2007
|
||
Python-Version: 3.0
|
||
Post-History:
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
The iterator protocol in Python 2.x consists of two methods:
|
||
``__iter__()`` called on an iterable object to yield an iterator, and
|
||
``next()`` called on an iterator object to yield the next item in the
|
||
sequence. Using a ``for`` loop to iterate over an iterable object
|
||
implicitly calls both of these methods. This PEP proposes that the
|
||
``next`` method be renamed to ``__next__``, consistent with all the
|
||
other protocols in Python in which a method is implicitly called as
|
||
part of a language-level protocol, and that a built-in function named
|
||
``next`` be introduced to invoke ``__next__`` method, consistent with
|
||
the manner in which other protocols are explicitly invoked.
|
||
|
||
|
||
Names With Double Underscores
|
||
=============================
|
||
|
||
In Python, double underscores before and after a name are used to
|
||
distinguish names that belong to the language itself. Attributes and
|
||
methods that are implicitly used or created by the interpreter employ
|
||
this naming convention; some examples are:
|
||
|
||
* ``__file__`` - an attribute automatically created by the interpreter
|
||
|
||
* ``__dict__`` - an attribute with special meaning to the interpreter
|
||
|
||
* ``__init__`` - a method implicitly called by the interpreter
|
||
|
||
Note that this convention applies to methods such as ``__init__`` that
|
||
are explicitly defined by the programmer, as well as attributes such as
|
||
``__file__`` that can only be accessed by naming them explicitly, so it
|
||
includes names that are used *or* created by the interpreter.
|
||
|
||
(Not all things that are called "protocols" are made of methods with
|
||
double-underscore names. For example, the ``__contains__`` method has
|
||
double underscores because the language construct ``x in y`` implicitly
|
||
calls ``__contains__``. But even though the ``read`` method is part of
|
||
the file protocol, it does not have double underscores because there is
|
||
no language construct that implicitly invokes ``x.read()``.)
|
||
|
||
The use of double underscores creates a separate namespace for names
|
||
that are part of the Python language definition, so that programmers
|
||
are free to create variables, attributes, and methods that start with
|
||
letters, without fear of silently colliding with names that have a
|
||
language-defined purpose. (Colliding with reserved keywords is still
|
||
a concern, but at least this will immediately yield a syntax error.)
|
||
|
||
The naming of the ``next`` method on iterators is an exception to
|
||
this convention. Code that nowhere contains an explicit call to a
|
||
``next`` method can nonetheless be silently affected by the presence
|
||
of such a method. Therefore, this PEP proposes that iterators should
|
||
have a ``__next__`` method instead of a ``next`` method (with no
|
||
change in semantics).
|
||
|
||
|
||
Double-Underscore Methods and Built-In Functions
|
||
================================================
|
||
|
||
The Python language defines several protocols that are implemented or
|
||
customized by defining methods with double-underscore names. In each
|
||
case, the protocol is provided by an internal method implemented as a
|
||
C function in the interpreter. For objects defined in Python, this
|
||
C function supports customization by implicitly invoking a Python method
|
||
with a double-underscore name (it often does a little bit of additional
|
||
work beyond just calling the Python method.)
|
||
|
||
Sometimes the protocol is invoked by a syntactic construct:
|
||
|
||
* ``x[y]`` --> internal ``tp_getitem`` --> ``x.__getitem__(y)``
|
||
|
||
* ``x + y`` --> internal ``nb_add`` --> ``x.__add__(y)``
|
||
|
||
* ``-x`` --> internal ``nb_negative`` --> ``x.__neg__()``
|
||
|
||
Sometimes there is no syntactic construct, but it is still useful to be
|
||
able to explicitly invoke the protocol. For such cases Python offers a
|
||
built-in function of the same name but without the double underscores.
|
||
|
||
* ``len(x)`` --> internal ``sq_length`` --> ``x.__len__()``
|
||
|
||
* ``hash(x)`` --> internal ``tp_hash`` --> ``x.__hash__()``
|
||
|
||
* ``iter(x)`` --> internal ``tp_iter`` --> ``x.__iter__()``
|
||
|
||
Following this pattern, the natural way to handle ``next`` is to add a
|
||
``next`` built-in function that behaves in exactly the same fashion.
|
||
|
||
* ``next(x)`` --> internal ``tp_iternext`` --> ``x.__next__()``
|
||
|
||
Further, it is proposed that the ``next`` built-in function accept a
|
||
sentinel value as an optional second argument, following the style of
|
||
the ``getattr`` and ``iter`` built-in functions. When called with two
|
||
arguments, ``next`` catches the StopIteration exception and returns
|
||
the sentinel value instead of propagating the exception. This creates
|
||
a nice duality between ``iter`` and ``next``:
|
||
|
||
iter(function, sentinel) <--> next(iterator, sentinel)
|
||
|
||
|
||
Previous Proposals
|
||
==================
|
||
|
||
This proposal is not a new idea. The idea proposed here was supported
|
||
by the BDFL on python-dev [1]_ and is even mentioned in the original
|
||
iterator PEP, :pep:`234`::
|
||
|
||
(In retrospect, it might have been better to go for __next__()
|
||
and have a new built-in, next(it), which calls it.__next__().
|
||
But alas, it's too late; this has been deployed in Python 2.2
|
||
since December 2001.)
|
||
|
||
|
||
Objections
|
||
==========
|
||
|
||
There have been a few objections to the addition of more built-ins.
|
||
In particular, Martin von Loewis writes [2]_::
|
||
|
||
I dislike the introduction of more builtins unless they have a true
|
||
generality (i.e. are likely to be needed in many programs). For this
|
||
one, I think the normal usage of __next__ will be with a for loop, so
|
||
I don't think one would often need an explicit next() invocation.
|
||
|
||
It is also not true that most protocols are explicitly invoked through
|
||
builtin functions. Instead, most protocols are can be explicitly invoked
|
||
through methods in the operator module. So following tradition, it
|
||
should be operator.next.
|
||
|
||
...
|
||
|
||
As an alternative, I propose that object grows a .next() method,
|
||
which calls __next__ by default.
|
||
|
||
|
||
Transition Plan
|
||
===============
|
||
|
||
Two additional transformations will be added to the 2to3 translation
|
||
tool [3]_:
|
||
|
||
* Method definitions named ``next`` will be renamed to ``__next__``.
|
||
|
||
* Explicit calls to the ``next`` method will be replaced with calls
|
||
to the built-in ``next`` function. For example, ``x.next()`` will
|
||
become ``next(x)``.
|
||
|
||
Collin Winter looked into the possibility of automatically deciding
|
||
whether to perform the second transformation depending on the presence
|
||
of a module-level binding to ``next`` [4]_ and found that it would be
|
||
"ugly and slow". Instead, the translation tool will emit warnings
|
||
upon detecting such a binding. Collin has proposed warnings for the
|
||
following conditions [5]_:
|
||
|
||
* Module-level assignments to ``next``.
|
||
|
||
* Module-level definitions of a function named ``next``.
|
||
|
||
* Module-level imports of the name ``next``.
|
||
|
||
* Assignments to ``__builtin__.next``.
|
||
|
||
|
||
Approval
|
||
========
|
||
|
||
This PEP was accepted by Guido on March 6, 2007 [5]_.
|
||
|
||
|
||
Implementation
|
||
==============
|
||
|
||
A patch with the necessary changes (except the 2to3 tool) was written
|
||
by Georg Brandl and committed as revision 54910.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] Single- vs. Multi-pass iterability (Guido van Rossum)
|
||
https://mail.python.org/pipermail/python-dev/2002-July/026814.html
|
||
|
||
.. [2] PEP: rename it.next() to it.__next__()... (Martin von Loewis)
|
||
https://mail.python.org/pipermail/python-3000/2007-March/005965.html
|
||
|
||
.. [3] 2to3 refactoring tool
|
||
http://svn.python.org/view/sandbox/trunk/2to3/
|
||
|
||
.. [4] PEP: rename it.next() to it.__next__()... (Collin Winter)
|
||
https://mail.python.org/pipermail/python-3000/2007-March/006020.html
|
||
|
||
.. [5] PEP 3113 transition plan
|
||
https://mail.python.org/pipermail/python-3000/2007-March/006044.html
|
||
|
||
.. [6] PEP: rename it.next() to it.__next__()... (Guido van Rossum)
|
||
https://mail.python.org/pipermail/python-3000/2007-March/006027.html
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|