Add a new PEP for reverse iteration methods
This commit is contained in:
parent
9b8f2c641d
commit
0876789232
|
@ -117,6 +117,7 @@ Index by Category
|
||||||
S 318 Function/Method Decorator Syntax Smith
|
S 318 Function/Method Decorator Syntax Smith
|
||||||
S 319 Python Synchronize/Asynchronize Block Pelletier
|
S 319 Python Synchronize/Asynchronize Block Pelletier
|
||||||
S 321 Date/Time Parsing and Formatting Kuchling
|
S 321 Date/Time Parsing and Formatting Kuchling
|
||||||
|
S 322 Reverse Iteration Methods Hettinger
|
||||||
S 754 IEEE 754 Floating Point Special Values Warnes
|
S 754 IEEE 754 Floating Point Special Values Warnes
|
||||||
|
|
||||||
Finished PEPs (done, implemented in CVS)
|
Finished PEPs (done, implemented in CVS)
|
||||||
|
@ -335,6 +336,7 @@ Numerical Index
|
||||||
S 319 Python Synchronize/Asynchronize Block Pelletier
|
S 319 Python Synchronize/Asynchronize Block Pelletier
|
||||||
I 320 Python 2.4 Release Schedule Warsaw
|
I 320 Python 2.4 Release Schedule Warsaw
|
||||||
S 321 Date/Time Parsing and Formatting Kuchling
|
S 321 Date/Time Parsing and Formatting Kuchling
|
||||||
|
S 322 Reverse Iteration Methods Hettinger
|
||||||
SR 666 Reject Foolish Indentation Creighton
|
SR 666 Reject Foolish Indentation Creighton
|
||||||
S 754 IEEE 754 Floating Point Special Values Warnes
|
S 754 IEEE 754 Floating Point Special Values Warnes
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,207 @@
|
||||||
|
PEP: 322
|
||||||
|
Title: Reverse Iteration Methods
|
||||||
|
Version: $Revision$
|
||||||
|
Last-Modified: $Date$
|
||||||
|
Author: Raymond Hettinger <python@rcn.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 24-Sep-2003
|
||||||
|
Python-Version: 2.4
|
||||||
|
Post-History: 24-Sep-2003
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
This proposal is to extend the API of several sequence types
|
||||||
|
to include methods for iterating over the sequence in reverse.
|
||||||
|
|
||||||
|
|
||||||
|
Motivation
|
||||||
|
==========
|
||||||
|
|
||||||
|
For indexable objects, current approaches for reverse iteration are
|
||||||
|
error prone, unnatural, and not especially readable::
|
||||||
|
|
||||||
|
for i in xrange(n-1, -1, -1):
|
||||||
|
print seqn[i]
|
||||||
|
|
||||||
|
One other current approach involves reversing a list before iterating
|
||||||
|
over it. That technique wastes computer cycles, memory, and lines of
|
||||||
|
code. Also, it only works with lists (strings, for example, do not
|
||||||
|
define a reverse method)::
|
||||||
|
|
||||||
|
rseqn = list(seqn)
|
||||||
|
rseqn.reverse()
|
||||||
|
for value in rseqn:
|
||||||
|
print value
|
||||||
|
|
||||||
|
Extending slicing minimizes the code overhead but does nothing for
|
||||||
|
memory efficiency, beauty, or clarity.
|
||||||
|
|
||||||
|
Reverse iteration is much less common than forward iteration, but it
|
||||||
|
does arise regularly in practice. See `Real World Use Cases`_ below.
|
||||||
|
|
||||||
|
|
||||||
|
Proposal
|
||||||
|
========
|
||||||
|
|
||||||
|
Add a method called *iter_backwards()* to sequence objects that can
|
||||||
|
benefit from it. The above examples then simplify to::
|
||||||
|
|
||||||
|
for i in xrange(n).iter_backwards():
|
||||||
|
print seqn[i]
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
for elem in seqn.iter_backwards():
|
||||||
|
print elem
|
||||||
|
|
||||||
|
The new protocol would be applied to lists, strings, xrange objects,
|
||||||
|
and possibly other sequence objects as well (depending on use cases
|
||||||
|
and implementation issues). It would not apply to unordered
|
||||||
|
collections like dicts and sets.
|
||||||
|
|
||||||
|
No language syntax changes are needed.
|
||||||
|
|
||||||
|
|
||||||
|
Alternative Method Names
|
||||||
|
========================
|
||||||
|
|
||||||
|
* *iterbackwards* -- like iteritems() but somewhat long
|
||||||
|
* *backwards* -- more pithy, less explicit
|
||||||
|
* *ireverse* -- reminiscent of imap(), izip(), and ifilter()
|
||||||
|
|
||||||
|
|
||||||
|
Open Issues
|
||||||
|
===========
|
||||||
|
|
||||||
|
* Should *tuple* objects be included? In the past, they have been
|
||||||
|
denied some list like behaviors such as count() and index().
|
||||||
|
|
||||||
|
* Should *file* objects be included? Implementing reverse iteration
|
||||||
|
may not be easy though it would be useful on occasion.
|
||||||
|
|
||||||
|
* Should *enumerate* objects be included? They can provide reverse
|
||||||
|
iteration only when the underlying sequences support *__len__*
|
||||||
|
and reverse iteration.
|
||||||
|
|
||||||
|
|
||||||
|
Real World Use Cases
|
||||||
|
====================
|
||||||
|
|
||||||
|
Here are some instances of reverse iteration taken from the standard
|
||||||
|
library and comments on why reverse iteration was necessary:
|
||||||
|
|
||||||
|
* atexit.exit_handlers() uses::
|
||||||
|
|
||||||
|
while _exithandlers:
|
||||||
|
func, targs, kargs = _exithandlers.pop()
|
||||||
|
. . .
|
||||||
|
|
||||||
|
The application dictates the need to run exit handlers in the
|
||||||
|
reverse order they were built. The ``while alist: alist.pop()``
|
||||||
|
form is readable and clean; however, it would be slightly faster
|
||||||
|
and clearer with::
|
||||||
|
|
||||||
|
for func, target, kargs in _exithandlers.iter_backwards():
|
||||||
|
. . .
|
||||||
|
del _exithandlers
|
||||||
|
|
||||||
|
* difflib.get_close_matches() uses::
|
||||||
|
|
||||||
|
result.sort() # Retain only the best n.
|
||||||
|
result = result[-n:] # Move best-scorer to head of list.
|
||||||
|
result.reverse() # Strip scores.
|
||||||
|
return [x for score, x in result]
|
||||||
|
|
||||||
|
The need for reverse iteration arises from a requirement to return
|
||||||
|
a portion of a sort in an order opposite of the sort criterion. The
|
||||||
|
list comprehension is incidental (the third step of a Schwartzian
|
||||||
|
transform). This particular use case can met with extended slicing,
|
||||||
|
but the code is somewhat unattractive, hard to visually verify,
|
||||||
|
and difficult for beginners to construct::
|
||||||
|
|
||||||
|
result.sort()
|
||||||
|
return [x for score, x in result[:-n-1:-1]]
|
||||||
|
|
||||||
|
The proposed form is much easier to construct and verify::
|
||||||
|
|
||||||
|
result.sort()
|
||||||
|
return [x for score, x in result[-n:].iter_backwards()]
|
||||||
|
|
||||||
|
* heapq.heapify() uses ``for i in xrange(n//2 - 1, -1, -1)`` because
|
||||||
|
higher-level orderings are more easily formed from pairs of
|
||||||
|
lower-level orderings. A forward version of this algorithm is
|
||||||
|
possible; however, that would complicate the rest of the heap code
|
||||||
|
which iterates over the underlying list in the opposite direction.
|
||||||
|
|
||||||
|
* mhlib.test() uses::
|
||||||
|
|
||||||
|
testfolders.reverse();
|
||||||
|
for t in testfolders:
|
||||||
|
do('mh.deletefolder(%s)' % `t`)
|
||||||
|
|
||||||
|
The need for reverse iteration arises because the tail of the
|
||||||
|
underlying list is altered during iteration.
|
||||||
|
|
||||||
|
* platform._dist_try_harder() uses
|
||||||
|
``for n in range(len(verfiles)-1,-1,-1)`` because the loop deletes
|
||||||
|
selected elements from *verfiles* but needs to leave the rest of
|
||||||
|
the list intact for further iteration. This use case could be
|
||||||
|
addressed with *itertools.ifilter()* but would require the
|
||||||
|
selection predicate to be in a *lambda* expression. The net
|
||||||
|
result is less clear and readable than the original. A better
|
||||||
|
reformulation is to replace the first line with the proposed
|
||||||
|
method.
|
||||||
|
|
||||||
|
* random.shuffle() uses ``for i in xrange(len(x)-1, 0, -1)`` because
|
||||||
|
the algorithm is most easily understood as randomly selecting
|
||||||
|
elements from an ever diminishing pool. In fact, the algorithm can
|
||||||
|
be run in a forward direction but is less intuitive and rarely
|
||||||
|
presented that way in literature.
|
||||||
|
|
||||||
|
* rfc822.Message.__delitem__() uses::
|
||||||
|
|
||||||
|
list.reverse()
|
||||||
|
for i in list:
|
||||||
|
del self.headers[i]
|
||||||
|
|
||||||
|
The need for reverse iteration arises because the tail of the
|
||||||
|
underlying list is altered during iteration.
|
||||||
|
|
||||||
|
|
||||||
|
Rejected Alternative Ideas
|
||||||
|
==========================
|
||||||
|
|
||||||
|
* Add a builtin function, *reverse()* which calls a magic method,
|
||||||
|
__riter__. I see this as more overhead for no additional benefit.
|
||||||
|
|
||||||
|
* Add a builtin function, *reverse()* which does the above, and
|
||||||
|
if *__riter__* is not found, constructs its own using
|
||||||
|
*__getitem__*, and if *__getitem__* is not found, builds a list
|
||||||
|
from *__iter__* and returns a reverse iterator over the new list.
|
||||||
|
The advantage is that one function takes care of almost everything
|
||||||
|
that is potentially reversible. A disadvantage is that it can
|
||||||
|
invisibility slip in to a low performance mode (in terms of time
|
||||||
|
and memory) which would be more visible with an explicit
|
||||||
|
``list(obj).reverse()``. Another problem is that *__getitem__*
|
||||||
|
is also used in mappings as well as sequences and that could lead
|
||||||
|
to bizarre results.
|
||||||
|
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
End:
|
Loading…
Reference in New Issue