218 lines
6.4 KiB
Plaintext
218 lines
6.4 KiB
Plaintext
PEP: 322
|
|
Title: Reverse Iteration Methods
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Raymond Hettinger <python@rcn.com>
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 24-Sep-2003
|
|
Python-Version: 2.4
|
|
Post-History: 24-Sep-2003
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This proposal is to extend the API of several sequence types
|
|
to include a method for iterating over the sequence in reverse.
|
|
|
|
|
|
Motivation
|
|
==========
|
|
|
|
For indexable objects, current approaches for reverse iteration are
|
|
error prone, unnatural, and not especially readable::
|
|
|
|
for i in xrange(n-1, -1, -1):
|
|
print seqn[i]
|
|
|
|
One other current approach involves reversing a list before iterating
|
|
over it. That technique wastes computer cycles, memory, and lines of
|
|
code::
|
|
|
|
rseqn = list(seqn)
|
|
rseqn.reverse()
|
|
for value in rseqn:
|
|
print value
|
|
|
|
Extended slicing is a third approach that minimizes the code overhead
|
|
but does nothing for memory efficiency, beauty, or clarity.
|
|
|
|
Reverse iteration is much less common than forward iteration, but it
|
|
does arise regularly in practice. See `Real World Use Cases`_ below.
|
|
|
|
|
|
Proposal
|
|
========
|
|
|
|
Add a method called *iterreverse()* to sequence objects that can
|
|
benefit from it. The above examples then simplify to::
|
|
|
|
for i in xrange(n).iterreverse():
|
|
print seqn[i]
|
|
|
|
::
|
|
|
|
for elem in seqn.iterreverse():
|
|
print elem
|
|
|
|
The new protocol would be applied to lists, tuples, strings, and
|
|
xrange objects. It would not apply to unordered collections like
|
|
dicts and sets.
|
|
|
|
No language syntax changes are needed.
|
|
|
|
|
|
Alternative Method Names
|
|
========================
|
|
|
|
* *iterbackwards* -- like iteritems() but somewhat long
|
|
* *backwards* -- more pithy, less explicit
|
|
* *ireverse* -- reminiscent of imap(), izip(), and ifilter()
|
|
|
|
|
|
Other Issues
|
|
============
|
|
|
|
* Should *tuple* objects be included? In the past, they have been
|
|
denied some list like behaviors such as count() and index(). I
|
|
prefer that it be included.
|
|
|
|
* Should *file* objects be included? Implementing reverse iteration
|
|
may not be easy though it would be useful on occasion. I think
|
|
this one should be skipped.
|
|
|
|
* Should *enumerate* objects be included? They can provide reverse
|
|
iteration only when the underlying sequences support *__len__*
|
|
and reverse iteration. I think this can be saved for another
|
|
day if the need arises.
|
|
|
|
|
|
Real World Use Cases
|
|
====================
|
|
|
|
Here are some instances of reverse iteration taken from the standard
|
|
library and comments on why reverse iteration was necessary:
|
|
|
|
* atexit.exit_handlers() uses::
|
|
|
|
while _exithandlers:
|
|
func, targs, kargs = _exithandlers.pop()
|
|
. . .
|
|
|
|
The application dictates the need to run exit handlers in the
|
|
reverse order they were built. The ``while alist: alist.pop()``
|
|
form is readable and clean; however, it would be slightly faster
|
|
and clearer with::
|
|
|
|
for func, target, kargs in _exithandlers.iterreverse():
|
|
. . .
|
|
del _exithandlers
|
|
|
|
Note, if the order of deletion is important, then the first form
|
|
is still needed.
|
|
|
|
* difflib.get_close_matches() uses::
|
|
|
|
result.sort() # Retain only the best n.
|
|
result = result[-n:] # Move best-scorer to head of list.
|
|
result.reverse() # Strip scores.
|
|
return [x for score, x in result]
|
|
|
|
The need for reverse iteration arises from a requirement to return
|
|
a portion of a sort in an order opposite of the sort criterion. The
|
|
list comprehension is incidental (the third step of a Schwartzian
|
|
transform). This particular use case can met with extended slicing,
|
|
but the code is somewhat unattractive, hard to visually verify,
|
|
and difficult for beginners to construct::
|
|
|
|
result.sort()
|
|
return [x for score, x in result[:-n-1:-1]]
|
|
|
|
The proposed form is much easier to construct and verify::
|
|
|
|
result.sort()
|
|
return [x for score, x in result[-n:].iterreverse()]
|
|
|
|
* heapq.heapify() uses ``for i in xrange(n//2 - 1, -1, -1)`` because
|
|
higher-level orderings are more easily formed from pairs of
|
|
lower-level orderings. A forward version of this algorithm is
|
|
possible; however, that would complicate the rest of the heap code
|
|
which iterates over the underlying list in the opposite direction.
|
|
|
|
* mhlib.test() uses::
|
|
|
|
testfolders.reverse();
|
|
for t in testfolders:
|
|
do('mh.deletefolder(%s)' % `t`)
|
|
|
|
The need for reverse iteration arises because the tail of the
|
|
underlying list is altered during iteration.
|
|
|
|
* platform._dist_try_harder() uses
|
|
``for n in range(len(verfiles)-1,-1,-1)`` because the loop deletes
|
|
selected elements from *verfiles* but needs to leave the rest of
|
|
the list intact for further iteration.
|
|
|
|
* random.shuffle() uses ``for i in xrange(len(x)-1, 0, -1)`` because
|
|
the algorithm is most easily understood as randomly selecting
|
|
elements from an ever diminishing pool. In fact, the algorithm can
|
|
be run in a forward direction but is less intuitive and rarely
|
|
presented that way in literature. The replacement code
|
|
``for i in xrange(1, len(x)).iterreverse()`` is much easier
|
|
to mentally verify.
|
|
|
|
* rfc822.Message.__delitem__() uses::
|
|
|
|
list.reverse()
|
|
for i in list:
|
|
del self.headers[i]
|
|
|
|
The need for reverse iteration arises because the tail of the
|
|
underlying list is altered during iteration.
|
|
|
|
|
|
Alternative Ideas
|
|
=================
|
|
|
|
* Add a builtin function, *riter()* which calls a magic method,
|
|
*__riter__*. I see this as more overhead for no additional benefit.
|
|
|
|
* Several variants were submitted that provided fallback behavior
|
|
when *__riter__* is not defined:
|
|
|
|
- fallback to: ``for i in xrange(len(obj)-1,-1,-1): yield obj[i]``
|
|
- fallback to: ``for i in itertools.count(): yield obj[-i]``
|
|
- fallback to: ``tmp=list(obj); tmp.reverse(); return iter(tmp)``
|
|
|
|
All of these attempt to save implementing some object methods at the
|
|
expense of adding a new builtin function and of creating a new magic
|
|
method name.
|
|
|
|
The approaches using *__getitem__()* are slower than using a custom
|
|
method for each object. Also, the *__getitem__()* variants produce
|
|
bizarre results when applied to mappings.
|
|
|
|
All of the variants crash when applied to an infinite iterator.
|
|
|
|
The last variant can invisibly slip into a low performance mode
|
|
(in terms of time and memory) which could be made more visible with
|
|
an explicit ``ro=list(obj); ro.reverse()``.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
End:
|