python-peps/pep-0322.txt

PEP: 322
Title: Reverse Iteration
Version: $Revision$
Last-Modified: $Date$
Author: Raymond Hettinger <python@rcn.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 24-Sep-2003
Python-Version: 2.4
Post-History: 24-Sep-2003


Abstract
========

This proposal is to add a builtin function to support reverse
iteration over sequences.


Motivation
==========

For indexable objects, current approaches for reverse iteration are
error prone, unnatural, and not especially readable::

    for i in xrange(n-1, -1, -1):
        print seqn[i]

One other current approach involves reversing a list before iterating
over it.  That technique wastes computer cycles, memory, and lines of
code::

    rseqn = list(seqn)
    rseqn.reverse()
    for value in rseqn:
        print value

Extended slicing is a third approach that minimizes the code overhead
but does nothing for memory efficiency, beauty, or clarity.

Reverse iteration is much less common than forward iteration, but it
does arise regularly in practice.  See `Real World Use Cases`_ below.


Proposal
========

Add a builtin function called *inreverse()* that makes a reverse
iterator over sequence objects that support __getitem__() and
__len__().

The above examples then simplify to::

    for i in inreverse(xrange(n)):
        print seqn[i]

::

    for elem in inreverse(seqn):
        print elem

The core idea is that the clearest, least error-prone way of specifying
reverse iteration is to specify it in a forward direction and then say
*inreverse*.

The implementation could be as simple as::

    def inreverse(x):
        i = len(x)
        while i > 0:
            i -= 1
            yield x[i]

If *x* is a mapping, the implementation should return a ValueError with
a message noting that reverse iteration is undefined for mappings.

No language syntax changes are needed.  The proposal is fully backwards
compatible.


Alternative Method Names
========================

* *backwards* -- more pithy, less explicit
* *ireverse* -- reminiscent of imap(), izip(), and ifilter()


Custom Reverse
==============

Objects may optionally provide an *__inreverse__* method that returns
a custom reverse iterator.

This allows reverse() to be applied to objects that do not have 
__getitem__() and __len__() but still have some useful way of
providing reverse iteration.

Using this protocol, enumerate() can be extended to support reversal
whenever the underlying iterator supports it also.


Real World Use Cases
====================

Here are some instances of reverse iteration taken from the standard
library and comments on why reverse iteration was necessary:

* atexit.exit_handlers() uses::

    while _exithandlers:
        func, targs, kargs = _exithandlers.pop()
            . . .

  In this application popping is required, so the new function would
  not help.

* heapq.heapify() uses ``for i in xrange(n//2 - 1, -1, -1)`` because
  higher-level orderings are more easily formed from pairs of
  lower-level orderings.  A forward version of this algorithm is
  possible; however, that would complicate the rest of the heap code
  which iterates over the underlying list in the opposite direction.

* mhlib.test() uses::

    testfolders.reverse();
    for t in testfolders:
        do('mh.deletefolder(%s)' % `t`)

  The need for reverse iteration arises because the tail of the
  underlying list is altered during iteration.

* platform._dist_try_harder() uses
  ``for n in range(len(verfiles)-1,-1,-1)`` because the loop deletes
  selected elements from *verfiles* but needs to leave the rest of
  the list intact for further iteration.

* random.shuffle() uses ``for i in xrange(len(x)-1, 0, -1)`` because
  the algorithm is most easily understood as randomly selecting
  elements from an ever diminishing pool.  In fact, the algorithm can
  be run in a forward direction but is less intuitive and rarely
  presented that way in literature.  The replacement code
  ``for i in inreverse(xrange(1, len(x)))`` is much easier
  to verify visually.

* rfc822.Message.__delitem__() uses::

    list.reverse()
    for i in list:
        del self.headers[i]

  The need for reverse iteration arises because the tail of the
  underlying list is altered during iteration.


Active Alternative
==================

A simpler, but limited alternative is to create a builtin that takes
the same arguments as range() but returns a reverse iterator over the
range.  The idea is that much of the benefit of inreverse() comes
reducing the intellectual effort it takes to express the arguments for
[x]range() when going backwards.  A good name is needed for this
alternative -- revrange() is cleanest so far.


Rejected Alternative
====================

Several variants were submitted that attempted to apply inreverse()
to all iterables by running the iterable to completion, saving the
results, and then returning a reverse iterator over the results.
While satisfying some notions of full generality, running the input
to the end is contrary to the purpose of using iterators
in the first place.  Also, a small disaster ensues if the underlying
iterator is infinite.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: