python-peps/peps/pep-0309.rst

PEP: 309
Title: Partial Function Application
Version: $Revision$
Last-Modified: $Date$
Author: Peter Harris <scav@blueyonder.co.uk>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 08-Feb-2003
Python-Version: 2.5
Post-History: 10-Feb-2003, 27-Feb-2003, 22-Feb-2004, 28-Apr-2006


Note
====

Following the acceptance of this PEP, further discussion on python-dev and
comp.lang.python revealed a desire for several tools that operated on
function objects, but were not related to functional programming.  Rather
than create a new module for these tools, it was agreed [1]_ that the
"functional" module be renamed to "functools" to reflect its newly-widened
focus.

References in this PEP to a "functional" module have been left in for
historical reasons.


Abstract
========

This proposal is for a function or callable class that allows a new
callable to be constructed from a callable and a partial argument list
(including positional and keyword arguments).

I propose a standard library module called "functional", to hold
useful higher-order functions, including the implementation of
partial().

An implementation has been submitted to SourceForge [2]_.


Acceptance
==========

Patch #941881 was accepted and applied in 2005 for Py2.5.  It is
essentially as outlined here, a partial() type constructor binding
leftmost positional arguments and any keywords.  The partial object has
three read-only attributes func, args, and keywords.  Calls to the partial
object can specify keywords that override those in the object itself.

There is a separate and continuing discussion of whether to modify the
partial implementation with a __get__ method to more closely emulate
the behavior of an equivalent function.

Motivation
==========

In functional programming, function currying is a way of implementing
multi-argument functions in terms of single-argument functions.  A
function with N arguments is really a function with 1 argument that
returns another function taking (N-1) arguments.  Function application
in languages like Haskell and ML works such that a function call::

    f x y z

actually means::

   (((f x) y) z)

This would be only an obscure theoretical issue except that in actual
programming it turns out to be very useful.  Expressing a function in
terms of partial application of arguments to another function can be
both elegant and powerful, and in functional languages it is heavily
used.

In some functional languages, (e.g. Miranda) you can use an expression
such as ``(+1)`` to mean the equivalent of Python's
``(lambda x: x + 1)``.

In general, languages like that are strongly typed, so the compiler
always knows the number of arguments expected and can do the right
thing when presented with a functor and less arguments than expected.

Python does not implement multi-argument functions by currying, so if
you want a function with partially-applied arguments you would
probably use a lambda as above, or define a named function for each
instance.

However, lambda syntax is not to everyone's taste, so say the least.
Furthermore, Python's flexible parameter passing using both positional
and keyword presents an opportunity to generalise the idea of partial
application and do things that lambda cannot.


Example Implementation
======================

Here is one way to do a create a callable with partially-applied
arguments in Python.  The implementation below is based on improvements
provided by Scott David Daniels::

    class partial(object):

        def __init__(*args, **kw):
            self = args[0]
            self.fn, self.args, self.kw = (args[1], args[2:], kw)

        def __call__(self, *args, **kw):
            if kw and self.kw:
                d = self.kw.copy()
                d.update(kw)
            else:
                d = kw or self.kw
            return self.fn(*(self.args + args), **d)

(A recipe similar to this has been in the Python Cookbook for some
time [3]_.)

Note that when the object is called as though it were a function,
positional arguments are appended to those provided to the
constructor, and keyword arguments override and augment those provided
to the constructor.

Positional arguments, keyword arguments or both can be supplied at
when creating the object and when calling it.


Examples of Use
===============

So ``partial(operator.add, 1)`` is a bit like ``(lambda x: 1 + x)``.
Not an example where you see the benefits, of course.

Note too, that you could wrap a class in the same way, since classes
themselves are callable factories for objects.  So in some cases,
rather than defining a subclass, you can specialise classes by partial
application of the arguments to the constructor.

For example, ``partial(Tkinter.Label, fg='blue')`` makes Tkinter
Labels that have a blue foreground by default.

Here's a simple example that uses partial application to construct
callbacks for Tkinter widgets on the fly::

    from Tkinter import Tk, Canvas, Button
    import sys
    from functional import partial

    win = Tk()
    c = Canvas(win,width=200,height=50)
    c.pack()

    for colour in sys.argv[1:]:
        b = Button(win, text=colour,
                   command=partial(c.config, bg=colour))
        b.pack(side='left')

    win.mainloop()


Abandoned Syntax Proposal
=========================

I originally suggested the syntax ``fn@(*args, **kw)``, meaning the
same as ``partial(fn, *args, **kw)``.

The @ sign is used in some assembly languages to imply register
indirection, and the use here is also a kind of indirection.
``f@(x)`` is not ``f(x)``, but a thing that becomes ``f(x)`` when you
call it.

It was not well-received, so I have withdrawn this part of the
proposal.  In any case, @ has been taken for the new decorator syntax.


Feedback from comp.lang.python and python-dev
=============================================

Among the opinions voiced were the following (which I summarise):

* Lambda is good enough.

* The @ syntax is ugly (unanimous).

* It's really a curry rather than a closure.  There is an almost
  identical implementation of a curry class on ActiveState's Python
  Cookbook.

* A curry class would indeed be a useful addition to the standard
  library.

* It isn't function currying, but partial application.  Hence the
  name is now proposed to be partial().

* It maybe isn't useful enough to be in the built-ins.

* The idea of a module called ``functional`` was well received, and
  there are other things that belong there (for example function
  composition).

* For completeness, another object that appends partial arguments
  after those supplied in the function call (maybe called
  ``rightcurry``) has been suggested.

I agree that lambda is usually good enough, just not always.  And I
want the possibility of useful introspection and subclassing.

I disagree that @ is particularly ugly, but it may be that I'm just
weird.  We have dictionary, list and tuple literals neatly
differentiated by special punctuation -- a way of directly expressing
partially-applied function literals is not such a stretch.  However,
not one single person has said they like it, so as far as I'm
concerned it's a dead parrot.

I concur with calling the class partial rather than curry or closure,
so I have amended the proposal in this PEP accordingly.  But not
throughout: some incorrect references to 'curry' have been left in
since that's where the discussion was at the time.

Partially applying arguments from the right, or inserting arguments at
arbitrary positions creates its own problems, but pending discovery of
a good implementation and non-confusing semantics, I don't think it
should be ruled out.

Carl Banks posted an implementation as a real functional closure::

    def curry(fn, *cargs, **ckwargs):
        def call_fn(*fargs, **fkwargs):
            d = ckwargs.copy()
            d.update(fkwargs)
            return fn(*(cargs + fargs), **d)
        return call_fn

which he assures me is more efficient.

I also coded the class in Pyrex, to estimate how the performance
might be improved by coding it in C::

    cdef class curry:

        cdef object fn, args, kw

        def __init__(self, fn, *args, **kw):
            self.fn=fn
            self.args=args
            self.kw = kw

        def __call__(self, *args, **kw):
            if self.kw:        # from Python Cookbook version
                d = self.kw.copy()
                d.update(kw)
            else:
                d=kw
            return self.fn(*(self.args + args), **d)

The performance gain in Pyrex is less than 100% over the nested
function implementation, since to be fully general it has to operate
by Python API calls.  For the same reason, a C implementation will be
unlikely to be much faster, so the case for a built-in coded in C is
not very strong.


Summary
=======

I prefer that some means to partially-apply functions and other
callables should be present in the standard library.

A standard library module ``functional`` should contain an
implementation of ``partial``, and any other higher-order functions
the community want.  Other functions that might belong there fall
outside the scope of this PEP though.

Patches for the implementation, documentation and unit tests (SF
patches 931005_, 931007_, and 931010_ respectively) have been
submitted but not yet checked in.

A C implementation by Hye-Shik Chang has also been submitted, although
it is not expected to be included until after the Python
implementation has proven itself useful enough to be worth optimising.


References
==========

.. [1] https://mail.python.org/pipermail/python-dev/2006-March/062290.html

.. [2] Patches 931005_, 931007_, and 931010_.

.. [3] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52549

.. _931005: https://bugs.python.org/issue931005
.. _931007: https://bugs.python.org/issue931007
.. _931010: https://bugs.python.org/issue931010


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: