278 lines
10 KiB
Plaintext
278 lines
10 KiB
Plaintext
PEP: 443
|
||
Title: Single-dispatch generic functions
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Łukasz Langa <lukasz@langa.pl>
|
||
Discussions-To: Python-Dev <python-dev@python.org>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 22-May-2013
|
||
Post-History: 22-May-2013
|
||
Replaces: 245, 246, 3124
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
This PEP proposes a new mechanism in the ``functools`` standard library
|
||
module that provides a simple form of generic programming known as
|
||
single-dispatch generic functions.
|
||
|
||
A **generic function** is composed of multiple functions sharing the
|
||
same name. Which form should be used during a call is determined by the
|
||
dispatch algorithm. When the implementation is chosen based on the type
|
||
of a single argument, this is known as **single dispatch**.
|
||
|
||
|
||
Rationale and Goals
|
||
===================
|
||
|
||
Python has always provided a variety of built-in and standard-library
|
||
generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``,
|
||
``copy.copy()``, and most of the functions in the ``operator`` module.
|
||
However, it currently:
|
||
|
||
1. does not have a simple or straightforward way for developers to
|
||
create new generic functions,
|
||
|
||
2. does not have a standard way for methods to be added to existing
|
||
generic functions (i.e., some are added using registration
|
||
functions, others require defining ``__special__`` methods, possibly
|
||
by monkeypatching).
|
||
|
||
In addition, it is currently a common anti-pattern for Python code to
|
||
inspect the types of received arguments, in order to decide what to do
|
||
with the objects. For example, code may wish to accept either an object
|
||
of some type, or a sequence of objects of that type.
|
||
|
||
Currently, the "obvious way" to do this is by type inspection, but this
|
||
is brittle and closed to extension. Abstract Base Classes make it easier
|
||
to discover present behaviour, but don't help adding new behaviour.
|
||
A developer using an already-written library may be unable to change how
|
||
their objects are treated by such code, especially if the objects they
|
||
are using were created by a third party.
|
||
|
||
Therefore, this PEP proposes a uniform API to address dynamic
|
||
overloading using decorators.
|
||
|
||
|
||
User API
|
||
========
|
||
|
||
To define a generic function, decorate it with the ``@singledispatch``
|
||
decorator. Note that the dispatch happens on the type of the first
|
||
argument, create your function accordingly::
|
||
|
||
>>> from functools import singledispatch
|
||
>>> @singledispatch
|
||
... def fun(arg, verbose=False):
|
||
... if verbose:
|
||
... print("Let me just say,", end=" ")
|
||
... print(arg)
|
||
|
||
To add overloaded implementations to the function, use the
|
||
``register()`` attribute of the generic function. It takes a type
|
||
parameter::
|
||
|
||
>>> @fun.register(int)
|
||
... def _(arg, verbose=False):
|
||
... if verbose:
|
||
... print("Strength in numbers, eh?", end=" ")
|
||
... print(arg)
|
||
...
|
||
>>> @fun.register(list)
|
||
... def _(arg, verbose=False):
|
||
... if verbose:
|
||
... print("Enumerate this:")
|
||
... for i, elem in enumerate(arg):
|
||
... print(i, elem)
|
||
|
||
To enable registering lambdas and pre-existing functions, the
|
||
``register()`` attribute can be used in a functional form::
|
||
|
||
>>> def nothing(arg, verbose=False):
|
||
... print("Nothing.")
|
||
...
|
||
>>> fun.register(type(None), nothing)
|
||
|
||
When called, the function dispatches on the first argument::
|
||
|
||
>>> fun("Hello, world.")
|
||
Hello, world.
|
||
>>> fun("test.", verbose=True)
|
||
Let me just say, test.
|
||
>>> fun(42, verbose=True)
|
||
Strength in numbers, eh? 42
|
||
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
|
||
Enumerate this:
|
||
0 spam
|
||
1 spam
|
||
2 eggs
|
||
3 spam
|
||
>>> fun(None)
|
||
Nothing.
|
||
|
||
The proposed API is intentionally limited and opinionated, as to ensure
|
||
it is easy to explain and use, as well as to maintain consistency with
|
||
existing members in the ``functools`` module.
|
||
|
||
|
||
Implementation Notes
|
||
====================
|
||
|
||
The functionality described in this PEP is already implemented in the
|
||
``pkgutil`` standard library module as ``simplegeneric``. Because this
|
||
implementation is mature, the goal is to move it largely as-is. Several
|
||
open issues remain:
|
||
|
||
* the current implementation relies on ``__mro__`` alone, making it
|
||
incompatible with Abstract Base Classes'
|
||
``register()``/``unregister()`` functionality. A possible solution has
|
||
been proposed by PJE on the original issue for exposing
|
||
``pkgutil.simplegeneric`` as part of the ``functools`` API
|
||
[#issue-5135]_.
|
||
|
||
* the dispatch type is currently specified as a decorator argument. The
|
||
implementation could allow a form using argument annotations. This
|
||
usage pattern is out of scope for the standard library [#pep-0008]_.
|
||
However, whether this registration form would be acceptable for
|
||
general usage, is up to debate.
|
||
|
||
Based on the current ``pkgutil.simplegeneric`` implementation and
|
||
following the convention on registering virtual subclasses on Abstract
|
||
Base Classes, the dispatch registry will not be thread-safe.
|
||
|
||
|
||
Usage Patterns
|
||
==============
|
||
|
||
This PEP proposes extending behaviour only of functions specifically
|
||
marked as generic. Just as a base class method may be overridden by
|
||
a subclass, so too may a function be overloaded to provide custom
|
||
functionality for a given type.
|
||
|
||
Universal overloading does not equal *arbitrary* overloading, in the
|
||
sense that we need not expect people to randomly redefine the behavior
|
||
of existing functions in unpredictable ways. To the contrary, generic
|
||
function usage in actual programs tends to follow very predictable
|
||
patterns and overloads are highly-discoverable in the common case.
|
||
|
||
If a module is defining a new generic operation, it will usually also
|
||
define any required overloads for existing types in the same place.
|
||
Likewise, if a module is defining a new type, then it will usually
|
||
define overloads there for any generic functions that it knows or cares
|
||
about. As a result, the vast majority of overloads can be found adjacent
|
||
to either the function being overloaded, or to a newly-defined type for
|
||
which the overload is adding support.
|
||
|
||
It is only in rather infrequent cases that one will have overloads in
|
||
a module that contains neither the function nor the type(s) for which
|
||
the overload is added. In the absence of incompetence or deliberate
|
||
intention to be obscure, the few overloads that are not adjacent to the
|
||
relevant type(s) or function(s), will generally not need to be
|
||
understood or known about outside the scope where those overloads are
|
||
defined. (Except in the "support modules" case, where best practice
|
||
suggests naming them accordingly.)
|
||
|
||
As mentioned earlier, single-dispatch generics are already prolific
|
||
throughout the standard library. A clean, standard way of doing them
|
||
provides a way forward to refactor those custom implementations to use
|
||
a common one, opening them up for user extensibility at the same time.
|
||
|
||
|
||
Alternative approaches
|
||
======================
|
||
|
||
In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution
|
||
with overloading based on arbitrary rule sets (with the default
|
||
implementation dispatching on argument types), as well as interfaces,
|
||
adaptation and method combining. PEAK-Rules [#peak-rules]_ is
|
||
a reference implementation of the concepts described in PJE's PEP.
|
||
|
||
Such a broad approach is inherently complex, which makes reaching
|
||
a consensus hard. In contrast, this PEP focuses on a single piece of
|
||
functionality that is simple to reason about. It's important to note
|
||
this does not preclude the use of other approaches now or in the future.
|
||
|
||
In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents
|
||
a generic function implementation that dispatches on types of all
|
||
arguments on a function. The same approach was chosen in Andrey Popp's
|
||
``generic`` package available on PyPI [#pypi-generic]_, as well as David
|
||
Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_.
|
||
|
||
While this seems desirable at first, I agree with Fredrik Lundh's
|
||
comment that "if you design APIs with pages of logic just to sort out
|
||
what code a function should execute, you should probably hand over the
|
||
API design to someone else". In other words, the single argument
|
||
approach proposed in this PEP is not only easier to implement but also
|
||
clearly communicates that dispatching on a more complex state is an
|
||
anti-pattern. It also has the virtue of corresponding directly with the
|
||
familiar method dispatch mechanism in object oriented programming. The
|
||
only difference is whether the custom implementation is associated more
|
||
closely with the data (object-oriented methods) or the algorithm
|
||
(single-dispatch overloading).
|
||
|
||
PyPy's RPython offers ``extendabletype`` [#pairtype]_, a metaclass which
|
||
enables classes to be externally extended. In combination with
|
||
``pairtype()`` and ``pair()`` factories, this offers a form of
|
||
single-dispatch generics.
|
||
|
||
|
||
Acknowledgements
|
||
================
|
||
|
||
Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and
|
||
PEAK-Rules, influences include Paul Moore's original issue
|
||
[#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part
|
||
of the ``functools`` API, Guido van Rossum's article on multimethods
|
||
[#artima2005]_, and discussions with Raymond Hettinger on a general
|
||
pprint rewrite. Huge thanks to Nick Coghlan for encouraging me to create
|
||
this PEP and providing initial feedback.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [#issue-5135] http://bugs.python.org/issue5135
|
||
|
||
.. [#pep-0008] PEP 8 states in the "Programming Recommendations"
|
||
section that "the Python standard library will not use function
|
||
annotations as that would result in a premature commitment to
|
||
a particular annotation style".
|
||
(http://www.python.org/dev/peps/pep-0008)
|
||
|
||
.. [#pep-3124] http://www.python.org/dev/peps/pep-3124/
|
||
|
||
.. [#peak-rules] http://peak.telecommunity.com/DevCenter/PEAK_2dRules
|
||
|
||
.. [#artima2005]
|
||
http://www.artima.com/weblogs/viewpost.jsp?thread=101605
|
||
|
||
.. [#pypi-generic] http://pypi.python.org/pypi/generic
|
||
|
||
.. [#gnosis-multimethods]
|
||
http://gnosis.cx/publish/programming/charming_python_b12.html
|
||
|
||
.. [#pairtype]
|
||
https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|
||
|
||
|