PEP 443 - Single-dispatch generic functions. Initial draft
This commit is contained in:
Łukasz Langa
parent
51d8e64d24
commit
593595891d
|
|
@ -0,0 +1,276 @@
|
|||
PEP: 443
|
||||
Title: Single-dispatch generic functions
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Łukasz Langa <lukasz@langa.pl>
|
||||
Discussions-To: Python-Dev <python-dev@python.org>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 22-May-2013
|
||||
Post-History: 22-May-2013
|
||||
Replaces: 245, 246, 3124
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes a new mechanism in the ``functools`` standard library
|
||||
module that provides a simple form of generic programming known as
|
||||
single-dispatch generic functions.
|
||||
|
||||
A **generic function** is composed of multiple functions sharing the
|
||||
same name. Which form should be used during a call is determined by the
|
||||
dispatch algorithm. When the implementation is chosen based on the type
|
||||
of a single argument, this is known as **single dispatch**.
|
||||
|
||||
|
||||
Rationale and Goals
|
||||
===================
|
||||
|
||||
Python has always provided a variety of built-in and standard-library
|
||||
generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``,
|
||||
``copy.copy()``, and most of the functions in the ``operator`` module.
|
||||
However, it currently:
|
||||
|
||||
1. does not have a simple or straightforward way for developers to
|
||||
create new generic functions,
|
||||
|
||||
2. does not have a standard way for methods to be added to existing
|
||||
generic functions (i.e., some are added using registration
|
||||
functions, others require defining ``__special__`` methods, possibly
|
||||
by monkeypatching).
|
||||
|
||||
In addition, it is currently a common anti-pattern for Python code to
|
||||
inspect the types of received arguments, in order to decide what to do
|
||||
with the objects. For example, code may wish to accept either an object
|
||||
of some type, or a sequence of objects of that type.
|
||||
|
||||
Currently, the "obvious way" to do this is by type inspection, but this
|
||||
is brittle and closed to extension. Abstract Base Classes make it easier
|
||||
to discover present behaviour, but don't help adding new behaviour.
|
||||
A developer using an already-written library may be unable to change how
|
||||
their objects are treated by such code, especially if the objects they
|
||||
are using were created by a third party.
|
||||
|
||||
Therefore, this PEP proposes a uniform API to address dynamic
|
||||
overloading using decorators.
|
||||
|
||||
|
||||
User API
|
||||
========
|
||||
|
||||
To define a generic function, decorate it with the ``@singledispatch``
|
||||
decorator. Note that the dispatch happens on the type of the first
|
||||
argument, create your function accordingly:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> from functools import singledispatch
|
||||
>>> @singledispatch
|
||||
... def fun(arg, verbose=False):
|
||||
... if verbose:
|
||||
... print("Let me just say,", end=" ")
|
||||
... print(arg)
|
||||
|
||||
To add overloaded implementations to the function, use the
|
||||
``register()`` attribute of the generic function. It takes a type
|
||||
parameter:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> @fun.register(int)
|
||||
... def _(arg, verbose=False):
|
||||
... if verbose:
|
||||
... print("Strength in numbers, eh?", end=" ")
|
||||
... print(arg)
|
||||
...
|
||||
>>> @fun.register(list)
|
||||
... def _(arg, verbose=False):
|
||||
... if verbose:
|
||||
... print("Enumerate this:")
|
||||
... for i, elem in enumerate(arg):
|
||||
... print(i, elem)
|
||||
|
||||
To enable registering lambdas and pre-existing functions, the
|
||||
``register()`` attribute can be used in a functional form:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> def nothing(arg, verbose=False):
|
||||
... print("Nothing.")
|
||||
...
|
||||
>>> fun.register(type(None), nothing)
|
||||
|
||||
When called, the function dispatches on the first argument:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> fun("Hello, world.")
|
||||
Hello, world.
|
||||
>>> fun("test.", verbose=True)
|
||||
Let me just say, test.
|
||||
>>> fun(42, verbose=True)
|
||||
Strength in numbers, eh? 42
|
||||
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
|
||||
Enumerate this:
|
||||
0 spam
|
||||
1 spam
|
||||
2 eggs
|
||||
3 spam
|
||||
>>> fun(None)
|
||||
Nothing.
|
||||
|
||||
The proposed API is intentionally limited and opinionated, as to ensure
|
||||
it is easy to explain and use, as well as to maintain consistency with
|
||||
existing members in the ``functools`` module.
|
||||
|
||||
|
||||
Implementation Notes
|
||||
====================
|
||||
|
||||
The functionality described in this PEP is already implemented in the
|
||||
``pkgutil`` standard library module as ``simplegeneric``. Because this
|
||||
implementation is mature, the goal is to move it largely as-is. Several
|
||||
open issues remain:
|
||||
|
||||
* the current implementation relies on ``__mro__`` alone, making it
|
||||
incompatible with Abstract Base Classes'
|
||||
``register()``/``unregister()`` functionality. A possible solution has
|
||||
been proposed by PJE on the original issue for exposing
|
||||
``pkgutil.simplegeneric`` as part of the ``functools`` API
|
||||
[#issue-5135]_.
|
||||
|
||||
* the dispatch type is currently specified as a decorator argument. The
|
||||
implementation could allow a form using argument annotations. This
|
||||
usage pattern is out of scope for the standard library [#pep-0008]_.
|
||||
However, whether this registration form would be acceptable for
|
||||
general usage, is up to debate.
|
||||
|
||||
Based on the current ``pkgutil.simplegeneric`` implementation and
|
||||
following the convention on registering virtual subclasses on Abstract
|
||||
Base Classes, the dispatch registry will not be thread-safe.
|
||||
|
||||
|
||||
Usage Patterns
|
||||
==============
|
||||
|
||||
This PEP proposes extending behaviour only of functions specifically
|
||||
marked as generic. Just as a base class method may be overridden by
|
||||
a subclass, so too may a function be overloaded to provide custom
|
||||
functionality for a given type.
|
||||
|
||||
Universal overloading does not equal *arbitrary* overloading, in the
|
||||
sense that we need not expect people to randomly redefine the behavior
|
||||
of existing functions in unpredictable ways. To the contrary, generic
|
||||
function usage in actual programs tends to follow very predictable
|
||||
patterns and overloads are highly-discoverable in the common case.
|
||||
|
||||
If a module is defining a new generic operation, it will usually also
|
||||
define any required overloads for existing types in the same place.
|
||||
Likewise, if a module is defining a new type, then it will usually
|
||||
define overloads there for any generic functions that it knows or cares
|
||||
about. As a result, the vast majority of overloads can be found adjacent
|
||||
to either the function being overloaded, or to a newly-defined type for
|
||||
which the overload is adding support.
|
||||
|
||||
It is only in rather infrequent cases that one will have overloads in
|
||||
a module that contains neither the function nor the type(s) for which
|
||||
the overload is added. In the absence of incompetence or deliberate
|
||||
intention to be obscure, the few overloads that are not adjacent to the
|
||||
relevant type(s) or function(s), will generally not need to be
|
||||
understood or known about outside the scope where those overloads are
|
||||
defined. (Except in the "support modules" case, where best practice
|
||||
suggests naming them accordingly.)
|
||||
|
||||
As mentioned earlier, single-dispatch generics are already prolific
|
||||
throughout the standard library. A clean, standard way of doing them
|
||||
provides a way forward to refactor those custom implementations to use
|
||||
a common one, opening them up for user extensibility at the same time.
|
||||
|
||||
|
||||
Alternative approaches
|
||||
======================
|
||||
|
||||
In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution
|
||||
with overloading based on arbitrary rule sets (with the default
|
||||
implementation dispatching on argument types), as well as interfaces,
|
||||
adaptation and method combining. PEAK-Rules [#peak-rules]_ is
|
||||
a reference implementation of the concepts described in PJE's PEP.
|
||||
|
||||
Such a broad approach is inherently complex, which makes reaching
|
||||
a consensus hard. In contrast, this PEP focuses on a single piece of
|
||||
functionality that is simple to reason about. It's important to note
|
||||
this does not preclude the use of other approaches now or in the future.
|
||||
|
||||
In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents
|
||||
a generic function implementation that dispatches on types of all
|
||||
arguments on a function. The same approach was chosen in Andrey Popp's
|
||||
``generic`` package available on PyPI [#pypi-generic]_, as well as David
|
||||
Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_.
|
||||
|
||||
While this seems desirable at first, I agree with Fredrik Lundh's
|
||||
comment that "if you design APIs with pages of logic just to sort out
|
||||
what code a function should execute, you should probably hand over the
|
||||
API design to someone else". In other words, the single argument
|
||||
approach proposed in this PEP is not only easier to implement but also
|
||||
clearly communicates that dispatching on a more complex state is an
|
||||
anti-pattern. It also has the virtue of corresponding directly with the
|
||||
familiar method dispatch mechanism in object oriented programming. The
|
||||
only difference is whether the custom implementation is associated more
|
||||
closely with the data (object-oriented methods) or the algorithm
|
||||
(single-dispatch overloading).
|
||||
|
||||
|
||||
Acknowledgements
|
||||
================
|
||||
|
||||
Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and
|
||||
PEAK-Rules, influences include Paul Moore's original issue
|
||||
[#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part
|
||||
of the ``functools`` API, Guido van Rossum's article on multimethods
|
||||
[#artima2005]_, and discussions with Raymond Hettinger on a general
|
||||
pprint rewrite.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [#issue-5135] http://bugs.python.org/issue5135
|
||||
|
||||
.. [#pep-0008] PEP 8 states in the "Programming Recommendations"
|
||||
section that "the Python standard library will not use function
|
||||
annotations as that would result in a premature commitment to
|
||||
a particular annotation style".
|
||||
(http://www.python.org/dev/peps/pep-0008)
|
||||
|
||||
.. [#pep-3124] http://www.python.org/dev/peps/pep-3124/
|
||||
|
||||
.. [#peak-rules] http://peak.telecommunity.com/DevCenter/PEAK_2dRules
|
||||
|
||||
.. [#artima2005]
|
||||
http://www.artima.com/weblogs/viewpost.jsp?thread=101605
|
||||
|
||||
.. [#pypi-generic] http://pypi.python.org/pypi/generic
|
||||
|
||||
.. [#gnosis-multimethods]
|
||||
http://gnosis.cx/publish/programming/charming_python_b12.html
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
|
||||
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
||||
|
||||
|
||||
Loading…
Reference in New Issue