PEP 422 rewrite to present an idea that a) isn't crazy and b) it turns out Thomas Heller proposed back in 2001
This commit is contained in:
Nick Coghlan
parent
e8131ca8b6
commit
7bac3da1c4
345
pep-0422.txt
345
pep-0422.txt
|
|
@ -1,5 +1,5 @@
|
|||
PEP: 422
|
||||
Title: Dynamic class decorators
|
||||
Title: Simple class initialisation hook
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||||
|
|
@ -7,148 +7,287 @@ Status: Draft
|
|||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 5-Jun-2012
|
||||
Python-Version: 3.4
|
||||
Post-History: 5-Jun-2012
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
Classes currently support two mechanisms for modification of the class at
|
||||
definition time: metaclasses and lexical decorators.
|
||||
In Python 2, the body of a class definition could modify the way a class
|
||||
was created (or simply arrange to run other code after the class was created)
|
||||
by setting the ``__metaclass__`` attribute in the class body. While doing
|
||||
this implicitly from called code required the use of an implementation detail
|
||||
(specifically, ``sys._getframes()``), it could also be done explicitly in a
|
||||
fully supported fashion (for example, by passing ``locals()`` to an
|
||||
function that calculated a suitable ``__metaclass__`` value)
|
||||
|
||||
Metaclasses can be awkward and challenging to use correctly in conjunction
|
||||
with multiple inheritance and lexical decorators don't interact with class
|
||||
inheritance at all.
|
||||
There is currently no corresponding mechanism in Python 3 that allows the
|
||||
code executed in the class body to directly influence how the class object
|
||||
is created. Instead, the class creation process is fully defined by the
|
||||
class header, before the class body even begins executing.
|
||||
|
||||
This PEP proposes a new mechanism for dynamic class decoration that
|
||||
interacts more cleanly with class inheritance mechanisms.
|
||||
This PEP proposes a mechanism that will once again allow the body of a
|
||||
class definition to more directly influence the way a class is created
|
||||
(albeit in a more constrained fashion), as well as replacing some current
|
||||
uses of metaclasses with a simpler, easier to understand alternative.
|
||||
|
||||
|
||||
Specification
|
||||
=============
|
||||
Background
|
||||
==========
|
||||
|
||||
This PEP proposes that a new step be added to the class creation process,
|
||||
after the metaclass invocation to construct the class instance and before
|
||||
the application of lexical decorators.
|
||||
For an already created class ``cls``, the term "metaclass" has a clear
|
||||
meaning: it is the value of ``type(cls)``.
|
||||
|
||||
This step will walk the class MRO in reverse order, looking for
|
||||
``__decorators__`` entries in each class dictionary. These entries are
|
||||
expected to be iterables that are also walked in reverse order to retrieve
|
||||
class decorators that are automatically applied to the class being defined::
|
||||
*During* class creation, it has another meaning: it is also used to refer to
|
||||
the metaclass hint that may be provided as part of the class definition.
|
||||
While in many cases these two meanings end up referring to one and the same
|
||||
object, there are two situations where that is not the case:
|
||||
|
||||
for entry in reversed(cls.mro()):
|
||||
decorators = entry.__dict__.get("__decorators__", ())
|
||||
for deco in reversed(decorators):
|
||||
cls = deco(cls)
|
||||
* If the metaclass hint refers to an instance of ``type``, then it is
|
||||
considered as a candidate metaclass along with the metaclasses of all of
|
||||
the parents of the class being defined. If a more appropriate metaclass is
|
||||
found amongst the candidates, then it will be used instead of the one
|
||||
given in the metaclass hint.
|
||||
* Otherwise, an explicit metaclass hint is assumed to be a factory function
|
||||
and is called directly to create the class object. In this case, the final
|
||||
metaclass will be determined by the factory function definition. In the
|
||||
typical case (where the factory functions just calls ``type``, or, in
|
||||
Python 3.3 or later, ``types.new_class``) the actual metaclass is then
|
||||
determined based on the parent classes.
|
||||
|
||||
This step in the class creation process will be an implicit part of the
|
||||
class statement and also part of the behaviour of ``types.new_class()``.
|
||||
It is notable that only the actual metaclass is inherited - a factory
|
||||
function used as a metaclass hook sees only the class currently being
|
||||
defined, and is not invoked for any subclasses.
|
||||
|
||||
In Python 3, the metaclass hint is provided using the ``metaclass=Meta``
|
||||
keyword syntax in the class header. This allows the ``__prepare__`` method
|
||||
on the metaclass to be used to create the ``locals()`` namespace used during
|
||||
execution of the class body (for example, specifying the use of
|
||||
``collections.OrderedDict`` instead of a regular ``dict``).
|
||||
|
||||
In Python 2, there was no ``__prepare__`` method (that API was added for
|
||||
Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__``
|
||||
attribute, and the class creation process would extract that value from the
|
||||
class namespace to use as the metaclass hint. There is `published code`_ that
|
||||
makes use of this feature.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
Proposal
|
||||
========
|
||||
|
||||
When decorator support was added to classes, the lexical decoration syntax
|
||||
was copied directly from function decorators::
|
||||
This PEP proposes that a mechanism be added to Python 3 that meets the
|
||||
following criteria:
|
||||
|
||||
@decorator
|
||||
class Example:
|
||||
# Subclasses will not be decorated automatically
|
||||
pass
|
||||
# Restores the ability for class namespaces to have some influence on the
|
||||
class creation process (above and beyond populating the namespace itself),
|
||||
but potentially without the full flexibility of the Python 2 style
|
||||
``__metaclass__`` hook
|
||||
# Integrates nicely with class inheritance structures (including mixins and
|
||||
multiple inheritance)
|
||||
# Integrates nicely with the implicit ``__class__`` reference and
|
||||
zero-argument ``super()`` syntax introduced by PEP 3135
|
||||
# Can be added to an existing base class without a significant risk of
|
||||
introducing backwards compatibility problems
|
||||
|
||||
This mechanism works well, so long as it is considered acceptable that the
|
||||
decorator is *not* applied automatically to any subclasses. If it is
|
||||
desired that the behaviour be inherited, it is currently necessary to
|
||||
make the step up to defining a `custom metaclass`_::
|
||||
One mechanism that would achieve this goal is to add a new class
|
||||
initialisation hook, modelled directly on the existing instance
|
||||
initialisation hook. However, the signature would be constrained to ensure
|
||||
that correctly supporting multiple inheritance is kept as simple as possible.
|
||||
|
||||
class DynamicDecorators(type):
|
||||
"""Metaclass for dynamic decorator support
|
||||
Specifically, it is proposed that class definitions be able to provide a
|
||||
class initialisation hook as follows::
|
||||
|
||||
Creates the class normally, then runs through the MRO looking for
|
||||
__decorators__ attributes and applying the contained decorators to
|
||||
the newly created class
|
||||
"""
|
||||
def __new__(meta, name, bases, ns):
|
||||
cls = super(DynamicDecorators, meta).__new__(meta, name, bases, ns)
|
||||
for entry in reversed(cls.mro()):
|
||||
decorators = entry.__dict__.get("__decorators__", ())
|
||||
for deco in reversed(decorators):
|
||||
cls = deco(cls)
|
||||
class Example:
|
||||
@classmethod
|
||||
def __init_class__(cls):
|
||||
# This is invoked after the class is created, but before any
|
||||
# explicit decorators are called
|
||||
# The usual super() mechanisms are used to correctly support
|
||||
# multiple inheritance. The simple, decorator style invocation
|
||||
# ensures that this is as simple as possible.
|
||||
|
||||
If present on the created object, this new hook will be called by the class
|
||||
creation machinery *after* the ``__class__`` reference has been initialised.
|
||||
For ``types.new_class()``, it will be called as the last step before
|
||||
returning the created class object. Calling the hook automatically from
|
||||
``type.__init__`` unfortunately doesn't work, as it would mean the
|
||||
``__init_class__`` method would be unable to call any methods that relied
|
||||
on the ``__class__`` reference (or used the zero-argument form of
|
||||
``super()``).
|
||||
|
||||
If a metaclass wishes to block class initialisation for some reason, it
|
||||
must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
|
||||
|
||||
This general proposal is not a new idea (it was first suggested `more than
|
||||
10 years ago`_), but I believe the situation has changed sufficiently in
|
||||
that time that the idea is worth reconsidering.
|
||||
|
||||
|
||||
Key Benefits
|
||||
============
|
||||
|
||||
|
||||
Replaces dynamic setting of ``__metaclass__``
|
||||
---------------------------------------------
|
||||
|
||||
For use cases that didn't involve completely replacing the defined class,
|
||||
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
|
||||
set ``__init_class__`` instead. For more advanced use cases, introduction of
|
||||
an explicit metaclass will still be necessary in order to support Python 3.
|
||||
|
||||
|
||||
Easier inheritance of definition time behaviour
|
||||
-----------------------------------------------
|
||||
|
||||
Understanding Python's metaclass system requires a deep understanding of
|
||||
the type system and the class construction process. This is legitimately
|
||||
seen as confusing, due to the need to keep multiple moving parts (the code,
|
||||
the metaclass hint, the actual metaclass, the class object, instances of the
|
||||
class object) clearly distinct in your mind.
|
||||
|
||||
Understanding the proposed class initialisation hook requires understanding
|
||||
decorators and ordinary method inheritance, which is a much simpler prospect.
|
||||
|
||||
|
||||
Reduced chance of metaclass conflicts
|
||||
-------------------------------------
|
||||
|
||||
One of the big issues that makes library authors reluctant to use metaclasses
|
||||
(even when it would be appropriate) is the risk of metaclass conflicts.
|
||||
These occur whenever two unrelated metaclasses are used by the desired
|
||||
parents of a class definition. This risk also makes it very difficult to
|
||||
*add* a metaclass to a class that has previously been published without one.
|
||||
|
||||
By contrast, adding an ``__init_class__`` method to an existing type poses
|
||||
a similar level of risk to adding an ``__init__`` method: technically, there
|
||||
is a risk of breaking poorly implemented subclasses, but when that occurs,
|
||||
it is recognised as a bug in the subclass rather than the library author
|
||||
breaching backwards compatibility guarantees. In fact, due to the constrained
|
||||
signature, the risk in this case is actually even lower than in the case of
|
||||
``__init__``.
|
||||
|
||||
|
||||
Integrates cleanly with PEP 3135
|
||||
--------------------------------
|
||||
|
||||
Unlike code that runs as part of the metaclass, code that runs as part of
|
||||
the new hook will be able to freely invoke class methods that rely on the
|
||||
implicit ``__class__`` reference introduced by PEP 3135, including methods
|
||||
that use the zero argument form of ``super()``.
|
||||
|
||||
|
||||
Alternatives
|
||||
============
|
||||
|
||||
|
||||
The Python 3 Status Quo
|
||||
-----------------------
|
||||
|
||||
The Python 3 status quo already offers a great deal of flexibility. For
|
||||
changes which only affect a single class definition and which can be
|
||||
specified at the time the code is written, then class decorators can be
|
||||
used to modify a class explicitly. Class decorators largely ignore class
|
||||
inheritance and can make full use of methods that rely on the ``__class__``
|
||||
reference being populated.
|
||||
|
||||
Using a custom metaclass provides the same level of power as it did in
|
||||
Python 2. However, it's notable that, unlike class decorators, a metaclass
|
||||
cannot call any methods that rely on the ``__class__`` reference, as that
|
||||
reference is not populated until after the metaclass constructor returns
|
||||
control to the class creation code.
|
||||
|
||||
One major use case for metaclasses actually closely resembles the use of
|
||||
class decorators. It occurs whenever a metaclass has an implementation that
|
||||
uses the following pattern::
|
||||
|
||||
class Metaclass(type):
|
||||
def __new__(meta, *args, **kwds):
|
||||
cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
|
||||
# Do something with cls
|
||||
return cls
|
||||
|
||||
class Example(metaclass=DynamicDecorators):
|
||||
# Subclasses *will* be decorated automatically
|
||||
__decorators__ = [decorator]
|
||||
The key difference between this pattern and a class decorator is that it
|
||||
is automatically inherited by subclasses. However, it also comes with a
|
||||
major disadvantage: Python does not allow you to inherit from classes with
|
||||
unrelated metaclasses.
|
||||
|
||||
The main potential problem with this approach, is that it can place
|
||||
significant constraints on the type heirarchy, as it requires that all
|
||||
metaclasses used be well behaved with respect to multiple inheritance.
|
||||
Thus, the status quo requires that developers choose between the following
|
||||
two alternatives:
|
||||
|
||||
By making dynamic decorators an inherent part of the class creation process,
|
||||
many current use cases of metaclasses may be replaced with dynamic decorators
|
||||
instead, greatly reducing the likelihood of metaclass conflicts, as well
|
||||
as being substantially easier to write correctly in the first place.
|
||||
* Use a class decorator, meaning that behaviour is not inherited and must be
|
||||
requested explicitly on every subclass
|
||||
* Use a metaclass, meaning that behaviour is inherited, but metaclass
|
||||
conflicts may make integration with other libraries and frameworks more
|
||||
difficult than it otherwise would be
|
||||
|
||||
If this PEP is ultimately rejected, then this is the existing design that
|
||||
will remain in place by default.
|
||||
|
||||
|
||||
Design Discussion
|
||||
=================
|
||||
Restoring the Python 2 metaclass hook
|
||||
-------------------------------------
|
||||
|
||||
One simple alternative would be to restore support for a Python 2 style
|
||||
``metaclass`` hook in the class body. This would be checked after the class
|
||||
body was executed, potentially overwriting the metaclass hint provided in the
|
||||
class header.
|
||||
|
||||
The main attraction of such an approach is that it would simplify porting
|
||||
Python 2 applications that make use of this hook (especially those that do
|
||||
so dynamically).
|
||||
|
||||
However, this approach does nothing to simplify the process of adding
|
||||
*inherited* class definition time behaviour, nor does it interoperate
|
||||
cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
|
||||
any metaclass based solution, the ``__metaclass__`` hook would have to run
|
||||
before the ``__class__`` reference has been populated.
|
||||
|
||||
|
||||
Allowing metaclasses to override the dynamic decoration process
|
||||
---------------------------------------------------------------
|
||||
Dynamic class decorators
|
||||
------------------------
|
||||
|
||||
This PEP does not provide a mechanism that allows metaclasses to override the
|
||||
dynamic decoration process. If this feature is deemed desirable in the
|
||||
future, then it can be added by moving the functionality described in
|
||||
this PEP into a new method on the metaclass (for example, ``__decorate__``),
|
||||
with ``type`` providing a suitable default implementation that matches
|
||||
the behaviour described here.
|
||||
The original version of this PEP was called "Dynamic class decorators" and
|
||||
focused solely on a significantly more complicated proposal than that
|
||||
presented in the current version.
|
||||
|
||||
This PEP chose the simplicity of the current approach, as lexical decorators
|
||||
are currently outside the scope of metaclass control, so it seems reasonable
|
||||
to pursue the simpler strategy in the absence of a solid use case for
|
||||
making this behaviour configurable.
|
||||
As with the current version, it proposed that a new step be added to the
|
||||
class creation process, after the metaclass invocation to construct the
|
||||
class instance and before the application of lexical decorators. However,
|
||||
instead of a simple process of calling a single class method that relies
|
||||
on normal inheritance mechanisms, it proposed a far more complicated
|
||||
procedure that walked the class MRO looking for decorators stored in
|
||||
iterable ``__decorators__`` attributes.
|
||||
|
||||
Using the current version of the PEP, the scheme originally proposed could
|
||||
be implemented as::
|
||||
|
||||
Iterating over decorator entries in reverse order
|
||||
-------------------------------------------------
|
||||
class DynamicDecorators:
|
||||
@classmethod
|
||||
def __init_class__(cls):
|
||||
super(DynamicDecorators, cls).__init_class__()
|
||||
for entry in reversed(cls.mro()):
|
||||
decorators = entry.__dict__.get("__decorators__", ())
|
||||
for deco in reversed(decorators):
|
||||
cls = deco(cls)
|
||||
|
||||
This order was chosen to match the layout of lexical decorators when
|
||||
converted to ordinary function calls. Just as the following are equivalent::
|
||||
Any subclasses of this type would automatically have the contents of any
|
||||
``__decorators__`` attributes processed and invoked.
|
||||
|
||||
@deco2
|
||||
@deco1
|
||||
class C:
|
||||
pass
|
||||
The mechanism in the current PEP is considered superior, as many issues
|
||||
to do with ordering and the same decorator being invoked multiple times
|
||||
simple go away, as that kind of thing is taken care of through the use of an
|
||||
ordinary class method invocation.
|
||||
|
||||
class C:
|
||||
pass
|
||||
C = deco2(deco1(C))
|
||||
|
||||
So too will the following be roughly equivalent (aside from inheritance)::
|
||||
|
||||
class C:
|
||||
__decorators__ = [deco2, deco1]
|
||||
|
||||
class C:
|
||||
pass
|
||||
C = deco2(deco1(C))
|
||||
|
||||
|
||||
Iterating over the MRO in reverse order
|
||||
---------------------------------------
|
||||
|
||||
The order of iteration over the MRO for decorator application was chosen to
|
||||
match the order of actual call *evaluation* when using ``super`` to invoke
|
||||
parent class implementations: the first method to run to completion is that
|
||||
closest to the base of the class hierarchy.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. _custom metaclass:
|
||||
https://bitbucket.org/ncoghlan/misc/src/default/pep422.py
|
||||
.. _published code:
|
||||
http://mail.python.org/pipermail/python-dev/2012-June/119878.html
|
||||
|
||||
.. _more than 10 years ago:
|
||||
http://mail.python.org/pipermail/python-dev/2001-November/018651.html
|
||||
|
||||
|
||||
Copyright
|
||||
|
|
|
|||
Loading…
Reference in New Issue