Revised PEP 422, now with the namespace setting
This revealed an interesting conflict between the use of __prepare__ to share a namespace between two different class instances, and the assumption in the cache invalidation for type attribute lookup thatthe class fully controls the contents of the underlying namespace.
This commit is contained in:
Nick Coghlan
parent
41ca0b81e0
commit
043d7cfaaa
293
pep-0422.txt
293
pep-0422.txt
|
|
@ -1,5 +1,5 @@
|
|||
PEP: 422
|
||||
Title: Simple class initialisation hook
|
||||
Title: Simpler customisation of class creation
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>,
|
||||
|
|
@ -9,30 +9,34 @@ Type: Standards Track
|
|||
Content-Type: text/x-rst
|
||||
Created: 5-Jun-2012
|
||||
Python-Version: 3.4
|
||||
Post-History: 5-Jun-2012, 10-Feb-2012
|
||||
Post-History: 5-Jun-2012, 10-Feb-2013
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
In Python 2, the body of a class definition could modify the way a class
|
||||
was created (or simply arrange to run other code after the class was created)
|
||||
by setting the ``__metaclass__`` attribute in the class body. While doing
|
||||
this implicitly from called code required the use of an implementation detail
|
||||
(specifically, ``sys._getframes()``), it could also be done explicitly in a
|
||||
fully supported fashion (for example, by passing ``locals()`` to a
|
||||
function that calculated a suitable ``__metaclass__`` value)
|
||||
Currently, customising class creation requires the use of a custom metaclass.
|
||||
This custom metaclass then persists for the entire lifecycle of the class,
|
||||
creating the potential for spurious metaclass conflicts.
|
||||
|
||||
There is currently no corresponding mechanism in Python 3 that allows the
|
||||
code executed in the class body to directly influence how the class object
|
||||
is created. Instead, the class creation process is fully defined by the
|
||||
class header, before the class body even begins executing.
|
||||
This PEP proposes to instead support a wide range of customisation
|
||||
scenarios through a new ``namespace`` parameter in the class header, and
|
||||
a new ``__init_class__`` hook in the class body.
|
||||
|
||||
This PEP proposes a mechanism that will once again allow the body of a
|
||||
class definition to more directly influence the way a class is created
|
||||
(albeit in a more constrained fashion), as well as replacing some current
|
||||
uses of metaclasses with a simpler, easier to understand alternative.
|
||||
The new mechanism is also much easier to understand and use than
|
||||
implementing a custom metaclass, and thus should provide a gentler
|
||||
introduction to the full power Python's metaclass machinery.
|
||||
|
||||
.. note::
|
||||
|
||||
This PEP, in particular the use of __prepare__ to share a single
|
||||
namespace amongst multiple class objects, highlights a possible issue
|
||||
with the attribute lookup caching: when the underlying mapping is updated
|
||||
by other means, the attribute lookup cache is not invalidated correctly.
|
||||
Since the optimisation provided by that cache is highly desirable,
|
||||
some of the ideas in this PEP may need to be declared as officially
|
||||
unsupported (since the observed behaviour is rather odd when the
|
||||
caches get out of sync).
|
||||
|
||||
Background
|
||||
==========
|
||||
|
|
@ -81,25 +85,32 @@ metaclass could not call methods that referenced the class by name (as the
|
|||
name had not yet been bound in the containing scope), similarly, Python 3
|
||||
metaclasses cannot call methods that rely on the implicit ``__class__``
|
||||
reference (as it is not populated until after the metaclass has returned
|
||||
control to the class creation machiner).
|
||||
control to the class creation machinery).
|
||||
|
||||
Finally, when a class uses a custom metaclass, it can pose additional
|
||||
challenges to the use of multiple inheritance, as a new class cannot
|
||||
inherit from parent classes with unrelated metaclasses. This means that
|
||||
it is impossible to add a metaclass to an already published class: such
|
||||
an addition is a backwards incompatible change due to the risk of metaclass
|
||||
conflicts.
|
||||
|
||||
|
||||
Proposal
|
||||
========
|
||||
|
||||
This PEP proposes that a mechanism be added to Python 3 that meets the
|
||||
following criteria:
|
||||
This PEP proposes that a new mechanism to customise class creation be
|
||||
added to Python 3.4 that meets the following criteria:
|
||||
|
||||
1. Restores the ability for class namespaces to have some influence on the
|
||||
1. Integrates nicely with class inheritance structures (including mixins and
|
||||
multiple inheritance)
|
||||
2. Integrates nicely with the implicit ``__class__`` reference and
|
||||
zero-argument ``super()`` syntax introduced by PEP 3135
|
||||
3. Can be added to an existing base class without a significant risk of
|
||||
introducing backwards compatibility problems
|
||||
4. Restores the ability for class namespaces to have some influence on the
|
||||
class creation process (above and beyond populating the namespace itself),
|
||||
but potentially without the full flexibility of the Python 2 style
|
||||
``__metaclass__`` hook
|
||||
2. Integrates nicely with class inheritance structures (including mixins and
|
||||
multiple inheritance)
|
||||
3. Integrates nicely with the implicit ``__class__`` reference and
|
||||
zero-argument ``super()`` syntax introduced by PEP 3135
|
||||
4. Can be added to an existing base class without a significant risk of
|
||||
introducing backwards compatibility problems
|
||||
|
||||
One mechanism that can achieve this goal is to add a new class
|
||||
initialisation hook, modelled directly on the existing instance
|
||||
|
|
@ -110,7 +121,6 @@ Specifically, it is proposed that class definitions be able to provide a
|
|||
class initialisation hook as follows::
|
||||
|
||||
class Example:
|
||||
@classmethod
|
||||
def __init_class__(cls):
|
||||
# This is invoked after the class is created, but before any
|
||||
# explicit decorators are called
|
||||
|
|
@ -121,13 +131,15 @@ class initialisation hook as follows::
|
|||
If present on the created object, this new hook will be called by the class
|
||||
creation machinery *after* the ``__class__`` reference has been initialised.
|
||||
For ``types.new_class()``, it will be called as the last step before
|
||||
returning the created class object.
|
||||
returning the created class object. ``__init_class__`` is implicitly
|
||||
converted to a class method when the class is created (prior to the hook
|
||||
being invoked).
|
||||
|
||||
If a metaclass wishes to block class initialisation for some reason, it
|
||||
must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
|
||||
|
||||
Note, that when ``__init_class__`` is called, the name of the class is not
|
||||
bound to the new class object yet. As a consequence, the two argument form
|
||||
yet bound to the new class object. As a consequence, the two argument form
|
||||
of ``super()`` cannot be used to call methods (e.g., ``super(Example, cls)``
|
||||
wouldn't work in the example above). However, the zero argument form of
|
||||
``super()`` works as expected, since the ``__class__`` reference is already
|
||||
|
|
@ -139,19 +151,31 @@ similar mechanism has long been supported by `Zope's ExtensionClass`_),
|
|||
but the situation has changed sufficiently in recent years that
|
||||
the idea is worth reconsidering.
|
||||
|
||||
However, the introduction of the metaclass ``__prepare__`` method in PEP
|
||||
3115 allows a further enhancement that was not possible in Python 2: this
|
||||
PEP also proposes that ``type.__prepare__`` be updated to accept a
|
||||
``namespace`` keyword-only argument. If present, the value provided as the
|
||||
``namespace`` argument will be returned from ``type.__prepare__`` instead of
|
||||
a freshly created dictionary instance. For example, the following will use
|
||||
the ordered dictionary created in the header as the class namespace::
|
||||
|
||||
class OrderedExample(namespace=collections.OrderedDict()):
|
||||
def __init_class__(cls):
|
||||
# cls.__dict__ is still a read-only proxy to the class namespace,
|
||||
# but the underlying storage is the OrderedDict instance
|
||||
|
||||
|
||||
Key Benefits
|
||||
============
|
||||
|
||||
|
||||
Replaces many use cases for dynamic setting of ``__metaclass__``
|
||||
-----------------------------------------------------------------
|
||||
Easier use of custom namespaces for a class
|
||||
-------------------------------------------
|
||||
|
||||
For use cases that don't involve completely replacing the defined class,
|
||||
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
|
||||
set ``__init_class__`` instead. For more advanced use cases, introduction of
|
||||
an explicit metaclass (possibly made available as a required base class) will
|
||||
still be necessary in order to support Python 3.
|
||||
Currently, to use a different type (such as ``collections.OrderedDict``) for
|
||||
a class namespace, or to use a pre-populated namespace, it is necessary to
|
||||
write and use a custom metaclass. With this PEP, using a custom namespace
|
||||
becomes as simple as specifying it in the class header.
|
||||
|
||||
|
||||
Easier inheritance of definition time behaviour
|
||||
|
|
@ -201,137 +225,89 @@ implicit ``__class__`` reference introduced by PEP 3135, including methods
|
|||
that use the zero argument form of ``super()``.
|
||||
|
||||
|
||||
Alternatives
|
||||
============
|
||||
Replaces many use cases for dynamic setting of ``__metaclass__``
|
||||
-----------------------------------------------------------------
|
||||
|
||||
For use cases that don't involve completely replacing the defined class,
|
||||
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
|
||||
set ``__init_class__`` instead. For more advanced use cases, introduction of
|
||||
an explicit metaclass (possibly made available as a required base class) will
|
||||
still be necessary in order to support Python 3.
|
||||
|
||||
|
||||
The Python 3 Status Quo
|
||||
-----------------------
|
||||
New Ways of Using Classes
|
||||
=========================
|
||||
|
||||
The Python 3 status quo already offers a great deal of flexibility. For
|
||||
changes which only affect a single class definition and which can be
|
||||
specified at the time the code is written, then class decorators can be
|
||||
used to modify a class explicitly. Class decorators largely ignore class
|
||||
inheritance and can make full use of methods that rely on the ``__class__``
|
||||
reference being populated.
|
||||
The new ``namespace`` keyword in the class header enables a number of
|
||||
interesting options for controlling the way a class is initialised,
|
||||
including some aspects of the object models of both Javascript and Ruby.
|
||||
|
||||
Using a custom metaclass provides the same level of power as it did in
|
||||
Python 2. However, it's notable that, unlike class decorators, a metaclass
|
||||
cannot call any methods that rely on the ``__class__`` reference, as that
|
||||
reference is not populated until after the metaclass constructor returns
|
||||
control to the class creation code.
|
||||
All of the examples below are actually possible today through the use of a
|
||||
custom metaclass::
|
||||
|
||||
One major use case for metaclasses actually closely resembles the use of
|
||||
class decorators. It occurs whenever a metaclass has an implementation that
|
||||
uses the following pattern::
|
||||
class CustomNamespace(type):
|
||||
@classmethod
|
||||
def __prepare__(meta, name, bases, *, namespace=None, **kwds):
|
||||
parent_namespace = super().__prepare__(name, bases, **kwds)
|
||||
return namespace if namespace is not None else parent_namespace
|
||||
|
||||
class Metaclass(type):
|
||||
def __new__(meta, *args, **kwds):
|
||||
cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
|
||||
# Do something with cls
|
||||
return cls
|
||||
def __new__(meta, name, bases, ns, *, namespace=None, **kwds):
|
||||
return super().__new__(meta, name, bases, ns, **kwds)
|
||||
|
||||
The key difference between this pattern and a class decorator is that it
|
||||
is automatically inherited by subclasses. However, it also comes with a
|
||||
major disadvantage: Python does not allow you to inherit from classes with
|
||||
unrelated metaclasses.
|
||||
def __init__(cls, name, bases, ns, *, namespace=None, **kwds):
|
||||
return super().__init__(name, bases, ns, **kwds)
|
||||
|
||||
Thus, the status quo requires that developers choose between the following
|
||||
two alternatives:
|
||||
|
||||
* Use a class decorator, meaning that behaviour is not inherited and must be
|
||||
requested explicitly on every subclass
|
||||
* Use a metaclass, meaning that behaviour is inherited, but metaclass
|
||||
conflicts may make integration with other libraries and frameworks more
|
||||
difficult than it otherwise would be
|
||||
|
||||
If this PEP is ultimately rejected, then this is the existing design that
|
||||
will remain in place by default.
|
||||
The advantage of implementing the new keyword directly in
|
||||
``type.__prepare__`` is that the *only* persistent effect is
|
||||
the change in the underlying storage of the class attributes. The metaclass
|
||||
of the class remains unchanged, eliminating many of the drawbacks
|
||||
typically associated with these kinds of customisations.
|
||||
|
||||
|
||||
Restoring the Python 2 metaclass hook
|
||||
-------------------------------------
|
||||
|
||||
One simple alternative would be to restore support for a Python 2 style
|
||||
``metaclass`` hook in the class body. This would be checked after the class
|
||||
body was executed, potentially overwriting the metaclass hint provided in the
|
||||
class header.
|
||||
|
||||
The main attraction of such an approach is that it would simplify porting
|
||||
Python 2 applications that make use of this hook (especially those that do
|
||||
so dynamically).
|
||||
|
||||
However, this approach does nothing to simplify the process of adding
|
||||
*inherited* class definition time behaviour, nor does it interoperate
|
||||
cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
|
||||
any metaclass based solution, the ``__metaclass__`` hook would have to run
|
||||
before the ``__class__`` reference has been populated.
|
||||
|
||||
|
||||
Dynamic class decorators
|
||||
Order preserving classes
|
||||
------------------------
|
||||
|
||||
The original version of this PEP was called "Dynamic class decorators" and
|
||||
focused solely on a significantly more complicated proposal than that
|
||||
presented in the current version.
|
||||
::
|
||||
class OrderedClass(namespace=collections.OrderedDict()):
|
||||
a = 1
|
||||
b = 2
|
||||
c = 3
|
||||
|
||||
As with the current version, it proposed that a new step be added to the
|
||||
class creation process, after the metaclass invocation to construct the
|
||||
class instance and before the application of lexical decorators. However,
|
||||
instead of a simple process of calling a single class method that relies
|
||||
on normal inheritance mechanisms, it proposed a far more complicated
|
||||
procedure that walked the class MRO looking for decorators stored in
|
||||
iterable ``__decorators__`` attributes.
|
||||
|
||||
Using the current version of the PEP, the scheme originally proposed could
|
||||
be implemented as::
|
||||
Prepopulated namespaces
|
||||
-----------------------
|
||||
|
||||
class DynamicDecorators(Base):
|
||||
@classmethod
|
||||
def __init_class__(cls):
|
||||
# Process any classes later in the MRO
|
||||
try:
|
||||
mro_chain = super().__init_class__
|
||||
except AttributeError:
|
||||
::
|
||||
seed_data = dict(a=1, b=2, c=3)
|
||||
class PrepopulatedClass(namespace=seed_data.copy()):
|
||||
pass
|
||||
else:
|
||||
mro_chain()
|
||||
# Process any __decorators__ attributes in the MRO
|
||||
for entry in reversed(cls.mro()):
|
||||
decorators = entry.__dict__.get("__decorators__", ())
|
||||
for deco in reversed(decorators):
|
||||
cls = deco(cls)
|
||||
|
||||
Any subclasses of ``DynamicDecorators`` would then automatically have the
|
||||
contents of any ``__decorators__`` attributes processed and invoked.
|
||||
|
||||
The mechanism in the current PEP is considered superior, as many issues
|
||||
to do with ordering and the same decorator being invoked multiple times
|
||||
just go away, as that kind of thing is taken care of through the use of an
|
||||
ordinary class method invocation.
|
||||
|
||||
|
||||
Automatic metaclass derivation
|
||||
------------------------------
|
||||
Cloning a prototype class
|
||||
-------------------------
|
||||
|
||||
When no appropriate metaclass is found, it's theoretically possible to
|
||||
automatically derive a metaclass for a new type based on the metaclass hint
|
||||
and the metaclasses of the bases.
|
||||
|
||||
While adding such a mechanism would reduce the risk of spurious metaclass
|
||||
conflicts, it would do nothing to improve integration with PEP 3135, would
|
||||
not help with porting Python 2 code that set ``__metaclass__`` dynamically
|
||||
and would not provide a more straightforward inherited mechanism for invoking
|
||||
additional operations after the class invocation is complete.
|
||||
|
||||
In addition, there would still be a risk of metaclass conflicts in cases
|
||||
where the base metaclasses were not written with multiple inheritance in
|
||||
mind. In such situations, there's a chance of introducing latent defects
|
||||
if one or more metaclasses are not invoked correctly.
|
||||
::
|
||||
class NewClass(namespace=Prototype.__dict__.copy()):
|
||||
pass
|
||||
|
||||
|
||||
Calling the new hook from ``type.__init__``
|
||||
-------------------------------------------
|
||||
Defining an extensible class
|
||||
----------------------------
|
||||
|
||||
::
|
||||
class Extensible:
|
||||
namespace = locals()
|
||||
|
||||
class ExtendingClass(namespace=Extensible.namespace):
|
||||
pass
|
||||
|
||||
|
||||
Rejected Design Options
|
||||
=======================
|
||||
|
||||
|
||||
Calling ``__init_class__`` from ``type.__init__``
|
||||
-------------------------------------------------
|
||||
|
||||
Calling the new hook automatically from ``type.__init__``, would achieve most
|
||||
of the goals of this PEP. However, using that approach would mean that
|
||||
|
|
@ -340,11 +316,32 @@ relied on the ``__class__`` reference (or used the zero-argument form of
|
|||
``super()``), and could not make use of those features themselves.
|
||||
|
||||
|
||||
Requiring an explict decorator on ``__init_class__``
|
||||
----------------------------------------------------
|
||||
|
||||
Originally, this PEP required the explicit use of ``@classmethod`` on the
|
||||
``__init_class__`` decorator. It was made implicit since there's no
|
||||
sensible interpretation for leaving it out, and that case would need to be
|
||||
detected anyway in order to give a useful error message.
|
||||
|
||||
This decision was reinforced after noticing that the user experience of
|
||||
defining ``__prepare__`` and forgetting the ``@classmethod`` method
|
||||
decorator is singularly incomprehensible (particularly since PEP 3115
|
||||
documents it as an ordinary method, and the current documentation doesn't
|
||||
explicitly say anything one way or the other).
|
||||
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
The reference implementation has been posted to the `issue tracker`_.
|
||||
A reference implementation for __init_class__ has been posted to the
|
||||
`issue tracker`_. It does not yet include the new ``namespace`` parameter
|
||||
for ``type.__prepare__``.
|
||||
|
||||
TODO
|
||||
====
|
||||
|
||||
* address the 5 points in http://mail.python.org/pipermail/python-dev/2013-February/123970.html
|
||||
|
||||
References
|
||||
==========
|
||||
|
|
|
|||
Loading…
Reference in New Issue