iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Revised PEP 422, now with the namespace setting 

This revealed an interesting conflict between the use of __prepare__ to share a namespace between two different class instances, and the assumption in the cache invalidation for type attribute lookup thatthe class fully controls the contents of the underlying namespace.
This commit is contained in:
Nick Coghlan 2013-03-05 01:54:19 +10:00
parent 41ca0b81e0
commit 043d7cfaaa
1 changed files with 149 additions and 152 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

301
pep-0422.txt
View File

@ -1,5 +1,5 @@
PEP: 422
Title: Simple class initialisation hook
Title: Simpler customisation of class creation
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>,
@ -9,30 +9,34 @@ Type: Standards Track
Content-Type: text/x-rst
Created: 5-Jun-2012
Python-Version: 3.4
Post-History: 5-Jun-2012, 10-Feb-2012
Post-History: 5-Jun-2012, 10-Feb-2013
Abstract
========
In Python 2, the body of a class definition could modify the way a class
was created (or simply arrange to run other code after the class was created)
by setting the ``__metaclass__`` attribute in the class body. While doing
this implicitly from called code required the use of an implementation detail
(specifically, ``sys._getframes()``), it could also be done explicitly in a
fully supported fashion (for example, by passing ``locals()`` to a
function that calculated a suitable ``__metaclass__`` value)
Currently, customising class creation requires the use of a custom metaclass.
This custom metaclass then persists for the entire lifecycle of the class,
creating the potential for spurious metaclass conflicts.
There is currently no corresponding mechanism in Python 3 that allows the
code executed in the class body to directly influence how the class object
is created. Instead, the class creation process is fully defined by the
class header, before the class body even begins executing.
This PEP proposes to instead support a wide range of customisation
scenarios through a new ``namespace`` parameter in the class header, and
a new ``__init_class__`` hook in the class body.
This PEP proposes a mechanism that will once again allow the body of a
class definition to more directly influence the way a class is created
(albeit in a more constrained fashion), as well as replacing some current
uses of metaclasses with a simpler, easier to understand alternative.
The new mechanism is also much easier to understand and use than
implementing a custom metaclass, and thus should provide a gentler
introduction to the full power Python's metaclass machinery.
.. note::
This PEP, in particular the use of __prepare__ to share a single
namespace amongst multiple class objects, highlights a possible issue
with the attribute lookup caching: when the underlying mapping is updated
by other means, the attribute lookup cache is not invalidated correctly.
Since the optimisation provided by that cache is highly desirable,
some of the ideas in this PEP may need to be declared as officially
unsupported (since the observed behaviour is rather odd when the
caches get out of sync).
Background
==========
@ -81,25 +85,32 @@ metaclass could not call methods that referenced the class by name (as the
name had not yet been bound in the containing scope), similarly, Python 3
metaclasses cannot call methods that rely on the implicit ``__class__``
reference (as it is not populated until after the metaclass has returned
control to the class creation machiner).
control to the class creation machinery).
Finally, when a class uses a custom metaclass, it can pose additional
challenges to the use of multiple inheritance, as a new class cannot
inherit from parent classes with unrelated metaclasses. This means that
it is impossible to add a metaclass to an already published class: such
an addition is a backwards incompatible change due to the risk of metaclass
conflicts.
Proposal
========
This PEP proposes that a mechanism be added to Python 3 that meets the
following criteria:
This PEP proposes that a new mechanism to customise class creation be
added to Python 3.4 that meets the following criteria:
1. Restores the ability for class namespaces to have some influence on the
1. Integrates nicely with class inheritance structures (including mixins and
multiple inheritance)
2. Integrates nicely with the implicit ``__class__`` reference and
zero-argument ``super()`` syntax introduced by PEP 3135
3. Can be added to an existing base class without a significant risk of
introducing backwards compatibility problems
4. Restores the ability for class namespaces to have some influence on the
class creation process (above and beyond populating the namespace itself),
but potentially without the full flexibility of the Python 2 style
``__metaclass__`` hook
2. Integrates nicely with class inheritance structures (including mixins and
multiple inheritance)
3. Integrates nicely with the implicit ``__class__`` reference and
zero-argument ``super()`` syntax introduced by PEP 3135
4. Can be added to an existing base class without a significant risk of
introducing backwards compatibility problems
One mechanism that can achieve this goal is to add a new class
initialisation hook, modelled directly on the existing instance
@ -110,7 +121,6 @@ Specifically, it is proposed that class definitions be able to provide a
class initialisation hook as follows::
class Example:
@classmethod
def __init_class__(cls):
# This is invoked after the class is created, but before any
# explicit decorators are called
@ -121,13 +131,15 @@ class initialisation hook as follows::
If present on the created object, this new hook will be called by the class
creation machinery *after* the ``__class__`` reference has been initialised.
For ``types.new_class()``, it will be called as the last step before
returning the created class object.
returning the created class object. ``__init_class__`` is implicitly
converted to a class method when the class is created (prior to the hook
being invoked).
If a metaclass wishes to block class initialisation for some reason, it
must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
Note, that when ``__init_class__`` is called, the name of the class is not
bound to the new class object yet. As a consequence, the two argument form
yet bound to the new class object. As a consequence, the two argument form
of ``super()`` cannot be used to call methods (e.g., ``super(Example, cls)``
wouldn't work in the example above). However, the zero argument form of
``super()`` works as expected, since the ``__class__`` reference is already
@ -139,19 +151,31 @@ similar mechanism has long been supported by `Zope's ExtensionClass`_),
but the situation has changed sufficiently in recent years that
the idea is worth reconsidering.
However, the introduction of the metaclass ``__prepare__`` method in PEP
3115 allows a further enhancement that was not possible in Python 2: this
PEP also proposes that ``type.__prepare__`` be updated to accept a
``namespace`` keyword-only argument. If present, the value provided as the
``namespace`` argument will be returned from ``type.__prepare__`` instead of
a freshly created dictionary instance. For example, the following will use
the ordered dictionary created in the header as the class namespace::
class OrderedExample(namespace=collections.OrderedDict()):
def __init_class__(cls):
# cls.__dict__ is still a read-only proxy to the class namespace,
# but the underlying storage is the OrderedDict instance
Key Benefits
============
Replaces many use cases for dynamic setting of ``__metaclass__``
-----------------------------------------------------------------
Easier use of custom namespaces for a class
-------------------------------------------
For use cases that don't involve completely replacing the defined class,
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
set ``__init_class__`` instead. For more advanced use cases, introduction of
an explicit metaclass (possibly made available as a required base class) will
still be necessary in order to support Python 3.
Currently, to use a different type (such as ``collections.OrderedDict``) for
a class namespace, or to use a pre-populated namespace, it is necessary to
write and use a custom metaclass. With this PEP, using a custom namespace
becomes as simple as specifying it in the class header.
Easier inheritance of definition time behaviour
@ -201,137 +225,89 @@ implicit ``__class__`` reference introduced by PEP 3135, including methods
that use the zero argument form of ``super()``.
Alternatives
============
Replaces many use cases for dynamic setting of ``__metaclass__``
-----------------------------------------------------------------
For use cases that don't involve completely replacing the defined class,
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
set ``__init_class__`` instead. For more advanced use cases, introduction of
an explicit metaclass (possibly made available as a required base class) will
still be necessary in order to support Python 3.
The Python 3 Status Quo
-----------------------
New Ways of Using Classes
=========================
The Python 3 status quo already offers a great deal of flexibility. For
changes which only affect a single class definition and which can be
specified at the time the code is written, then class decorators can be
used to modify a class explicitly. Class decorators largely ignore class
inheritance and can make full use of methods that rely on the ``__class__``
reference being populated.
The new ``namespace`` keyword in the class header enables a number of
interesting options for controlling the way a class is initialised,
including some aspects of the object models of both Javascript and Ruby.
Using a custom metaclass provides the same level of power as it did in
Python 2. However, it's notable that, unlike class decorators, a metaclass
cannot call any methods that rely on the ``__class__`` reference, as that
reference is not populated until after the metaclass constructor returns
control to the class creation code.
All of the examples below are actually possible today through the use of a
custom metaclass::
One major use case for metaclasses actually closely resembles the use of
class decorators. It occurs whenever a metaclass has an implementation that
uses the following pattern::
class CustomNamespace(type):
@classmethod
def __prepare__(meta, name, bases, *, namespace=None, **kwds):
parent_namespace = super().__prepare__(name, bases, **kwds)
return namespace if namespace is not None else parent_namespace
class Metaclass(type):
def __new__(meta, *args, **kwds):
cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
# Do something with cls
return cls
def __new__(meta, name, bases, ns, *, namespace=None, **kwds):
return super().__new__(meta, name, bases, ns, **kwds)
The key difference between this pattern and a class decorator is that it
is automatically inherited by subclasses. However, it also comes with a
major disadvantage: Python does not allow you to inherit from classes with
unrelated metaclasses.
def __init__(cls, name, bases, ns, *, namespace=None, **kwds):
return super().__init__(name, bases, ns, **kwds)
Thus, the status quo requires that developers choose between the following
two alternatives:
* Use a class decorator, meaning that behaviour is not inherited and must be
requested explicitly on every subclass
* Use a metaclass, meaning that behaviour is inherited, but metaclass
conflicts may make integration with other libraries and frameworks more
difficult than it otherwise would be
If this PEP is ultimately rejected, then this is the existing design that
will remain in place by default.
The advantage of implementing the new keyword directly in
``type.__prepare__`` is that the *only* persistent effect is
the change in the underlying storage of the class attributes. The metaclass
of the class remains unchanged, eliminating many of the drawbacks
typically associated with these kinds of customisations.
Restoring the Python 2 metaclass hook
-------------------------------------
One simple alternative would be to restore support for a Python 2 style
``metaclass`` hook in the class body. This would be checked after the class
body was executed, potentially overwriting the metaclass hint provided in the
class header.
The main attraction of such an approach is that it would simplify porting
Python 2 applications that make use of this hook (especially those that do
so dynamically).
However, this approach does nothing to simplify the process of adding
*inherited* class definition time behaviour, nor does it interoperate
cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
any metaclass based solution, the ``__metaclass__`` hook would have to run
before the ``__class__`` reference has been populated.
Dynamic class decorators
Order preserving classes
------------------------
The original version of this PEP was called "Dynamic class decorators" and
focused solely on a significantly more complicated proposal than that
presented in the current version.
As with the current version, it proposed that a new step be added to the
class creation process, after the metaclass invocation to construct the
class instance and before the application of lexical decorators. However,
instead of a simple process of calling a single class method that relies
on normal inheritance mechanisms, it proposed a far more complicated
procedure that walked the class MRO looking for decorators stored in
iterable ``__decorators__`` attributes.
Using the current version of the PEP, the scheme originally proposed could
be implemented as::
class DynamicDecorators(Base):
@classmethod
def __init_class__(cls):
# Process any classes later in the MRO
try:
mro_chain = super().__init_class__
except AttributeError:
pass
else:
mro_chain()
# Process any __decorators__ attributes in the MRO
for entry in reversed(cls.mro()):
decorators = entry.__dict__.get("__decorators__", ())
for deco in reversed(decorators):
cls = deco(cls)
Any subclasses of ``DynamicDecorators`` would then automatically have the
contents of any ``__decorators__`` attributes processed and invoked.
The mechanism in the current PEP is considered superior, as many issues
to do with ordering and the same decorator being invoked multiple times
just go away, as that kind of thing is taken care of through the use of an
ordinary class method invocation.
::
class OrderedClass(namespace=collections.OrderedDict()):
a = 1
b = 2
c = 3
Automatic metaclass derivation
------------------------------
Prepopulated namespaces
-----------------------
When no appropriate metaclass is found, it's theoretically possible to
automatically derive a metaclass for a new type based on the metaclass hint
and the metaclasses of the bases.
While adding such a mechanism would reduce the risk of spurious metaclass
conflicts, it would do nothing to improve integration with PEP 3135, would
not help with porting Python 2 code that set ``__metaclass__`` dynamically
and would not provide a more straightforward inherited mechanism for invoking
additional operations after the class invocation is complete.
In addition, there would still be a risk of metaclass conflicts in cases
where the base metaclasses were not written with multiple inheritance in
mind. In such situations, there's a chance of introducing latent defects
if one or more metaclasses are not invoked correctly.
::
seed_data = dict(a=1, b=2, c=3)
class PrepopulatedClass(namespace=seed_data.copy()):
pass
Calling the new hook from ``type.__init__``
-------------------------------------------
Cloning a prototype class
-------------------------
::
class NewClass(namespace=Prototype.__dict__.copy()):
pass
Defining an extensible class
----------------------------
::
class Extensible:
namespace = locals()
class ExtendingClass(namespace=Extensible.namespace):
pass
Rejected Design Options
=======================
Calling ``__init_class__`` from ``type.__init__``
-------------------------------------------------
Calling the new hook automatically from ``type.__init__``, would achieve most
of the goals of this PEP. However, using that approach would mean that
@ -340,11 +316,32 @@ relied on the ``__class__`` reference (or used the zero-argument form of
``super()``), and could not make use of those features themselves.
Requiring an explict decorator on ``__init_class__``
----------------------------------------------------
Originally, this PEP required the explicit use of ``@classmethod`` on the
``__init_class__`` decorator. It was made implicit since there's no
sensible interpretation for leaving it out, and that case would need to be
detected anyway in order to give a useful error message.
This decision was reinforced after noticing that the user experience of
defining ``__prepare__`` and forgetting the ``@classmethod`` method
decorator is singularly incomprehensible (particularly since PEP 3115
documents it as an ordinary method, and the current documentation doesn't
explicitly say anything one way or the other).
Reference Implementation
========================
The reference implementation has been posted to the `issue tracker`_.
A reference implementation for __init_class__ has been posted to the
`issue tracker`_. It does not yet include the new ``namespace`` parameter
for ``type.__prepare__``.
TODO
====
* address the 5 points in http://mail.python.org/pipermail/python-dev/2013-February/123970.html
References
==========