314 lines
12 KiB
ReStructuredText
314 lines
12 KiB
ReStructuredText
PEP: 3115
|
|
Title: Metaclasses in Python 3000
|
|
Author: Talin <viridia@gmail.com>
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 07-Mar-2007
|
|
Python-Version: 3.0
|
|
Post-History: 11-Mar-2007, 14-Mar-2007
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP proposes changing the syntax for declaring metaclasses,
|
|
and alters the semantics for how classes with metaclasses are
|
|
constructed.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
There are two rationales for this PEP, both of which are somewhat
|
|
subtle.
|
|
|
|
The primary reason for changing the way metaclasses work, is that
|
|
there are a number of interesting use cases that require the
|
|
metaclass to get involved earlier in the class construction process
|
|
than is currently possible. Currently, the metaclass mechanism is
|
|
essentially a post-processing step. With the advent of class
|
|
decorators, much of these post-processing chores can be taken over
|
|
by the decorator mechanism.
|
|
|
|
In particular, there is an important body of use cases where it
|
|
would be useful to preserve the order in which a class members are
|
|
declared. Ordinary Python objects store their members in a
|
|
dictionary, in which ordering is unimportant, and members are
|
|
accessed strictly by name. However, Python is often used to
|
|
interface with external systems in which the members are organized
|
|
according to an implicit ordering. Examples include declaration of C
|
|
structs; COM objects; Automatic translation of Python classes into
|
|
IDL or database schemas, such as used in an ORM; and so on.
|
|
|
|
In such cases, it would be useful for a Python programmer to specify
|
|
such ordering directly using the declaration order of class members.
|
|
Currently, such orderings must be specified explicitly, using some
|
|
other mechanism (see the ctypes module for an example.)
|
|
|
|
Unfortunately, the current method for declaring a metaclass does
|
|
not allow for this, since the ordering information has already been
|
|
lost by the time the metaclass comes into play. By allowing the
|
|
metaclass to get involved in the class construction process earlier,
|
|
the new system allows the ordering or other early artifacts of
|
|
construction to be preserved and examined.
|
|
|
|
There proposed metaclass mechanism also supports a number of other
|
|
interesting use cases beyond preserving the ordering of declarations.
|
|
One use case is to insert symbols into the namespace of the class
|
|
body which are only valid during class construction. An example of
|
|
this might be "field constructors", small functions that are used in
|
|
the creation of class members. Another interesting possibility is
|
|
supporting forward references, i.e. references to Python
|
|
symbols that are declared further down in the class body.
|
|
|
|
The other, weaker, rationale is purely cosmetic: The current method
|
|
for specifying a metaclass is by assignment to the special variable
|
|
``__metaclass__``, which is considered by some to be aesthetically less
|
|
than ideal. Others disagree strongly with that opinion. This PEP
|
|
will not address this issue, other than to note it, since aesthetic
|
|
debates cannot be resolved via logical proofs.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
In the new model, the syntax for specifying a metaclass is via a
|
|
keyword argument in the list of base classes::
|
|
|
|
class Foo(base1, base2, metaclass=mymeta):
|
|
...
|
|
|
|
Additional keywords will also be allowed here, and will be passed to
|
|
the metaclass, as in the following example::
|
|
|
|
class Foo(base1, base2, metaclass=mymeta, private=True):
|
|
...
|
|
|
|
Note that this PEP makes no attempt to define what these other
|
|
keywords might be - that is up to metaclass implementors to
|
|
determine.
|
|
|
|
More generally, the parameter list passed to a class definition will
|
|
now support all of the features of a function call, meaning that you
|
|
can now use ``*args`` and ``**kwargs``-style arguments in the class base
|
|
list::
|
|
|
|
class Foo(*bases, **kwds):
|
|
...
|
|
|
|
|
|
Invoking the Metaclass
|
|
======================
|
|
|
|
In the current metaclass system, the metaclass object can be any
|
|
callable type. This does not change, however in order to fully
|
|
exploit all of the new features the metaclass will need to have an
|
|
extra attribute which is used during class pre-construction.
|
|
|
|
This attribute is named ``__prepare__``, which is invoked as a function
|
|
before the evaluation of the class body. The ``__prepare__`` function
|
|
takes two positional arguments, and an arbitrary number of keyword
|
|
arguments. The two positional arguments are:
|
|
|
|
======= ====================================
|
|
*name* the name of the class being created.
|
|
*bases* the list of base classes.
|
|
======= ====================================
|
|
|
|
The interpreter always tests for the existence of ``__prepare__`` before
|
|
calling it; If it is not present, then a regular dictionary is used,
|
|
as illustrated in the following Python snippet.
|
|
|
|
::
|
|
|
|
def prepare_class(name, *bases, metaclass=None, **kwargs):
|
|
if metaclass is None:
|
|
metaclass = compute_default_metaclass(bases)
|
|
prepare = getattr(metaclass, '__prepare__', None)
|
|
if prepare is not None:
|
|
return prepare(name, bases, **kwargs)
|
|
else:
|
|
return dict()
|
|
|
|
The example above illustrates how the arguments to 'class' are
|
|
interpreted. The class name is the first argument, followed by
|
|
an arbitrary length list of base classes. After the base classes,
|
|
there may be one or more keyword arguments, one of which can be
|
|
*metaclass*. Note that the *metaclass* argument is not included
|
|
in *kwargs*, since it is filtered out by the normal parameter
|
|
assignment algorithm. (Note also that *metaclass* is a
|
|
keyword-only argument as per :pep:`3102`.)
|
|
|
|
Even though ``__prepare__`` is not required, the default metaclass
|
|
('type') implements it, for the convenience of subclasses calling
|
|
it via super().
|
|
|
|
``__prepare__`` returns a dictionary-like object which is used to store
|
|
the class member definitions during evaluation of the class body.
|
|
In other words, the class body is evaluated as a function block
|
|
(just like it is now), except that the local variables dictionary
|
|
is replaced by the dictionary returned from ``__prepare__``. This
|
|
dictionary object can be a regular dictionary or a custom mapping
|
|
type.
|
|
|
|
This dictionary-like object is not required to support the full
|
|
dictionary interface. A dictionary which supports a limited set of
|
|
dictionary operations will restrict what kinds of actions can occur
|
|
during evaluation of the class body. A minimal implementation might
|
|
only support adding and retrieving values from the dictionary - most
|
|
class bodies will do no more than that during evaluation. For some
|
|
classes, it may be desirable to support deletion as well. Many
|
|
metaclasses will need to make a copy of this dictionary afterwards,
|
|
so iteration or other means for reading out the dictionary contents
|
|
may also be useful.
|
|
|
|
The ``__prepare__`` method will most often be implemented as a class
|
|
method rather than an instance method because it is called before
|
|
the metaclass instance (i.e. the class itself) is created.
|
|
|
|
Once the class body has finished evaluating, the metaclass will be
|
|
called (as a callable) with the class dictionary, which is no
|
|
different from the current metaclass mechanism.
|
|
|
|
Typically, a metaclass will create a custom dictionary - either a
|
|
subclass of dict, or a wrapper around it - that will contain
|
|
additional properties that are set either before or during the
|
|
evaluation of the class body. Then in the second phase, the
|
|
metaclass can use these additional properties to further customize
|
|
the class.
|
|
|
|
An example would be a metaclass that uses information about the
|
|
ordering of member declarations to create a C struct. The metaclass
|
|
would provide a custom dictionary that simply keeps a record of the
|
|
order of insertions. This does not need to be a full 'ordered dict'
|
|
implementation, but rather just a Python list of (key,value) pairs
|
|
that is appended to for each insertion.
|
|
|
|
Note that in such a case, the metaclass would be required to deal
|
|
with the possibility of duplicate keys, but in most cases that is
|
|
trivial. The metaclass can use the first declaration, the last,
|
|
combine them in some fashion, or simply throw an exception. It's up
|
|
to the metaclass to decide how it wants to handle that case.
|
|
|
|
|
|
Example
|
|
=======
|
|
|
|
Here's a simple example of a metaclass which creates a list of
|
|
the names of all class members, in the order that they were
|
|
declared::
|
|
|
|
# The custom dictionary
|
|
class member_table(dict):
|
|
def __init__(self):
|
|
self.member_names = []
|
|
|
|
def __setitem__(self, key, value):
|
|
# if the key is not already defined, add to the
|
|
# list of keys.
|
|
if key not in self:
|
|
self.member_names.append(key)
|
|
|
|
# Call superclass
|
|
dict.__setitem__(self, key, value)
|
|
|
|
# The metaclass
|
|
class OrderedClass(type):
|
|
|
|
# The prepare function
|
|
@classmethod
|
|
def __prepare__(metacls, name, bases): # No keywords in this case
|
|
return member_table()
|
|
|
|
# The metaclass invocation
|
|
def __new__(cls, name, bases, classdict):
|
|
# Note that we replace the classdict with a regular
|
|
# dict before passing it to the superclass, so that we
|
|
# don't continue to record member names after the class
|
|
# has been created.
|
|
result = type.__new__(cls, name, bases, dict(classdict))
|
|
result.member_names = classdict.member_names
|
|
return result
|
|
|
|
class MyClass(metaclass=OrderedClass):
|
|
# method1 goes in array element 0
|
|
def method1(self):
|
|
pass
|
|
|
|
# method2 goes in array element 1
|
|
def method2(self):
|
|
pass
|
|
|
|
|
|
Sample Implementation
|
|
=====================
|
|
|
|
Guido van Rossum has created a patch which implements the new
|
|
functionality: https://bugs.python.org/issue1681101
|
|
|
|
|
|
Alternate Proposals
|
|
===================
|
|
|
|
Josiah Carlson proposed using the name 'type' instead of
|
|
'metaclass', on the theory that what is really being specified is
|
|
the type of the type. While this is technically correct, it is also
|
|
confusing from the point of view of a programmer creating a new
|
|
class. From the application programmer's point of view, the 'type'
|
|
that they are interested in is the class that they are writing; the
|
|
type of that type is the metaclass.
|
|
|
|
There were some objections in the discussion to the 'two-phase'
|
|
creation process, where the metaclass is invoked twice, once to
|
|
create the class dictionary and once to 'finish' the class. Some
|
|
people felt that these two phases should be completely separate, in
|
|
that there ought to be separate syntax for specifying the custom
|
|
dict as for specifying the metaclass. However, in most cases, the
|
|
two will be intimately tied together, and the metaclass will most
|
|
likely have an intimate knowledge of the internal details of the
|
|
class dict. Requiring the programmer to insure that the correct dict
|
|
type and the correct metaclass type are used together creates an
|
|
additional and unneeded burden on the programmer.
|
|
|
|
Another good suggestion was to simply use an ordered dict for all
|
|
classes, and skip the whole 'custom dict' mechanism. This was based
|
|
on the observation that most use cases for a custom dict were for
|
|
the purposes of preserving order information. However, this idea has
|
|
several drawbacks, first because it means that an ordered dict
|
|
implementation would have to be added to the set of built-in types
|
|
in Python, and second because it would impose a slight speed (and
|
|
complexity) penalty on all class declarations. Later, several people
|
|
came up with ideas for use cases for custom dictionaries other
|
|
than preserving field orderings, so this idea was dropped.
|
|
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
It would be possible to leave the existing ``__metaclass__`` syntax in
|
|
place. Alternatively, it would not be too difficult to modify the
|
|
syntax rules of the Py3K translation tool to convert from the old to
|
|
the new syntax.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
[1] [Python-3000] Metaclasses in Py3K (original proposal)
|
|
\ https://mail.python.org/pipermail/python-3000/2006-December/005030.html
|
|
|
|
[2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax)
|
|
\ https://mail.python.org/pipermail/python-3000/2006-December/005033.html
|
|
|
|
[3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init)
|
|
\ https://mail.python.org/pipermail/python-3000/2006-December/005108.html
|
|
|
|
[4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict)
|
|
\ https://mail.python.org/pipermail/python-3000/2006-December/005118.html
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|