310 lines
13 KiB
Plaintext
310 lines
13 KiB
Plaintext
PEP: 3115
|
||
Title: Metaclasses in Python 3000
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Talin <talin@acm.org>
|
||
Status: Accepted
|
||
Type: Standards Track
|
||
Content-Type: text/plain
|
||
Created: 07-Mar-2007
|
||
Python-Version: 3.0
|
||
Post-History: 11-March-2007, 14-March-2007
|
||
|
||
Abstract
|
||
|
||
This PEP proposes changing the syntax for declaring metaclasses,
|
||
and alters the semantics for how classes with metaclasses are
|
||
constructed.
|
||
|
||
|
||
Rationale
|
||
|
||
There are two rationales for this PEP, both of which are somewhat
|
||
subtle.
|
||
|
||
The primary reason for changing the way metaclasses work, is that
|
||
there are a number of interesting use cases that require the
|
||
metaclass to get involved earlier in the class construction process
|
||
than is currently possible. Currently, the metaclass mechanism is
|
||
essentially a post-processing step. With the advent of class
|
||
decorators, much of these post-processing chores can be taken over
|
||
by the decorator mechanism.
|
||
|
||
In particular, there is an important body of use cases where it
|
||
would be useful to preserve the order in which a class members are
|
||
declared. Ordinary Python objects store their members in a
|
||
dictionary, in which ordering is unimportant, and members are
|
||
accessed strictly by name. However, Python is often used to
|
||
interface with external systems in which the members are organized
|
||
according to an implicit ordering. Examples include declaration of C
|
||
structs; COM objects; Automatic translation of Python classes into
|
||
IDL or database schemas, such as used in an ORM; and so on.
|
||
|
||
In such cases, it would be useful for a Python programmer to specify
|
||
such ordering directly using the declaration order of class members.
|
||
Currently, such orderings must be specified explicitly, using some
|
||
other mechanism (see the ctypes module for an example.)
|
||
|
||
Unfortunately, the current method for declaring a metaclass does
|
||
not allow for this, since the ordering information has already been
|
||
lost by the time the metaclass comes into play. By allowing the
|
||
metaclass to get involved in the class construction process earlier,
|
||
the new system allows the ordering or other early artifacts of
|
||
construction to be preserved and examined.
|
||
|
||
There proposed metaclass mechanism also supports a number of other
|
||
interesting use cases beyond preserving the ordering of declarations.
|
||
One use case is to insert symbols into the namespace of the class
|
||
body which are only valid during class construction. An example of
|
||
this might be "field constructors", small functions that are used in
|
||
the creation of class members. Another interesting possibility is
|
||
supporting forward references, i.e. references to Python
|
||
symbols that are declared further down in the class body.
|
||
|
||
The other, weaker, rationale is purely cosmetic: The current method
|
||
for specifying a metaclass is by assignment to the special variable
|
||
__metaclass__, which is considered by some to be aesthetically less
|
||
than ideal. Others disagree strongly with that opinion. This PEP
|
||
will not address this issue, other than to note it, since aesthetic
|
||
debates cannot be resolved via logical proofs.
|
||
|
||
|
||
Specification
|
||
|
||
In the new model, the syntax for specifying a metaclass is via a
|
||
keyword argument in the list of base classes:
|
||
|
||
class Foo(base1, base2, metaclass=mymeta):
|
||
...
|
||
|
||
Additional keywords will also be allowed here, and will be passed to
|
||
the metaclass, as in the following example:
|
||
|
||
class Foo(base1, base2, metaclass=mymeta, private=True):
|
||
...
|
||
|
||
Note that this PEP makes no attempt to define what these other
|
||
keywords might be - that is up to metaclass implementors to
|
||
determine.
|
||
|
||
More generally, the parameter list passed to a class definition will
|
||
now support all of the features of a function call, meaning that you
|
||
can now use *args and **kwargs-style arguments in the class base
|
||
list:
|
||
|
||
class Foo(*bases, **kwds):
|
||
...
|
||
|
||
Invoking the Metaclass
|
||
|
||
In the current metaclass system, the metaclass object can be any
|
||
callable type. This does not change, however in order to fully
|
||
exploit all of the new features the metaclass will need to have an
|
||
extra attribute which is used during class pre-construction.
|
||
|
||
This attribute is named __prepare__, which is invoked as a function
|
||
before the evaluation of the class body. The __prepare__ function
|
||
takes two positional arguments, and an arbitrary number of keyword
|
||
arguments. The two positional arguments are:
|
||
|
||
'name' - the name of the class being created.
|
||
'bases' - the list of base classes.
|
||
|
||
The interpreter always tests for the existence of __prepare__ before
|
||
calling it; If it is not present, then a regular dictionary is used,
|
||
as illustrated in the following Python snippet.
|
||
|
||
def prepare_class(name, *bases, metaclass=None, **kwargs):
|
||
if metaclass is None:
|
||
metaclass = compute_default_metaclass(bases)
|
||
prepare = getattr(metaclass, '__prepare__', None)
|
||
if prepare is not None:
|
||
return prepare(name, bases, **kwargs)
|
||
else:
|
||
return dict()
|
||
|
||
The example above illustrates how the arguments to 'class' are
|
||
interpreted. The class name is the first argument, followed by
|
||
an arbitrary length list of base classes. After the base classes,
|
||
there may be one or more keyword arguments, one of which can be
|
||
'metaclass'. Note that the 'metaclass' argument is not included
|
||
in kwargs, since it is filtered out by the normal parameter
|
||
assignment algorithm. (Note also that 'metaclass' is a keyword-
|
||
only argument as per PEP 3102 [6].)
|
||
|
||
__prepare__ returns a dictionary-like object which is used to store
|
||
the class member definitions during evaluation of the class body.
|
||
In other words, the class body is evaluated as a function block
|
||
(just like it is now), except that the local variables dictionary
|
||
is replaced by the dictionary returned from __prepare__. This
|
||
dictionary object can be a regular dictionary or a custom mapping
|
||
type.
|
||
|
||
This dictionary-like object is not required to support the full
|
||
dictionary interface. A dictionary which supports a limited set of
|
||
dictionary operations will restrict what kinds of actions can occur
|
||
during evaluation of the class body. A minimal implementation might
|
||
only support adding and retrieving values from the dictionary - most
|
||
class bodies will do no more than that during evaluation. For some
|
||
classes, it may be desirable to support deletion as well. Many
|
||
metaclasses will need to make a copy of this dictionary afterwards,
|
||
so iteration or other means for reading out the dictionary contents
|
||
may also be useful.
|
||
|
||
The __prepare__ method will most often be implemented as a class
|
||
method rather than an instance method because it is called before
|
||
the metaclass instance (i.e. the class itself) is created.
|
||
|
||
Once the class body has finished evaluating, the metaclass will be
|
||
called (as a callable) with the class dictionary, which is no
|
||
different from the current metaclass mechanism.
|
||
|
||
Typically, a metaclass will create a custom dictionary - either a
|
||
subclass of dict, or a wrapper around it - that will contain
|
||
additional properties that are set either before or during the
|
||
evaluation of the class body. Then in the second phase, the
|
||
metaclass can use these additional properties to further customize
|
||
the class.
|
||
|
||
An example would be a metaclass that uses information about the
|
||
ordering of member declarations to create a C struct. The metaclass
|
||
would provide a custom dictionary that simply keeps a record of the
|
||
order of insertions. This does not need to be a full 'ordered dict'
|
||
implementation, but rather just a Python list of (key,value) pairs
|
||
that is appended to for each insertion.
|
||
|
||
Note that in such a case, the metaclass would be required to deal
|
||
with the possibility of duplicate keys, but in most cases that is
|
||
trivial. The metaclass can use the first declaration, the last,
|
||
combine them in some fashion, or simply throw an exception. It's up
|
||
to the metaclass to decide how it wants to handle that case.
|
||
|
||
Example:
|
||
|
||
Here's a simple example of a metaclass which creates a list of
|
||
the names of all class members, in the order that they were
|
||
declared:
|
||
|
||
# The custom dictionary
|
||
class member_table(dict):
|
||
def __init__(self):
|
||
self.member_names = []
|
||
|
||
def __setitem__(self, key, value):
|
||
# if the key is not already defined, add to the
|
||
# list of keys.
|
||
if key not in self:
|
||
self.member_names.append(key)
|
||
|
||
# Call superclass
|
||
dict.__setitem__(self, key, value)
|
||
|
||
# The metaclass
|
||
class OrderedClass(type):
|
||
|
||
# The prepare function
|
||
@classmethod
|
||
def __prepare__(metacls, name, bases): # No keywords in this case
|
||
return member_table()
|
||
|
||
# The metaclass invocation
|
||
def __new__(cls, name, bases, classdict):
|
||
# Note that we replace the classdict with a regular
|
||
# dict before passing it to the superclass, so that we
|
||
# don't continue to record member names after the class
|
||
# has been created.
|
||
result = type.__new__(cls, name, bases, dict(classdict))
|
||
result.member_names = classdict.member_names
|
||
return result
|
||
|
||
class MyClass(metaclass=OrderedClass):
|
||
# method1 goes in array element 0
|
||
def method1(self):
|
||
pass
|
||
|
||
# method2 goes in array element 1
|
||
def method2(self):
|
||
pass
|
||
|
||
Sample Implementation:
|
||
|
||
Guido van Rossum has created a patch which implements the new
|
||
functionality:
|
||
|
||
http://python.org/sf/1681101
|
||
|
||
Alternate Proposals
|
||
|
||
Josiah Carlson proposed using the name 'type' instead of
|
||
'metaclass', on the theory that what is really being specified is
|
||
the type of the type. While this is technically correct, it is also
|
||
confusing from the point of view of a programmer creating a new
|
||
class. From the application programmer's point of view, the 'type'
|
||
that they are interested in is the class that they are writing; the
|
||
type of that type is the metaclass.
|
||
|
||
There were some objections in the discussion to the 'two-phase'
|
||
creation process, where the metaclass is invoked twice, once to
|
||
create the class dictionary and once to 'finish' the class. Some
|
||
people felt that these two phases should be completely separate, in
|
||
that there ought to be separate syntax for specifying the custom
|
||
dict as for specifying the metaclass. However, in most cases, the
|
||
two will be intimately tied together, and the metaclass will most
|
||
likely have an intimate knowledge of the internal details of the
|
||
class dict. Requiring the programmer to insure that the correct dict
|
||
type and the correct metaclass type are used together creates an
|
||
additional and unneeded burden on the programmer.
|
||
|
||
Another good suggestion was to simply use an ordered dict for all
|
||
classes, and skip the whole 'custom dict' mechanism. This was based
|
||
on the observation that most use cases for a custom dict were for
|
||
the purposes of preserving order information. However, this idea has
|
||
several drawbacks, first because it means that an ordered dict
|
||
implementation would have to be added to the set of built-in types
|
||
in Python, and second because it would impose a slight speed (and
|
||
complexity) penalty on all class declarations. Later, several people
|
||
came up with ideas for use cases for custom dictionaries other
|
||
than preserving field orderings, so this idea was dropped.
|
||
|
||
|
||
Backwards Compatibility
|
||
|
||
It would be possible to leave the existing __metaclass__ syntax in
|
||
place. Alternatively, it would not be too difficult to modify the
|
||
syntax rules of the Py3K translation tool to convert from the old to
|
||
the new syntax.
|
||
|
||
|
||
References
|
||
|
||
[1] [Python-3000] Metaclasses in Py3K (original proposal)
|
||
http://mail.python.org/pipermail/python-3000/2006-December/005030.html
|
||
|
||
[2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax)
|
||
http://mail.python.org/pipermail/python-3000/2006-December/005033.html
|
||
|
||
[3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init)
|
||
http://mail.python.org/pipermail/python-3000/2006-December/005108.html
|
||
|
||
[4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict)
|
||
http://mail.python.org/pipermail/python-3000/2006-December/005118.html
|
||
|
||
[5] PEP 359: The 'make' statement -
|
||
http://www.python.org/dev/peps/pep-0359/
|
||
|
||
[6] PEP 3102: Keyword-only arguments -
|
||
http://www.python.org/dev/peps/pep-3102/
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|