iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

merged

This commit is contained in:
Martin v. Löwis 2012-06-18 16:28:24 +02:00
parent ae1ca8dba5 cfc6f4fc85
commit 7395ae5657
6 changed files with 760 additions and 271 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

651
pep-0362.txt
View File

@ -2,332 +2,468 @@ PEP: 362
Title: Function Signature Object
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>
Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>,
Yury Selivanov <yselivanov@sprymix.com>, Larry Hastings <larry@hastings.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 21-Aug-2006
Python-Version: 2.6
Post-History: 05-Sep-2007
Python-Version: 3.3
Post-History: 04-Jun-2012
Abstract
========
Python has always supported powerful introspection capabilities,
including that for functions and methods (for the rest of this PEP the
word "function" refers to both functions and methods). Taking a
function object, you can fully reconstruct the function's signature.
Unfortunately it is a little unruly having to look at all the
different attributes to pull together complete information for a
function's signature.
including introspecting functions and methods (for the rest of
this PEP, "function" refers to both functions and methods). By
examining a function object you can fully reconstruct the function's
signature. Unfortunately this information is stored in an inconvenient
manner, and is spread across a half-dozen deeply nested attributes.
This PEP proposes an object representation for function signatures.
This should help facilitate introspection on functions for various
uses. The introspection information contains all possible information
about the parameters in a signature (including Python 3.0 features).
This PEP proposes a new representation for function signatures.
The new representation contains all necessary information about a function
and its parameters, and makes introspection easy and straightforward.
This object, though, is not meant to replace existing ways of
introspection on a function's signature. The current solutions are
there to make Python's execution work in an efficient manner. The
proposed object representation is only meant to help make application
code have an easier time to query a function on its signature.
Purpose
=======
An object representation of a function's call signature should provide
an easy way to introspect what a function expects as arguments. It
does not need to be a "live" representation, though; the signature can
be inferred once and stored without changes to the signature object
representation affecting the function it represents (but this is an
`Open Issues`_).
Indirection of signature introspection can also occur. If a
decorator took a decorated function's signature object and set it on
the decorating function then introspection could be redirected to what
is actually expected instead of the typical ``*args, **kwargs``
signature of decorating functions.
However, this object does not replace the existing function
metadata, which is used by Python itself to execute those
functions. The new metadata object is intended solely to make
function introspection easier for Python programmers.
Signature Object
================
The overall signature of an object is represented by the Signature
object. This object is to store a `Parameter object`_ for each
parameter in the signature. It is also to store any information
about the function itself that is pertinent to the signature.
A Signature object represents the call signature of a function and
its return annotation. For each parameter accepted by the function
it stores a `Parameter object`_ in its ``parameters`` collection.
A Signature object has the following structure attributes:
A Signature object has the following public attributes and methods:
* name : str
Name of the function. This is not fully qualified because
function objects for methods do not know the class they are
contained within. This makes functions and methods
indistinguishable from one another when passed to decorators,
preventing proper creation of a fully qualified name.
* var_args : str
Name of the variable positional parameter (i.e., ``*args``), if
present, or the empty string.
* var_kw_args : str
Name of the variable keyword parameter (i.e., ``**kwargs``), if
present, or the empty string.
* var_annotations: dict(str, object)
Dict that contains the annotations for the variable parameters.
The keys are of the variable parameter with values of the
annotation. If an annotation does not exist for a variable
parameter then the key does not exist in the dict.
* return_annotation : object
If present, the attribute is set to the annotation for the return
type of the function.
* parameters : list(Parameter)
List of the parameters of the function as represented by
Parameter objects in the order of its definition (keyword-only
arguments are in the order listed by ``code.co_varnames``).
* bind(\*args, \*\*kwargs) -> dict(str, object)
Create a mapping from arguments to parameters. The keys are the
names of the parameter that an argument maps to with the value
being the value the parameter would have if this function was
called with the given arguments.
The annotation for the return type of the function if specified.
If the function has no annotation for its return type, this
attribute is not set.
* parameters : OrderedDict
An ordered mapping of parameters' names to the corresponding
Parameter objects (keyword-only arguments are in the same order
as listed in ``code.co_varnames``).
* bind(\*args, \*\*kwargs) -> BoundArguments
Creates a mapping from positional and keyword arguments to
parameters. Raises a ``TypeError`` if the passed arguments do
not match the signature.
* bind_partial(\*args, \*\*kwargs) -> BoundArguments
Works the same way as ``bind()``, but allows the omission
of some required arguments (mimics ``functools.partial``
behavior.) Raises a ``TypeError`` if the passed arguments do
not match the signature.
* format(...) -> str
Formats the Signature object to a string. Optional arguments allow
for custom render functions for parameter names,
annotations and default values, along with custom separators.
Signature objects also have the following methods:
Signature implements the ``__str__`` method, which fallbacks to the
``Signature.format()`` call.
* __getitem__(self, key : str) -> Parameter
Returns the Parameter object for the named parameter.
* __iter__(self)
Returns an iterator that returns Parameter objects in their
sequential order based on their 'position' attribute.
It's possible to test Signatures for equality. Two signatures
are equal when they have equal parameters and return annotations.
The Signature object is stored in the ``__signature__`` attribute of
a function. When it is to be created is discussed in
`Open Issues`_.
Changes to the Signature object, or to any of its data members,
do not affect the function itself.
Parameter Object
================
A function's signature is made up of several parameters. Python's
different kinds of parameters is quite large and rich and continues to
grow. Parameter objects represent any possible parameter.
Originally the plan was to represent parameters using a list of
parameter names on the Signature object along with various dicts keyed
on parameter names to disseminate the various pieces of information
one can know about a parameter. But the decision was made to
incorporate all information about a parameter in a single object so
as to make extending the information easier. This was originally put
forth by Talin and the preferred form of Guido (as discussed at the
2006 Google Sprint).
Python's expressive syntax means functions can accept many different
kinds of parameters with many subtle semantic differences. We
propose a rich Parameter object designed to represent any possible
function parameter.
The structure of the Parameter object is:
* name : (str | tuple(str))
The name of the parameter as a string if it is not a tuple. If
the argument is a tuple then a tuple of strings is used.
* position : int
The position of the parameter within the signature of the
function (zero-indexed). For keyword-only parameters the position
value is arbitrary while not conflicting with positional
parameters. The suggestion of setting the attribute to None or -1
to represent keyword-only parameters was rejected to prevent
variable type usage and as a possible point of errors,
respectively.
* default_value : object
The default value for the parameter, if present, else the
attribute does not exist.
* keyword_only : bool
True if the parameter is keyword-only, else False.
* annotation
Set to the annotation for the parameter. If ``has_annotation`` is
False then the attribute does not exist to prevent accidental use.
* name : str
The name of the parameter as a string.
* default : object
The default value for the parameter, if specified. If the
parameter has no default value, this attribute is not set.
* annotation : object
The annotation for the parameter if specified. If the
parameter has no annotation, this attribute is not set.
* kind : str
Describes how argument values are bound to the parameter.
Possible values:
* ``Parameter.POSITIONAL_ONLY`` - value must be supplied
as a positional argument.
Python has no explicit syntax for defining positional-only
parameters, but many builtin and extension module functions
(especially those that accept only one or two parameters)
accept them.
* ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
supplied as either a keyword or positional argument
(this is the standard binding behaviour for functions
implemented in Python.)
* ``Parameter.KEYWORD_ONLY`` - value must be supplied
as a keyword argument. Keyword only parameters are those
which appear after a "*" or "\*args" entry in a Python
function definition.
* ``Parameter.VAR_POSITIONAL`` - a tuple of positional
arguments that aren't bound to any other parameter.
This corresponds to a "\*args" parameter in a Python
function definition.
* ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
that aren't bound to any other parameter. This corresponds
to a "\*\*kwds" parameter in a Python function definition.
* implemented : bool
True if the parameter is implemented for use. Some platforms
implement functions but can't support specific parameters
(e.g. "mode" for ``os.mkdir``). Passing in an unimplemented
parameter may result in the parameter being ignored,
or in NotImplementedError being raised. It is intended that
all conditions where ``implemented`` may be False be
thoroughly documented.
Two parameters are equal when all their attributes are equal.
BoundArguments Object
=====================
Result of a ``Signature.bind`` call. Holds the mapping of arguments
to the function's parameters.
Has the following public attributes:
* arguments : OrderedDict
An ordered, mutable mapping of parameters' names to arguments' values.
Does not contain arguments' default values.
* args : tuple
Tuple of positional arguments values. Dynamically computed from
the 'arguments' attribute.
* kwargs : dict
Dict of keyword arguments values. Dynamically computed from
the 'arguments' attribute.
The ``arguments`` attribute should be used in conjunction with
``Signature.parameters`` for any arguments processing purposes.
``args`` and ``kwargs`` properties can be used to invoke functions:
::
def test(a, *, b):
...
sig = signature(test)
ba = sig.bind(10, b=20)
test(*ba.args, **ba.kwargs)
Implementation
==============
An implementation can be found in Python's sandbox [#impl]_.
There is a function named ``signature()`` which
returns the value stored on the ``__signature__`` attribute if it
exists, else it creates the Signature object for the
function and sets ``__signature__``. For methods this is stored
directly on the im_func function object since that is what decorators
work with.
The implementation adds a new function ``signature()`` to the ``inspect``
module. The function is the preferred way of getting a ``Signature`` for
a callable object.
The function implements the following algorithm:
- If the object is not callable - raise a TypeError
- If the object has a ``__signature__`` attribute and if it
is not ``None`` - return a deepcopy of it
- If it is ``None`` and the object is an instance of
``BuiltinFunction``, raise a ``ValueError``
- If it has a ``__wrapped__`` attribute, return
``signature(object.__wrapped__)``
- If the object is a an instance of ``FunctionType`` construct
and return a new ``Signature`` for it
- If the object is a method or a classmethod, construct and return
a new ``Signature`` object, with its first parameter (usually
``self`` or ``cls``) removed
- If the object is a staticmethod, construct and return
a new ``Signature`` object
- If the object is an instance of ``functools.partial``, construct
a new ``Signature`` from its ``partial.func`` attribute, and
account for already bound ``partial.args`` and ``partial.kwargs``
- If the object is a class or metaclass:
- If the object's type has a ``__call__`` method defined in
its MRO, return a Signature for it
- If the object has a ``__new__`` method defined in its class,
return a Signature object for it
- If the object has a ``__init__`` method defined in its class,
return a Signature object for it
- Return ``signature(object.__call__)``
Note, that the ``Signature`` object is created in a lazy manner, and
is not automatically cached. If, however, the Signature object was
explicitly cached by the user, ``signature()`` returns a new deepcopy
of it on each invocation.
An implementation for Python 3.3 can be found at [#impl]_.
The python issue tracking the patch is [#issue]_.
Design Considerations
=====================
No implicit caching of Signature objects
----------------------------------------
The first PEP design had a provision for implicit caching of ``Signature``
objects in the ``inspect.signature()`` function. However, this has the
following downsides:
* If the ``Signature`` object is cached then any changes to the function
it describes will not be reflected in it. However, If the caching is
needed, it can be always done manually and explicitly
* It is better to reserve the ``__signature__`` attribute for the cases
when there is a need to explicitly set to a ``Signature`` object that
is different from the actual one
Examples
========
Visualizing Callable Objects' Signature
---------------------------------------
Let's define some classes and functions:
::
from inspect import signature
from functools import partial, wraps
class FooMeta(type):
def __new__(mcls, name, bases, dct, *, bar:bool=False):
return super().__new__(mcls, name, bases, dct)
def __init__(cls, name, bases, dct, **kwargs):
return super().__init__(name, bases, dct)
class Foo(metaclass=FooMeta):
def __init__(self, spam:int=42):
self.spam = spam
def __call__(self, a, b, *, c) -> tuple:
return a, b, c
def shared_vars(*shared_args):
"""Decorator factory that defines shared variables that are
passed to every invocation of the function"""
def decorator(f):
@wraps(f)
def wrapper(*args, **kwds):
full_args = shared_args + args
return f(*full_args, **kwds)
# Override signature
sig = wrapper.__signature__ = signature(f)
for __ in shared_args:
sig.parameters.popitem(last=False)
return wrapper
return decorator
@shared_vars({})
def example(_state, a, b, c):
return _state, a, b, c
def format_signature(obj):
return str(signature(obj))
Now, in the python REPL:
::
>>> format_signature(FooMeta)
'(name, bases, dct, *, bar:bool=False)'
>>> format_signature(Foo)
'(spam:int=42)'
>>> format_signature(Foo.__call__)
'(self, a, b, *, c) -> tuple'
>>> format_signature(Foo().__call__)
'(a, b, *, c) -> tuple'
>>> format_signature(partial(Foo().__call__, 1, c=3))
'(b, *, c=3) -> tuple'
>>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
'(*, c=20) -> tuple'
>>> format_signature(example)
'(a, b, c)'
>>> format_signature(partial(example, 1, 2))
'(c)'
>>> format_signature(partial(partial(example, 1, b=2), c=3))
'(b=2, c=3)'
Annotation Checker
------------------
::
def quack_check(fxn):
"""Decorator to verify arguments and return value quack as they should.
import inspect
import functools
Positional arguments.
>>> @quack_check
... def one_arg(x:int): pass
...
>>> one_arg(42)
>>> one_arg('a')
Traceback (most recent call last):
...
TypeError: 'a' does not quack like a <type 'int'>
def checktypes(func):
'''Decorator to verify arguments and return types
Example:
*args
>>> @quack_check
... def var_args(*args:int): pass
...
>>> var_args(*[1,2,3])
>>> var_args(*[1,'b',3])
Traceback (most recent call last):
...
TypeError: *args contains a a value that does not quack like a <type 'int'>
>>> @checktypes
... def test(a:int, b:str) -> int:
... return int(a * b)
**kwargs
>>> @quack_check
... def var_kw_args(**kwargs:int): pass
...
>>> var_kw_args(**{'a': 1})
>>> var_kw_args(**{'a': 'A'})
Traceback (most recent call last):
...
TypeError: **kwargs contains a value that does not quack like a <type 'int'>
>>> test(10, '1')
1111111111
Return annotations.
>>> @quack_check
... def returned(x) -> int: return x
...
>>> returned(42)
42
>>> returned('a')
Traceback (most recent call last):
...
TypeError: the return value 'a' does not quack like a <type 'int'>
>>> test(10, 1)
Traceback (most recent call last):
...
ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
'''
"""
# Get the signature; only needs to be calculated once.
sig = Signature(fxn)
def check(*args, **kwargs):
# Find out the variable -> value bindings.
bindings = sig.bind(*args, **kwargs)
# Check *args for the proper quack.
sig = inspect.signature(func)
types = {}
for param in sig.parameters.values():
# Iterate through function's parameters and build the list of
# arguments types
try:
duck = sig.var_annotations[sig.var_args]
except KeyError:
pass
type_ = param.annotation
except AttributeError:
continue
else:
# Check every value in *args.
for value in bindings[sig.var_args]:
if not isinstance(value, duck):
raise TypeError("*%s contains a a value that does not "
"quack like a %r" %
(sig.var_args, duck))
# Remove it from the bindings so as to not check it again.
del bindings[sig.var_args]
# **kwargs.
try:
duck = sig.var_annotations[sig.var_kw_args]
except (KeyError, AttributeError):
pass
else:
# Check every value in **kwargs.
for value in bindings[sig.var_kw_args].values():
if not isinstance(value, duck):
raise TypeError("**%s contains a value that does not "
"quack like a %r" %
(sig.var_kw_args, duck))
# Remove from bindings so as to not check again.
del bindings[sig.var_kw_args]
# For each remaining variable ...
for var, value in bindings.items():
# See if an annotation was set.
if not inspect.isclass(type_):
# Not a type, skip it
continue
types[param.name] = type_
# If the argument has a type specified, let's check that its
# default value (if present) conforms with the type.
try:
duck = sig[var].annotation
default = param.default
except AttributeError:
continue
# Check that the value quacks like it should.
if not isinstance(value, duck):
raise TypeError('%r does not quack like a %s' % (value, duck))
else:
# All the ducks quack fine; let the call proceed.
returned = fxn(*args, **kwargs)
# Check the return value.
else:
if not isinstance(default, type_):
raise ValueError("{func}: wrong type of a default value for {arg!r}". \
format(func=func.__qualname__, arg=param.name))
def check_type(sig, arg_name, arg_type, arg_value):
# Internal function that encapsulates arguments type checking
if not isinstance(arg_value, arg_type):
raise ValueError("{func}: wrong type of {arg!r} argument, " \
"{exp!r} expected, got {got!r}". \
format(func=func.__qualname__, arg=arg_name,
exp=arg_type.__name__, got=type(arg_value).__name__))
@functools.wraps(func)
def wrapper(*args, **kwargs):
# Let's bind the arguments
ba = sig.bind(*args, **kwargs)
for arg_name, arg in ba.arguments.items():
# And iterate through the bound arguments
try:
if not isinstance(returned, sig.return_annotation):
raise TypeError('the return value %r does not quack like '
'a %r' % (returned,
sig.return_annotation))
except AttributeError:
pass
return returned
# Full-featured version would set function metadata.
return check
type_ = types[arg_name]
except KeyError:
continue
else:
# OK, we have a type for the argument, lets get the corresponding
# parameter description from the signature object
param = sig.parameters[arg_name]
if param.kind == param.VAR_POSITIONAL:
# If this parameter is a variable-argument parameter,
# then we need to check each of its values
for value in arg:
check_type(sig, arg_name, type_, value)
elif param.kind == param.VAR_KEYWORD:
# If this parameter is a variable-keyword-argument parameter:
for subname, value in arg.items():
check_type(sig, arg_name + ':' + subname, type_, value)
else:
# And, finally, if this parameter a regular one:
check_type(sig, arg_name, type_, arg)
result = func(*ba.args, **ba.kwargs)
# The last bit - let's check that the result is correct
try:
return_type = sig.return_annotation
except AttributeError:
# Looks like we don't have any restriction on the return type
pass
else:
if isinstance(return_type, type) and not isinstance(result, return_type):
raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
format(func=func.__qualname__, exp=return_type.__name__,
got=type(result).__name__))
return result
return wrapper
Open Issues
===========
Render Function Signature to HTML
---------------------------------
When to construct the Signature object?
---------------------------------------
::
The Signature object can either be created in an eager or lazy
fashion. In the eager situation, the object can be created during
creation of the function object. In the lazy situation, one would
pass a function object to a function and that would generate the
Signature object and store it to ``__signature__`` if
needed, and then return the value of ``__signature__``.
import inspect
def format_to_html(func):
sig = inspect.signature(func)
Should ``Signature.bind`` return Parameter objects as keys?
-----------------------------------------------------------
html = sig.format(token_params_separator='<span class="t-comma">,</span>',
token_colon='<span class="t-colon">:</span>',
token_eq='<span class="t-eq">=</span>',
token_return_annotation='<span class="t-ra">-&gt;</span>',
token_left_paren='<span class="t-lp">(</span>',
token_right_paren='<span class="t-lp">)</span>',
token_kwonly_separator='<span class="t-ast">*</span>',
format_name=lambda name: '<span class="name">'+name+'</span>')
Instead of returning a dict with keys consisting of the name of the
parameters, would it be more useful to instead use Parameter
objects? The name of the argument can easily be retrieved from the
key (and the name would be used as the hash for a Parameter object).
Have ``var_args`` and ``_var_kw_args`` default to ``None``?
------------------------------------------------------------
It has been suggested by Fred Drake that these two attributes have a
value of ``None`` instead of empty strings when they do not exist.
The answer to this question will influence what the defaults are for
other attributes as well.
Deprecate ``inspect.getargspec()`` and ``.formatargspec()``?
-------------------------------------------------------------
Since the Signature object replicates the use of ``getargspec()``
from the ``inspect`` module it might make sense to deprecate it in
2.6. ``formatargspec()`` could also go if Signature objects gained a
__str__ representation.
Issue with that is types such as ``int``, when used as annotations,
do not lend themselves for output (e.g., ``"<type 'int'>"`` is the
string represenation for ``int``). The repr representation of types
would need to change in order to make this reasonable.
Have the objects be "live"?
---------------------------
Jim Jewett pointed out that Signature and Parameter objects could be
"live". That would mean requesting information would be done on the
fly instead of caching it on the objects. It would also allow for
mutating the function if the Signature or Parameter objects were
mutated.
return '<span class="py-func">{}</span>'.format(html)
References
==========
.. [#impl] pep362 directory in Python's sandbox
(http://svn.python.org/view/sandbox/trunk/pep362/)
.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview)
.. [#issue] issue 15008 (http://bugs.python.org/issue15008)
Copyright
@ -335,7 +471,6 @@ Copyright
This document has been placed in the public domain.
..
Local Variables:

4
pep-0398.txt
View File

@ -70,6 +70,7 @@ Implemented / Final PEPs:
* PEP 417: Including mock in the Standard Library
* PEP 418: Add monotonic time, performance counter, and process time functions
* PEP 420: Implicit Namespace Packages
* PEP 421: Adding sys.implementation
* PEP 3118: Revising the buffer protocol (protocol semantics finalised)
* PEP 3144: IP Address manipulation library
* PEP 3151: Reworking the OS and IO exception hierarchy
@ -87,8 +88,6 @@ Candidate PEPs:
* PEP 362: Function Signature Object
* PEP 397: Python launcher for Windows
* PEP 421: Adding sys.implementation
* PEP 3143: Standard daemon process library
* PEP 3154: Pickle protocol version 4
(Note that these are not accepted yet and even if they are, they might
@ -105,6 +104,7 @@ Other planned large-scale changes:
Deferred to post-3.3:
* PEP 395: Qualified Names for Modules
* PEP 3143: Standard daemon process library
* Breaking out standard library and docs in separate repos
Copyright

18
pep-0405.txt
View File

@ -4,7 +4,7 @@ Version: $Revision$
Last-Modified: $Date$
Author: Carl Meyer <carl@oddbird.net>
BDFL-Delegate: Nick Coghlan
Status: Accepted
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Jun-2011
@ -285,15 +285,15 @@ Include files
Current virtualenv handles include files in this way:
On POSIX systems where the installed Python's include files are found
in ``${base_prefix}/include/pythonX.X``, virtualenv creates
``${venv}/include/`` and symlink ``${base_prefix}/include/pythonX.X``
On POSIX systems where the installed Python's include files are found in
``${base_prefix}/include/pythonX.X``, virtualenv creates
``${venv}/include/`` and symlinks ``${base_prefix}/include/pythonX.X``
to ``${venv}/include/pythonX.X``. On Windows, where Python's include
files are found in ``{{ sys.prefix }}/Include`` and symlinks are not
reliably available, virtualenv copies ``{{ sys.prefix }}/Include`` to
``${venv}/Include``. This ensures that extension modules built and
installed within the virtualenv will always find the Python header
files they need in the expected location relative to ``sys.prefix``.
installed within the virtualenv will always find the Python header files
they need in the expected location relative to ``sys.prefix``.
This solution is not ideal when an extension module installs its own
header files, as the default installation location for those header
@ -467,10 +467,10 @@ than ``sys.site_prefix`` or the appropriate ``site`` API to find
site-packages directories.
The most notable case is probably `setuptools`_ and its fork
`distribute`_, which mostly use ``distutils``and ``sysconfig`` APIs,
`distribute`_, which mostly use ``distutils`` and ``sysconfig`` APIs,
but do use ``sys.prefix`` directly to build up a list of site
directories for pre-flight checking where ``pth`` files can usefully
be placed.
directories for pre-flight checking where ``pth`` files can usefully be
placed.
Otherwise, a `Google Code Search`_ turns up what appears to be a
roughly even mix of usage between packages using ``sys.prefix`` to

2
pep-0420.txt
View File

@ -3,7 +3,7 @@ Title: Implicit Namespace Packages
Version: $Revision$
Last-Modified: $Date$
Author: Eric V. Smith <eric@trueblade.com>
Status: Accepted
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 19-Apr-2012

2
pep-0421.txt
View File

@ -4,7 +4,7 @@ Version: $Revision$
Last-Modified: $Date$
Author: Eric Snow <ericsnowcurrently@gmail.com>
BDFL-Delegate: Barry Warsaw
Status: Accepted
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 26-April-2012

354
pep-0422.txt Normal file
View File

@ -0,0 +1,354 @@
PEP: 422
Title: Simple class initialisation hook
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 5-Jun-2012
Python-Version: 3.4
Post-History: 5-Jun-2012
Abstract
========
In Python 2, the body of a class definition could modify the way a class
was created (or simply arrange to run other code after the class was created)
by setting the ``__metaclass__`` attribute in the class body. While doing
this implicitly from called code required the use of an implementation detail
(specifically, ``sys._getframes()``), it could also be done explicitly in a
fully supported fashion (for example, by passing ``locals()`` to an
function that calculated a suitable ``__metaclass__`` value)
There is currently no corresponding mechanism in Python 3 that allows the
code executed in the class body to directly influence how the class object
is created. Instead, the class creation process is fully defined by the
class header, before the class body even begins executing.
This PEP proposes a mechanism that will once again allow the body of a
class definition to more directly influence the way a class is created
(albeit in a more constrained fashion), as well as replacing some current
uses of metaclasses with a simpler, easier to understand alternative.
Background
==========
For an already created class ``cls``, the term "metaclass" has a clear
meaning: it is the value of ``type(cls)``.
*During* class creation, it has another meaning: it is also used to refer to
the metaclass hint that may be provided as part of the class definition.
While in many cases these two meanings end up referring to one and the same
object, there are two situations where that is not the case:
* If the metaclass hint refers to a subclass of ``type``, then it is
considered as a candidate metaclass along with the metaclasses of all of
the parents of the class being defined. If a more appropriate metaclass is
found amongst the candidates, then it will be used instead of the one
given in the metaclass hint.
* Otherwise, an explicit metaclass hint is assumed to be a factory function
and is called directly to create the class object. In this case, the final
metaclass will be determined by the factory function definition. In the
typical case (where the factory functions just calls ``type``, or, in
Python 3.3 or later, ``types.new_class``) the actual metaclass is then
determined based on the parent classes.
It is notable that only the actual metaclass is inherited - a factory
function used as a metaclass hook sees only the class currently being
defined, and is not invoked for any subclasses.
In Python 3, the metaclass hint is provided using the ``metaclass=Meta``
keyword syntax in the class header. This allows the ``__prepare__`` method
on the metaclass to be used to create the ``locals()`` namespace used during
execution of the class body (for example, specifying the use of
``collections.OrderedDict`` instead of a regular ``dict``).
In Python 2, there was no ``__prepare__`` method (that API was added for
Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__``
attribute, and the class creation process would extract that value from the
class namespace to use as the metaclass hint. There is `published code`_ that
makes use of this feature.
Another new feature in Python 3 is the zero-argument form of the ``super()``
builtin, introduced by PEP 3135. This feature uses an implicit ``__class__``
reference to the class being defined to replace the "by name" references
required in Python 2. Just as code invoked during execution of a Python 2
metaclass could not call methods that referenced the class by name (as the
name had not yet been bound in the containing scope), similarly, Python 3
metaclasses cannot call methods that rely on the implicit ``__class__``
reference (as it is not populated until after the metaclass has returned
control to the class creation machiner).
Proposal
========
This PEP proposes that a mechanism be added to Python 3 that meets the
following criteria:
1. Restores the ability for class namespaces to have some influence on the
class creation process (above and beyond populating the namespace itself),
but potentially without the full flexibility of the Python 2 style
``__metaclass__`` hook
2. Integrates nicely with class inheritance structures (including mixins and
multiple inheritance)
3. Integrates nicely with the implicit ``__class__`` reference and
zero-argument ``super()`` syntax introduced by PEP 3135
4. Can be added to an existing base class without a significant risk of
introducing backwards compatibility problems
One mechanism that can achieve this goal is to add a new class
initialisation hook, modelled directly on the existing instance
initialisation hook, but with the signature constrained to match that
of an ordinary class decorator.
Specifically, it is proposed that class definitions be able to provide a
class initialisation hook as follows::
class Example:
@classmethod
def __init_class__(cls):
# This is invoked after the class is created, but before any
# explicit decorators are called
# The usual super() mechanisms are used to correctly support
# multiple inheritance. The decorator style invocation helps
# ensure that invoking the parent class is as simple as possible.
If present on the created object, this new hook will be called by the class
creation machinery *after* the ``__class__`` reference has been initialised.
For ``types.new_class()``, it will be called as the last step before
returning the created class object.
If a metaclass wishes to block class initialisation for some reason, it
must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
This general proposal is not a new idea (it was first suggested for
inclusion in the language definition `more than 10 years ago`_, and a
similar mechanism has long been supported by `Zope's ExtensionClass`_),
but I believe the situation has changed sufficiently in recent years that
the idea is worth reconsidering.
Key Benefits
============
Replaces many use cases for dynamic setting of ``__metaclass__``
-----------------------------------------------------------------
For use cases that don't involve completely replacing the defined class,
Python 2 code that dynamically set ``__metaclass__`` can now dynamically
set ``__init_class__`` instead. For more advanced use cases, introduction of
an explicit metaclass (possibly made available as a required base class) will
still be necessary in order to support Python 3.
Easier inheritance of definition time behaviour
-----------------------------------------------
Understanding Python's metaclasses requires a deep understanding of
the type system and the class construction process. This is legitimately
seen as challenging, due to the need to keep multiple moving parts (the code,
the metaclass hint, the actual metaclass, the class object, instances of the
class object) clearly distinct in your mind. Even when you know the rules,
it's still easy to make a mistake if you're not being extremely careful.
An earlier version of this PEP actually included such a mistake: it
stated "instance of type" for a constraint that is actually "subclass of
type".
Understanding the proposed class initialisation hook only requires
understanding decorators and ordinary method inheritance, which isn't
quite as daunting a task. The new hook provides a more gradual path
towards understanding all of the phases involved in the class definition
process.
Reduced chance of metaclass conflicts
-------------------------------------
One of the big issues that makes library authors reluctant to use metaclasses
(even when they would be appropriate) is the risk of metaclass conflicts.
These occur whenever two unrelated metaclasses are used by the desired
parents of a class definition. This risk also makes it very difficult to
*add* a metaclass to a class that has previously been published without one.
By contrast, adding an ``__init_class__`` method to an existing type poses
a similar level of risk to adding an ``__init__`` method: technically, there
is a risk of breaking poorly implemented subclasses, but when that occurs,
it is recognised as a bug in the subclass rather than the library author
breaching backwards compatibility guarantees. In fact, due to the constrained
signature of ``__init_class__``, the risk in this case is actually even
lower than in the case of ``__init__``.
Integrates cleanly with \PEP 3135
---------------------------------
Unlike code that runs as part of the metaclass, code that runs as part of
the new hook will be able to freely invoke class methods that rely on the
implicit ``__class__`` reference introduced by PEP 3135, including methods
that use the zero argument form of ``super()``.
Alternatives
============
The Python 3 Status Quo
-----------------------
The Python 3 status quo already offers a great deal of flexibility. For
changes which only affect a single class definition and which can be
specified at the time the code is written, then class decorators can be
used to modify a class explicitly. Class decorators largely ignore class
inheritance and can make full use of methods that rely on the ``__class__``
reference being populated.
Using a custom metaclass provides the same level of power as it did in
Python 2. However, it's notable that, unlike class decorators, a metaclass
cannot call any methods that rely on the ``__class__`` reference, as that
reference is not populated until after the metaclass constructor returns
control to the class creation code.
One major use case for metaclasses actually closely resembles the use of
class decorators. It occurs whenever a metaclass has an implementation that
uses the following pattern::
class Metaclass(type):
def __new__(meta, *args, **kwds):
cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
# Do something with cls
return cls
The key difference between this pattern and a class decorator is that it
is automatically inherited by subclasses. However, it also comes with a
major disadvantage: Python does not allow you to inherit from classes with
unrelated metaclasses.
Thus, the status quo requires that developers choose between the following
two alternatives:
* Use a class decorator, meaning that behaviour is not inherited and must be
requested explicitly on every subclass
* Use a metaclass, meaning that behaviour is inherited, but metaclass
conflicts may make integration with other libraries and frameworks more
difficult than it otherwise would be
If this PEP is ultimately rejected, then this is the existing design that
will remain in place by default.
Restoring the Python 2 metaclass hook
-------------------------------------
One simple alternative would be to restore support for a Python 2 style
``metaclass`` hook in the class body. This would be checked after the class
body was executed, potentially overwriting the metaclass hint provided in the
class header.
The main attraction of such an approach is that it would simplify porting
Python 2 applications that make use of this hook (especially those that do
so dynamically).
However, this approach does nothing to simplify the process of adding
*inherited* class definition time behaviour, nor does it interoperate
cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
any metaclass based solution, the ``__metaclass__`` hook would have to run
before the ``__class__`` reference has been populated.
Dynamic class decorators
------------------------
The original version of this PEP was called "Dynamic class decorators" and
focused solely on a significantly more complicated proposal than that
presented in the current version.
As with the current version, it proposed that a new step be added to the
class creation process, after the metaclass invocation to construct the
class instance and before the application of lexical decorators. However,
instead of a simple process of calling a single class method that relies
on normal inheritance mechanisms, it proposed a far more complicated
procedure that walked the class MRO looking for decorators stored in
iterable ``__decorators__`` attributes.
Using the current version of the PEP, the scheme originally proposed could
be implemented as::
class DynamicDecorators:
@classmethod
def __init_class__(cls):
super(DynamicDecorators, cls).__init_class__()
for entry in reversed(cls.mro()):
decorators = entry.__dict__.get("__decorators__", ())
for deco in reversed(decorators):
cls = deco(cls)
Any subclasses of this type would automatically have the contents of any
``__decorators__`` attributes processed and invoked.
The mechanism in the current PEP is considered superior, as many issues
to do with ordering and the same decorator being invoked multiple times
just go away, as that kind of thing is taken care of through the use of an
ordinary class method invocation.
Automatic metaclass derivation
------------------------------
When no appropriate metaclass is found, it's theoretically possible to
automatically derive a metaclass for a new type based on the metaclass hint
and the metaclasses of the bases.
While adding such a mechanism would reduce the risk of spurious metaclass
conflicts, it would do nothing to improve integration with PEP 3135, would
not help with porting Python 2 code that set ``__metaclass__`` dynamically
and would not provide a more straightforward inherited mechanism for invoking
additional operations after the class invocation is complete.
In addition, there would still be a risk of metaclass conflicts in cases
where the base metaclasses were not written with multiple inheritance in
mind. In such situations, there's a chance of introducing latent defects
if one or more metaclasses are not invoked correctly.
Calling the new hook from ``type.__init__``
-------------------------------------------
Calling the new hook automatically from ``type.__init__``, would achieve most
of the goals of this PEP. However, using that approach would mean that
``__init_class__`` implementations would be unable to call any methods that
relied on the ``__class__`` reference (or used the zero-argument form of
``super()``), and could not make use of those features themselves.
References
==========
.. _published code:
http://mail.python.org/pipermail/python-dev/2012-June/119878.html
.. _more than 10 years ago:
http://mail.python.org/pipermail/python-dev/2001-November/018651.html
.. _Zope's ExtensionClass:
http://docs.zope.org/zope_secrets/extensionclass.html
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: