merged

pep-0362.txt

@@ -2,332 +2,468 @@ PEP: 362
 Title: Function Signature Object
 Version: $Revision$
 Last-Modified: $Date$
-Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>
+Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>,
+        Yury Selivanov <yselivanov@sprymix.com>, Larry Hastings <larry@hastings.org>
 Status: Draft
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 21-Aug-2006
-Python-Version: 2.6
+Python-Version: 3.3
-Post-History: 05-Sep-2007
+Post-History: 04-Jun-2012
 
 
 Abstract
 ========
 
 Python has always supported powerful introspection capabilities,
-including that for functions and methods (for the rest of this PEP the
+including introspecting functions and methods (for the rest of
-word "function" refers to both functions and methods).  Taking a
+this PEP, "function" refers to both functions and methods).  By
-function object, you can fully reconstruct the function's signature.
+examining a function object you can fully reconstruct the function's
-Unfortunately it is a little unruly having to look at all the
+signature.  Unfortunately this information is stored in an inconvenient
-different attributes to pull together complete information for a
+manner, and is spread across a half-dozen deeply nested attributes.
-function's signature.
 
-This PEP proposes an object representation for function signatures.
+This PEP proposes a new representation for function signatures.
-This should help facilitate introspection on functions for various
+The new representation contains all necessary information about a function
-uses.  The introspection information contains all possible information
+and its parameters, and makes introspection easy and straightforward.
-about the parameters in a signature (including Python 3.0 features).
 
-This object, though, is not meant to replace existing ways of
+However, this object does not replace the existing function
-introspection on a function's signature.  The current solutions are
+metadata, which is used by Python itself to execute those
-there to make Python's execution work in an efficient manner.  The
+functions.  The new metadata object is intended solely to make
-proposed object representation is only meant to help make application
+function introspection easier for Python programmers.
-code have an easier time to query a function on its signature.
-
-
-Purpose
-=======
-
-An object representation of a function's call signature should provide
-an easy way to introspect what a function expects as arguments.  It
-does not need to be a "live" representation, though; the signature can
-be inferred once and stored without changes to the signature object
-representation affecting the function it represents (but this is an
-`Open Issues`_).
-
-Indirection of signature introspection can also occur.  If a
-decorator took a decorated function's signature object and set it on
-the decorating function then introspection could be redirected to what
-is actually expected instead of the typical ``*args, **kwargs``
-signature of decorating functions.
 
 
 Signature Object
 ================
 
-The overall signature of an object is represented by the Signature
+A Signature object represents the call signature of a function and
-object.  This object is to store a `Parameter object`_ for each
+its return annotation.  For each parameter accepted by the function
-parameter in the signature.  It is also to store any information
+it stores a `Parameter object`_ in its ``parameters`` collection.
-about the function itself that is pertinent to the signature.
 
-A Signature object has the following structure attributes:
+A Signature object has the following public attributes and methods:
 
-* name : str
-    Name of the function.  This is not fully qualified because
-    function objects for methods do not know the class they are
-    contained within.  This makes functions and methods
-    indistinguishable from one another when passed to decorators,
-    preventing proper creation of a fully qualified name.
-* var_args : str
-    Name of the variable positional parameter (i.e., ``*args``), if
-    present, or the empty string.
-* var_kw_args : str
-    Name of the variable keyword parameter (i.e., ``**kwargs``), if
-    present, or the empty string.
-* var_annotations: dict(str, object)
-    Dict that contains the annotations for the variable parameters.
-    The keys are of the variable parameter with values of the
-    annotation.  If an annotation does not exist for a variable
-    parameter then the key does not exist in the dict.
 * return_annotation : object
-    If present, the attribute is set to the annotation for the return
+    The annotation for the return type of the function if specified.
-    type of the function.
+    If the function has no annotation for its return type, this
-* parameters : list(Parameter)
+    attribute is not set.
-    List of the parameters of the function as represented by
+* parameters : OrderedDict
-    Parameter objects in the order of its definition (keyword-only
+    An ordered mapping of parameters' names to the corresponding
-    arguments are in the order listed by ``code.co_varnames``).
+    Parameter objects (keyword-only arguments are in the same order
-* bind(\*args, \*\*kwargs) -> dict(str, object)
+    as listed in ``code.co_varnames``).
-    Create a mapping from arguments to parameters.  The keys are the
+* bind(\*args, \*\*kwargs) -> BoundArguments
-    names of the parameter that an argument maps to with the value
+    Creates a mapping from positional and keyword arguments to
-    being the value the parameter would have if this function was
+    parameters.  Raises a ``TypeError`` if the passed arguments do
-    called with the given arguments.
+    not match the signature.
+* bind_partial(\*args, \*\*kwargs) -> BoundArguments
+    Works the same way as ``bind()``, but allows the omission
+    of some required arguments (mimics ``functools.partial``
+    behavior.)  Raises a ``TypeError`` if the passed arguments do
+    not match the signature.
+* format(...) -> str
+    Formats the Signature object to a string.  Optional arguments allow
+    for custom render functions for parameter names,
+    annotations and default values, along with custom separators.
 
-Signature objects also have the following methods:
+Signature implements the ``__str__`` method, which fallbacks to the
+``Signature.format()`` call.
 
-* __getitem__(self, key : str) -> Parameter
+It's possible to test Signatures for equality.  Two signatures
-    Returns the Parameter object for the named parameter.
+are equal when they have equal parameters and return annotations.
-* __iter__(self)
-    Returns an iterator that returns Parameter objects in their
-    sequential order based on their 'position' attribute.
 
-The Signature object is stored in the ``__signature__`` attribute of
+Changes to the Signature object, or to any of its data members,
-a function.  When it is to be created is discussed in
+do not affect the function itself.
-`Open Issues`_.
 
 
 Parameter Object
 ================
 
-A function's signature is made up of several parameters.  Python's
+Python's expressive syntax means functions can accept many different
-different kinds of parameters is quite large and rich and continues to
+kinds of parameters with many subtle semantic differences.  We
-grow.  Parameter objects represent any possible parameter.
+propose a rich Parameter object designed to represent any possible
-
+function parameter.
-Originally the plan was to represent parameters using a list of
-parameter names on the Signature object along with various dicts keyed
-on parameter names to disseminate the various pieces of information
-one can know about a parameter.  But the decision was made to
-incorporate all information about a parameter in a single object so
-as to make extending the information easier.  This was originally put
-forth by Talin and the preferred form of Guido (as discussed at the
-2006 Google Sprint).
 
 The structure of the Parameter object is:
 
-* name : (str | tuple(str))
+* name : str
-    The name of the parameter as a string if it is not a tuple.  If
+    The name of the parameter as a string.
-    the argument is a tuple then a tuple of strings is used.
+
-* position : int
+* default : object
-    The position of the parameter within the signature of the
+    The default value for the parameter, if specified.  If the
-    function (zero-indexed).  For keyword-only parameters the position
+    parameter has no default value, this attribute is not set.
-    value is arbitrary while not conflicting with positional
+
-    parameters.  The suggestion of setting the attribute to None or -1
+* annotation : object
-    to represent keyword-only parameters was rejected to prevent
+    The annotation for the parameter if specified.  If the
-    variable type usage and as a possible point of errors,
+    parameter has no annotation, this attribute is not set.
-    respectively.
+
-* default_value : object
+* kind : str
-    The default value for the parameter, if present, else the
+    Describes how argument values are bound to the parameter.
-    attribute does not exist.
+    Possible values:
-* keyword_only : bool
+
-    True if the parameter is keyword-only, else False.
+       * ``Parameter.POSITIONAL_ONLY`` - value must be supplied
-* annotation
+         as a positional argument.
-    Set to the annotation for the parameter.  If ``has_annotation`` is
+
-    False then the attribute does not exist to prevent accidental use.
+         Python has no explicit syntax for defining positional-only
+         parameters, but many builtin and extension module functions
+         (especially those that accept only one or two parameters)
+         accept them.
+
+       * ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
+         supplied as either a keyword or positional argument
+         (this is the standard binding behaviour for functions
+         implemented in Python.)
+
+       * ``Parameter.KEYWORD_ONLY`` - value must be supplied
+         as a keyword argument.  Keyword only parameters are those
+         which appear after a "*" or "\*args" entry in a Python
+         function definition.
+
+       * ``Parameter.VAR_POSITIONAL`` - a tuple of positional
+         arguments that aren't bound to any other parameter.
+         This corresponds to a "\*args" parameter in a Python
+         function definition.
+
+       * ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
+         that aren't bound to any other parameter. This corresponds
+         to a "\*\*kwds" parameter in a Python function definition.
+
+* implemented : bool
+    True if the parameter is implemented for use.  Some platforms
+    implement functions but can't support specific parameters
+    (e.g. "mode" for ``os.mkdir``).  Passing in an unimplemented
+    parameter may result in the parameter being ignored,
+    or in NotImplementedError being raised.  It is intended that
+    all conditions where ``implemented`` may be False be
+    thoroughly documented.
+
+Two parameters are equal when all their attributes are equal.
+
+
+BoundArguments Object
+=====================
+
+Result of a ``Signature.bind`` call.  Holds the mapping of arguments
+to the function's parameters.
+
+Has the following public attributes:
+
+* arguments : OrderedDict
+    An ordered, mutable mapping of parameters' names to arguments' values.
+    Does not contain arguments' default values.
+* args : tuple
+    Tuple of positional arguments values.  Dynamically computed from
+    the 'arguments' attribute.
+* kwargs : dict
+    Dict of keyword arguments values. Dynamically computed from
+    the 'arguments' attribute.
+
+The ``arguments`` attribute should be used in conjunction with
+``Signature.parameters`` for any arguments processing purposes.
+
+``args`` and ``kwargs`` properties can be used to invoke functions:
+::
+
+    def test(a, *, b):
+        ...
+
+    sig = signature(test)
+    ba = sig.bind(10, b=20)
+    test(*ba.args, **ba.kwargs)
 
 
 Implementation
 ==============
 
-An implementation can be found in Python's sandbox [#impl]_.
+The implementation adds a new function ``signature()`` to the ``inspect``
-There is a function named ``signature()`` which
+module.  The function is the preferred way of getting a ``Signature`` for
-returns the value stored on the ``__signature__`` attribute if it
+a callable object.
-exists, else it creates the Signature object for the
+
-function and sets ``__signature__``.  For methods this is stored
+The function implements the following algorithm:
-directly on the im_func function object since that is what decorators
+
-work with.
+    - If the object is not callable - raise a TypeError
+
+    - If the object has a ``__signature__`` attribute and if it
+      is not ``None`` - return a deepcopy of it
+
+        - If it is ``None`` and the object is an instance of
+          ``BuiltinFunction``, raise a ``ValueError``
+
+    - If it has a ``__wrapped__`` attribute, return
+      ``signature(object.__wrapped__)``
+
+    - If the object is a an instance of ``FunctionType`` construct
+      and return a new ``Signature`` for it
+
+    - If the object is a method or a classmethod, construct and return
+      a new ``Signature`` object, with its first parameter (usually
+      ``self`` or ``cls``) removed
+
+    - If the object is a staticmethod, construct and return
+      a new ``Signature`` object
+
+    - If the object is an instance of ``functools.partial``, construct
+      a new ``Signature`` from its ``partial.func`` attribute, and
+      account for already bound ``partial.args`` and ``partial.kwargs``
+
+    - If the object is a class or metaclass:
+
+        - If the object's type has a ``__call__`` method defined in
+          its MRO, return a Signature for it
+
+        - If the object has a ``__new__`` method defined in its class,
+          return a Signature object for it
+
+        - If the object has a ``__init__`` method defined in its class,
+          return a Signature object for it
+
+    - Return ``signature(object.__call__)``
+
+Note, that the ``Signature`` object is created in a lazy manner, and
+is not automatically cached.  If, however, the Signature object was
+explicitly cached by the user, ``signature()`` returns a new deepcopy
+of it on each invocation.
+
+An implementation for Python 3.3 can be found at [#impl]_.
+The python issue tracking the patch is [#issue]_.
+
+
+Design Considerations
+=====================
+
+No implicit caching of Signature objects
+----------------------------------------
+
+The first PEP design had a provision for implicit caching of ``Signature``
+objects in the ``inspect.signature()`` function.  However, this has the
+following downsides:
+
+  * If the ``Signature`` object is cached then any changes to the function
+    it describes will not be reflected in it.  However, If the caching is
+    needed, it can be always done manually and explicitly
+
+  * It is better to reserve the ``__signature__`` attribute for the cases
+    when there is a need to explicitly set to a ``Signature`` object that
+    is different from the actual one
 
 
 Examples
 ========
 
+Visualizing Callable Objects' Signature
+---------------------------------------
+
+Let's define some classes and functions:
+
+::
+
+    from inspect import signature
+    from functools import partial, wraps
+
+
+    class FooMeta(type):
+        def __new__(mcls, name, bases, dct, *, bar:bool=False):
+            return super().__new__(mcls, name, bases, dct)
+
+        def __init__(cls, name, bases, dct, **kwargs):
+            return super().__init__(name, bases, dct)
+
+
+    class Foo(metaclass=FooMeta):
+        def __init__(self, spam:int=42):
+            self.spam = spam
+
+        def __call__(self, a, b, *, c) -> tuple:
+            return a, b, c
+
+
+    def shared_vars(*shared_args):
+        """Decorator factory that defines shared variables that are
+           passed to every invocation of the function"""
+
+        def decorator(f):
+            @wraps(f)
+            def wrapper(*args, **kwds):
+                full_args = shared_args + args
+                return f(*full_args, **kwds)
+            # Override signature
+            sig = wrapper.__signature__ = signature(f)
+            for __ in shared_args:
+                sig.parameters.popitem(last=False)
+            return wrapper
+        return decorator
+
+
+    @shared_vars({})
+    def example(_state, a, b, c):
+        return _state, a, b, c
+
+
+    def format_signature(obj):
+        return str(signature(obj))
+
+
+Now, in the python REPL:
+
+::
+
+    >>> format_signature(FooMeta)
+    '(name, bases, dct, *, bar:bool=False)'
+
+    >>> format_signature(Foo)
+    '(spam:int=42)'
+
+    >>> format_signature(Foo.__call__)
+    '(self, a, b, *, c) -> tuple'
+
+    >>> format_signature(Foo().__call__)
+    '(a, b, *, c) -> tuple'
+
+    >>> format_signature(partial(Foo().__call__, 1, c=3))
+    '(b, *, c=3) -> tuple'
+
+    >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
+    '(*, c=20) -> tuple'
+
+    >>> format_signature(example)
+    '(a, b, c)'
+
+    >>> format_signature(partial(example, 1, 2))
+    '(c)'
+
+    >>> format_signature(partial(partial(example, 1, b=2), c=3))
+    '(b=2, c=3)'
+
+
 Annotation Checker
 ------------------
 ::
 
-    def quack_check(fxn):
+    import inspect
-        """Decorator to verify arguments and return value quack as they should.
+    import functools
 
-        Positional arguments.
+    def checktypes(func):
-        >>> @quack_check
+        '''Decorator to verify arguments and return types
-        ... def one_arg(x:int): pass
-        ... 
-        >>> one_arg(42)
-        >>> one_arg('a')
-        Traceback (most recent call last):
-            ...
-        TypeError: 'a' does not quack like a <type 'int'>
 
+        Example:
 
-        *args
+            >>> @checktypes
-        >>> @quack_check
+            ... def test(a:int, b:str) -> int:
-        ... def var_args(*args:int): pass
+            ...     return int(a * b)
-        ... 
-        >>> var_args(*[1,2,3])
-        >>> var_args(*[1,'b',3])
-        Traceback (most recent call last):
-            ...
-        TypeError: *args contains a a value that does not quack like a <type 'int'>
 
-        **kwargs
+            >>> test(10, '1')
-        >>> @quack_check
+            1111111111
-        ... def var_kw_args(**kwargs:int): pass
-        ... 
-        >>> var_kw_args(**{'a': 1})
-        >>> var_kw_args(**{'a': 'A'})
-        Traceback (most recent call last):
-            ...
-        TypeError: **kwargs contains a value that does not quack like a <type 'int'>
 
-        Return annotations.
+            >>> test(10, 1)
-        >>> @quack_check
+            Traceback (most recent call last):
-        ... def returned(x) -> int: return x
+              ...
-        ... 
+            ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
-        >>> returned(42)
+        '''
-        42
-        >>> returned('a')
-        Traceback (most recent call last):
-            ...
-        TypeError: the return value 'a' does not quack like a <type 'int'>
 
-        """
+        sig = inspect.signature(func)
-        # Get the signature; only needs to be calculated once.
+
-        sig = Signature(fxn)
+        types = {}
-        def check(*args, **kwargs):
+        for param in sig.parameters.values():
-            # Find out the variable -> value bindings.
+            # Iterate through function's parameters and build the list of
-            bindings = sig.bind(*args, **kwargs)
+            # arguments types
-            # Check *args for the proper quack.
             try:
-                duck = sig.var_annotations[sig.var_args]
+                type_ = param.annotation
-            except KeyError:
+            except AttributeError:
-                pass
+                continue
             else:
-                # Check every value in *args.
+                if not inspect.isclass(type_):
-                for value in bindings[sig.var_args]:
+                    # Not a type, skip it
-                    if not isinstance(value, duck):
+                    continue
-                        raise TypeError("*%s contains a a value that does not "
+
-                                        "quack like a %r" %
+                types[param.name] = type_
-                                        (sig.var_args, duck))
+
-                # Remove it from the bindings so as to not check it again.
+                # If the argument has a type specified, let's check that its
-                del bindings[sig.var_args]
+                # default value (if present) conforms with the type.
-            # **kwargs.
-            try:
-                duck = sig.var_annotations[sig.var_kw_args]
-            except (KeyError, AttributeError):
-                pass
-            else:
-                # Check every value in **kwargs.
-                for value in bindings[sig.var_kw_args].values():
-                    if not isinstance(value, duck):
-                        raise TypeError("**%s contains a value that does not "
-                                        "quack like a %r" %
-                                        (sig.var_kw_args, duck))
-                # Remove from bindings so as to not check again.
-                del bindings[sig.var_kw_args]
-            # For each remaining variable ...
-            for var, value in bindings.items():
-                # See if an annotation was set.
                 try:
-                    duck = sig[var].annotation
+                    default = param.default
                 except AttributeError:
                     continue
-                # Check that the value quacks like it should.
+                else:
-                if not isinstance(value, duck):
+                    if not isinstance(default, type_):
-                    raise TypeError('%r does not quack like a %s' % (value, duck))
+                        raise ValueError("{func}: wrong type of a default value for {arg!r}". \
-            else:
+                                         format(func=func.__qualname__, arg=param.name))
-                # All the ducks quack fine; let the call proceed.
+
-                returned = fxn(*args, **kwargs)
+        def check_type(sig, arg_name, arg_type, arg_value):
-                # Check the return value.
+            # Internal function that encapsulates arguments type checking
+            if not isinstance(arg_value, arg_type):
+                raise ValueError("{func}: wrong type of {arg!r} argument, " \
+                                 "{exp!r} expected, got {got!r}". \
+                                 format(func=func.__qualname__, arg=arg_name,
+                                        exp=arg_type.__name__, got=type(arg_value).__name__))
+
+        @functools.wraps(func)
+        def wrapper(*args, **kwargs):
+            # Let's bind the arguments
+            ba = sig.bind(*args, **kwargs)
+            for arg_name, arg in ba.arguments.items():
+                # And iterate through the bound arguments
                 try:
-                    if not isinstance(returned, sig.return_annotation):
+                    type_ = types[arg_name]
-                        raise TypeError('the return value %r does not quack like '
+                except KeyError:
-                                        'a %r' % (returned,
+                    continue
-                                            sig.return_annotation))
+                else:
-                except AttributeError:
+                    # OK, we have a type for the argument, lets get the corresponding
-                    pass
+                    # parameter description from the signature object
-                return returned
+                    param = sig.parameters[arg_name]
-        # Full-featured version would set function metadata.
+                    if param.kind == param.VAR_POSITIONAL:
-        return check
+                        # If this parameter is a variable-argument parameter,
+                        # then we need to check each of its values
+                        for value in arg:
+                            check_type(sig, arg_name, type_, value)
+                    elif param.kind == param.VAR_KEYWORD:
+                        # If this parameter is a variable-keyword-argument parameter:
+                        for subname, value in arg.items():
+                            check_type(sig, arg_name + ':' + subname, type_, value)
+                    else:
+                        # And, finally, if this parameter a regular one:
+                        check_type(sig, arg_name, type_, arg)
+
+            result = func(*ba.args, **ba.kwargs)
+            # The last bit - let's check that the result is correct
+            try:
+                return_type = sig.return_annotation
+            except AttributeError:
+                # Looks like we don't have any restriction on the return type
+                pass
+            else:
+                if isinstance(return_type, type) and not isinstance(result, return_type):
+                    raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
+                                     format(func=func.__qualname__, exp=return_type.__name__,
+                                            got=type(result).__name__))
+            return result
+
+        return wrapper
 
 
-Open Issues
+Render Function Signature to HTML
-===========
+---------------------------------
 
-When to construct the Signature object?
+::
----------------------------------------
 
-The Signature object can either be created in an eager or lazy
+    import inspect
-fashion.  In the eager situation, the object can be created during
-creation of the function object.  In the lazy situation, one would
-pass a function object to a function and that would generate the
-Signature object and store it to ``__signature__`` if
-needed, and then return the value of ``__signature__``.
 
+    def format_to_html(func):
+        sig = inspect.signature(func)
 
-Should ``Signature.bind`` return Parameter objects as keys?
+        html = sig.format(token_params_separator='<span class="t-comma">,</span>',
------------------------------------------------------------
+                          token_colon='<span class="t-colon">:</span>',
+                          token_eq='<span class="t-eq">=</span>',
+                          token_return_annotation='<span class="t-ra">-&gt;</span>',
+                          token_left_paren='<span class="t-lp">(</span>',
+                          token_right_paren='<span class="t-lp">)</span>',
+                          token_kwonly_separator='<span class="t-ast">*</span>',
+                          format_name=lambda name: '<span class="name">'+name+'</span>')
 
-Instead of returning a dict with keys consisting of the name of the
+        return '<span class="py-func">{}</span>'.format(html)
-parameters, would it be more useful to instead use Parameter
-objects?  The name of the argument can easily be retrieved from the
-key (and the name would be used as the hash for a Parameter object).
-
-
-Have ``var_args`` and ``_var_kw_args`` default to ``None``?
-------------------------------------------------------------
-
-It has been suggested by Fred Drake that these two attributes have a
-value of ``None`` instead of empty strings when they do not exist.
-The answer to this question will influence what the defaults are for
-other attributes as well.
-
-
-Deprecate ``inspect.getargspec()`` and ``.formatargspec()``?
--------------------------------------------------------------
-
-Since the Signature object replicates the use of ``getargspec()``
-from the ``inspect`` module it might make sense to deprecate it in
-2.6.  ``formatargspec()`` could also go if Signature objects gained a
-__str__ representation.
-
-Issue with that is types such as ``int``, when used as annotations,
-do not lend themselves for output (e.g., ``"<type 'int'>"`` is the
-string represenation for ``int``).  The repr representation of types
-would need to change in order to make this reasonable.
-
-
-Have the objects be "live"?
----------------------------
-
-Jim Jewett pointed out that Signature and Parameter objects could be
-"live".  That would mean requesting information would be done on the
-fly instead of caching it on the objects.  It would also allow for
-mutating the function if the Signature or Parameter objects were
-mutated.
 
 
 References
 ==========
 
-.. [#impl] pep362 directory in Python's sandbox
+.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview)
-   (http://svn.python.org/view/sandbox/trunk/pep362/)
+.. [#issue] issue 15008 (http://bugs.python.org/issue15008)
 
 
 Copyright
@@ -335,7 +471,6 @@ Copyright
 
 This document has been placed in the public domain.
 
-
 
 ..
    Local Variables:

pep-0398.txt

@@ -70,6 +70,7 @@ Implemented / Final PEPs:
 * PEP 417: Including mock in the Standard Library
 * PEP 418: Add monotonic time, performance counter, and process time functions
 * PEP 420: Implicit Namespace Packages
+* PEP 421: Adding sys.implementation
 * PEP 3118: Revising the buffer protocol (protocol semantics finalised)
 * PEP 3144: IP Address manipulation library
 * PEP 3151: Reworking the OS and IO exception hierarchy
@@ -87,8 +88,6 @@ Candidate PEPs:
 
 * PEP 362: Function Signature Object
 * PEP 397: Python launcher for Windows
-* PEP 421: Adding sys.implementation
-* PEP 3143: Standard daemon process library
 * PEP 3154: Pickle protocol version 4
 
 (Note that these are not accepted yet and even if they are, they might
@@ -105,6 +104,7 @@ Other planned large-scale changes:
 Deferred to post-3.3:
 
 * PEP 395: Qualified Names for Modules
+* PEP 3143: Standard daemon process library
 * Breaking out standard library and docs in separate repos
 
 Copyright

pep-0405.txt

@@ -4,7 +4,7 @@ Version: $Revision$
 Last-Modified: $Date$
 Author: Carl Meyer <carl@oddbird.net>
 BDFL-Delegate: Nick Coghlan
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 13-Jun-2011
@@ -285,15 +285,15 @@ Include files
 
 Current virtualenv handles include files in this way:
 
-On POSIX systems where the installed Python's include files are found
+On POSIX systems where the installed Python's include files are found in
-in ``${base_prefix}/include/pythonX.X``, virtualenv creates
+``${base_prefix}/include/pythonX.X``, virtualenv creates
-``${venv}/include/`` and symlink ``${base_prefix}/include/pythonX.X``
+``${venv}/include/`` and symlinks ``${base_prefix}/include/pythonX.X``
 to ``${venv}/include/pythonX.X``. On Windows, where Python's include
 files are found in ``{{ sys.prefix }}/Include`` and symlinks are not
 reliably available, virtualenv copies ``{{ sys.prefix }}/Include`` to
 ``${venv}/Include``. This ensures that extension modules built and
-installed within the virtualenv will always find the Python header
+installed within the virtualenv will always find the Python header files
-files they need in the expected location relative to ``sys.prefix``.
+they need in the expected location relative to ``sys.prefix``.
 
 This solution is not ideal when an extension module installs its own
 header files, as the default installation location for those header
@@ -467,10 +467,10 @@ than ``sys.site_prefix`` or the appropriate ``site`` API to find
 site-packages directories.
 
 The most notable case is probably `setuptools`_ and its fork
-`distribute`_, which mostly use ``distutils``and ``sysconfig`` APIs,
+`distribute`_, which mostly use ``distutils`` and ``sysconfig`` APIs,
 but do use ``sys.prefix`` directly to build up a list of site
-directories for pre-flight checking where ``pth`` files can usefully
+directories for pre-flight checking where ``pth`` files can usefully be
-be placed.
+placed.
 
 Otherwise, a `Google Code Search`_ turns up what appears to be a
 roughly even mix of usage between packages using ``sys.prefix`` to

pep-0420.txt

@@ -3,7 +3,7 @@ Title: Implicit Namespace Packages
 Version: $Revision$
 Last-Modified: $Date$
 Author: Eric V. Smith <eric@trueblade.com>
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 19-Apr-2012

pep-0421.txt

@@ -4,7 +4,7 @@ Version: $Revision$
 Last-Modified: $Date$
 Author: Eric Snow <ericsnowcurrently@gmail.com>
 BDFL-Delegate: Barry Warsaw
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 26-April-2012

pep-0422.txt

@@ -0,0 +1,354 @@
+PEP: 422
+Title: Simple class initialisation hook
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan@gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 5-Jun-2012
+Python-Version: 3.4
+Post-History: 5-Jun-2012
+
+
+Abstract
+========
+
+In Python 2, the body of a class definition could modify the way a class
+was created (or simply arrange to run other code after the class was created)
+by setting the ``__metaclass__`` attribute in the class body. While doing
+this implicitly from called code required the use of an implementation detail
+(specifically, ``sys._getframes()``), it could also be done explicitly in a
+fully supported fashion (for example, by passing ``locals()`` to an
+function that calculated a suitable ``__metaclass__`` value)
+
+There is currently no corresponding mechanism in Python 3 that allows the
+code executed in the class body to directly influence how the class object
+is created. Instead, the class creation process is fully defined by the
+class header, before the class body even begins executing.
+
+This PEP proposes a mechanism that will once again allow the body of a
+class definition to more directly influence the way a class is created
+(albeit in a more constrained fashion), as well as replacing some current
+uses of metaclasses with a simpler, easier to understand alternative.
+
+
+Background
+==========
+
+For an already created class ``cls``, the term "metaclass" has a clear
+meaning: it is the value of ``type(cls)``.
+
+*During* class creation, it has another meaning: it is also used to refer to
+the metaclass hint that may be provided as part of the class definition.
+While in many cases these two meanings end up referring to one and the same
+object, there are two situations where that is not the case:
+
+* If the metaclass hint refers to a subclass of ``type``, then it is
+  considered as a candidate metaclass along with the metaclasses of all of
+  the parents of the class being defined. If a more appropriate metaclass is
+  found amongst the candidates, then it will be used instead of the one
+  given in the metaclass hint.
+* Otherwise, an explicit metaclass hint is assumed to be a factory function
+  and is called directly to create the class object. In this case, the final
+  metaclass will be determined by the factory function definition. In the
+  typical case (where the factory functions just calls ``type``, or, in
+  Python 3.3 or later, ``types.new_class``) the actual metaclass is then
+  determined based on the parent classes.
+
+It is notable that only the actual metaclass is inherited - a factory
+function used as a metaclass hook sees only the class currently being
+defined, and is not invoked for any subclasses.
+
+In Python 3, the metaclass hint is provided using the ``metaclass=Meta``
+keyword syntax in the class header. This allows the ``__prepare__`` method
+on the metaclass to be used to create the ``locals()`` namespace used during
+execution of the class body (for example, specifying the use of
+``collections.OrderedDict`` instead of a regular ``dict``).
+
+In Python 2, there was no ``__prepare__`` method (that API was added for
+Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__``
+attribute, and the class creation process would extract that value from the
+class namespace to use as the metaclass hint. There is `published code`_ that
+makes use of this feature.
+
+Another new feature in Python 3 is the zero-argument form of the ``super()``
+builtin, introduced by PEP 3135. This feature uses an implicit ``__class__``
+reference to the class being defined to replace the "by name" references
+required in Python 2. Just as code invoked during execution of a Python 2
+metaclass could not call methods that referenced the class by name (as the
+name had not yet been bound in the containing scope), similarly, Python 3
+metaclasses cannot call methods that rely on the implicit ``__class__``
+reference (as it is not populated until after the metaclass has returned
+control to the class creation machiner).
+
+
+Proposal
+========
+
+This PEP proposes that a mechanism be added to Python 3 that meets the
+following criteria:
+
+1. Restores the ability for class namespaces to have some influence on the
+   class creation process (above and beyond populating the namespace itself),
+   but potentially without the full flexibility of the Python 2 style
+   ``__metaclass__`` hook
+2. Integrates nicely with class inheritance structures (including mixins and
+   multiple inheritance)
+3. Integrates nicely with the implicit ``__class__`` reference and
+   zero-argument ``super()`` syntax introduced by PEP 3135
+4. Can be added to an existing base class without a significant risk of
+   introducing backwards compatibility problems
+
+One mechanism that can achieve this goal is to add a new class
+initialisation hook, modelled directly on the existing instance
+initialisation hook, but with the signature constrained to match that
+of an ordinary class decorator.
+
+Specifically, it is proposed that class definitions be able to provide a
+class initialisation hook as follows::
+
+   class Example:
+       @classmethod
+       def __init_class__(cls):
+           # This is invoked after the class is created, but before any
+           # explicit decorators are called
+           # The usual super() mechanisms are used to correctly support
+           # multiple inheritance. The decorator style invocation helps
+           # ensure that invoking the parent class is as simple as possible.
+
+If present on the created object, this new hook will be called by the class
+creation machinery *after* the ``__class__`` reference has been initialised.
+For ``types.new_class()``, it will be called as the last step before
+returning the created class object.
+
+If a metaclass wishes to block class initialisation for some reason, it
+must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
+
+This general proposal is not a new idea (it was first suggested for
+inclusion in the language definition `more than 10 years ago`_, and a
+similar mechanism has long been supported by `Zope's ExtensionClass`_),
+but I believe the situation has changed sufficiently in recent years that
+the idea is worth reconsidering.
+
+
+Key Benefits
+============
+
+
+Replaces many use cases for dynamic setting of ``__metaclass__``
+-----------------------------------------------------------------
+
+For use cases that don't involve completely replacing the defined class,
+Python 2 code that dynamically set ``__metaclass__`` can now dynamically
+set ``__init_class__`` instead. For more advanced use cases, introduction of
+an explicit metaclass (possibly made available as a required base class) will
+still be necessary in order to support Python 3.
+
+
+Easier inheritance of definition time behaviour
+-----------------------------------------------
+
+Understanding Python's metaclasses requires a deep understanding of
+the type system and the class construction process. This is legitimately
+seen as challenging, due to the need to keep multiple moving parts (the code,
+the metaclass hint, the actual metaclass, the class object, instances of the
+class object) clearly distinct in your mind. Even when you know the rules,
+it's still easy to make a mistake if you're not being extremely careful.
+An earlier version of this PEP actually included such a mistake: it
+stated "instance of type" for a constraint that is actually "subclass of
+type".
+
+Understanding the proposed class initialisation hook only requires
+understanding decorators and ordinary method inheritance, which isn't
+quite as daunting a task. The new hook provides a more gradual path
+towards understanding all of the phases involved in the class definition
+process.
+
+
+Reduced chance of metaclass conflicts
+-------------------------------------
+
+One of the big issues that makes library authors reluctant to use metaclasses
+(even when they would be appropriate) is the risk of metaclass conflicts.
+These occur whenever two unrelated metaclasses are used by the desired
+parents of a class definition. This risk also makes it very difficult to
+*add* a metaclass to a class that has previously been published without one.
+
+By contrast, adding an ``__init_class__`` method to an existing type poses
+a similar level of risk to adding an ``__init__`` method: technically, there
+is a risk of breaking poorly implemented subclasses, but when that occurs,
+it is recognised as a bug in the subclass rather than the library author
+breaching backwards compatibility guarantees. In fact, due to the constrained
+signature of ``__init_class__``, the risk in this case is actually even
+lower than in the case of ``__init__``.
+
+
+Integrates cleanly with \PEP 3135
+---------------------------------
+
+Unlike code that runs as part of the metaclass, code that runs as part of
+the new hook will be able to freely invoke class methods that rely on the
+implicit ``__class__`` reference introduced by PEP 3135, including methods
+that use the zero argument form of ``super()``.
+
+
+Alternatives
+============
+
+
+The Python 3 Status Quo
+-----------------------
+
+The Python 3 status quo already offers a great deal of flexibility. For
+changes which only affect a single class definition and which can be
+specified at the time the code is written, then class decorators can be
+used to modify a class explicitly. Class decorators largely ignore class
+inheritance and can make full use of methods that rely on the ``__class__``
+reference being populated.
+
+Using a custom metaclass provides the same level of power as it did in
+Python 2. However, it's notable that, unlike class decorators, a metaclass
+cannot call any methods that rely on the ``__class__`` reference, as that
+reference is not populated until after the metaclass constructor returns
+control to the class creation code.
+
+One major use case for metaclasses actually closely resembles the use of
+class decorators. It occurs whenever a metaclass has an implementation that
+uses the following pattern::
+
+    class Metaclass(type):
+        def __new__(meta, *args, **kwds):
+            cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
+            # Do something with cls
+            return cls
+
+The key difference between this pattern and a class decorator is that it
+is automatically inherited by subclasses. However, it also comes with a
+major disadvantage: Python does not allow you to inherit from classes with
+unrelated metaclasses.
+
+Thus, the status quo requires that developers choose between the following
+two alternatives:
+
+* Use a class decorator, meaning that behaviour is not inherited and must be
+  requested explicitly on every subclass
+* Use a metaclass, meaning that behaviour is inherited, but metaclass
+  conflicts may make integration with other libraries and frameworks more
+  difficult than it otherwise would be
+
+If this PEP is ultimately rejected, then this is the existing design that
+will remain in place by default.
+
+
+Restoring the Python 2 metaclass hook
+-------------------------------------
+
+One simple alternative would be to restore support for a Python 2 style
+``metaclass`` hook in the class body. This would be checked after the class
+body was executed, potentially overwriting the metaclass hint provided in the
+class header.
+
+The main attraction of such an approach is that it would simplify porting
+Python 2 applications that make use of this hook (especially those that do
+so dynamically).
+
+However, this approach does nothing to simplify the process of adding
+*inherited* class definition time behaviour, nor does it interoperate
+cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
+any metaclass based solution, the ``__metaclass__`` hook would have to run
+before the ``__class__`` reference has been populated.
+
+
+Dynamic class decorators
+------------------------
+
+The original version of this PEP was called "Dynamic class decorators" and
+focused solely on a significantly more complicated proposal than that
+presented in the current version.
+
+As with the current version, it proposed that a new step be added to the
+class creation process, after the metaclass invocation to construct the
+class instance and before the application of lexical decorators. However,
+instead of a simple process of calling a single class method that relies
+on normal inheritance mechanisms, it proposed a far more complicated
+procedure that walked the class MRO looking for decorators stored in
+iterable ``__decorators__`` attributes.
+
+Using the current version of the PEP, the scheme originally proposed could
+be implemented as::
+
+   class DynamicDecorators:
+       @classmethod
+       def __init_class__(cls):
+           super(DynamicDecorators, cls).__init_class__()
+           for entry in reversed(cls.mro()):
+               decorators = entry.__dict__.get("__decorators__", ())
+               for deco in reversed(decorators):
+                   cls = deco(cls)
+
+Any subclasses of this type would automatically have the contents of any
+``__decorators__`` attributes processed and invoked.
+
+The mechanism in the current PEP is considered superior, as many issues
+to do with ordering and the same decorator being invoked multiple times
+just go away, as that kind of thing is taken care of through the use of an
+ordinary class method invocation.
+
+
+Automatic metaclass derivation
+------------------------------
+
+When no appropriate metaclass is found, it's theoretically possible to
+automatically derive a metaclass for a new type based on the metaclass hint
+and the metaclasses of the bases.
+
+While adding such a mechanism would reduce the risk of spurious metaclass
+conflicts, it would do nothing to improve integration with PEP 3135, would
+not help with porting Python 2 code that set ``__metaclass__`` dynamically
+and would not provide a more straightforward inherited mechanism for invoking
+additional operations after the class invocation is complete.
+
+In addition, there would still be a risk of metaclass conflicts in cases
+where the base metaclasses were not written with multiple inheritance in
+mind. In such situations, there's a chance of introducing latent defects
+if one or more metaclasses are not invoked correctly.
+
+
+Calling the new hook from ``type.__init__``
+-------------------------------------------
+
+Calling the new hook automatically from ``type.__init__``, would achieve most
+of the goals of this PEP. However, using that approach would mean that
+``__init_class__`` implementations would be unable to call any methods that
+relied on the ``__class__`` reference (or used the zero-argument form of
+``super()``), and could not make use of those features themselves.
+
+
+References
+==========
+
+.. _published code:
+   http://mail.python.org/pipermail/python-dev/2012-June/119878.html
+
+.. _more than 10 years ago:
+   http://mail.python.org/pipermail/python-dev/2001-November/018651.html
+
+.. _Zope's ExtensionClass:
+   http://docs.zope.org/zope_secrets/extensionclass.html
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End:
+