Update PEP 575 (#635)

This commit is contained in:
jdemeyer 2018-04-27 14:25:02 +02:00 committed by Chris Angelico
parent 499a7c25c6
commit 4d9827666e
1 changed files with 155 additions and 105 deletions

View File

@ -6,7 +6,7 @@ Type: Standards Track
Content-Type: text/x-rst Content-Type: text/x-rst
Created: 27-Mar-2018 Created: 27-Mar-2018
Python-Version: 3.8 Python-Version: 3.8
Post-History: 31-Mar-2018, 12-Apr-2018 Post-History: 31-Mar-2018, 12-Apr-2018, 27-Apr-2018
Abstract Abstract
@ -21,7 +21,8 @@ without sacrificing performance.
A new base class ``base_function`` is introduced and the various function A new base class ``base_function`` is introduced and the various function
classes, as well as ``method`` (renamed to ``bound_method``), inherit from it. classes, as well as ``method`` (renamed to ``bound_method``), inherit from it.
We also allow subclassing of some of these function classes. We also allow subclassing in some cases:
in particular the Python ``function`` class can be subclassed.
Motivation Motivation
========== ==========
@ -32,7 +33,7 @@ when defining a function with ``def`` or ``lambda``.
The second is built-in functions such as ``len``, ``isinstance`` or ``numpy.dot``. The second is built-in functions such as ``len``, ``isinstance`` or ``numpy.dot``.
These are implemented in C. These are implemented in C.
These two classes are completely independent with different functionality. These two classes are implemented completely independently and have different functionality.
In particular, it is currently not possible to implement a function efficiently in C In particular, it is currently not possible to implement a function efficiently in C
(only built-in functions can do that) (only built-in functions can do that)
while still allowing introspection like ``inspect.signature`` or ``inspect.getsourcefile`` while still allowing introspection like ``inspect.signature`` or ``inspect.getsourcefile``
@ -52,6 +53,14 @@ unbound methods of extension types (e.g. ``dict.get``) are a distinct class.
Bound methods of Python classes have a ``__func__`` attribute, Bound methods of Python classes have a ``__func__`` attribute,
bound methods of extension types do not. bound methods of extension types do not.
Third, this PEP allows great customization of functions.
The ``function`` class becomes subclassable and custom function
subclasses are also allowed for functions implemented in C.
In the latter case, this can be done with the same performance
as true built-in functions.
All functions can access the function object
(the ``self`` in ``__call__``), paving the way for PEP 573.
New classes New classes
=========== ===========
@ -80,54 +89,58 @@ base_function
------------- -------------
The class ``base_function`` becomes a new base class for all function types. The class ``base_function`` becomes a new base class for all function types.
It behaves like the existing ``builtin_function_or_method`` It is based on the existing ``builtin_function_or_method`` class,
with some differences: but with the following differences and new features:
#. It acts as a descriptor implementing ``__get__`` to turn a function into a method #. It acts as a descriptor implementing ``__get__`` to turn a function into a method
if ``m_self`` is ``NULL``. if ``m_self`` is ``NULL``.
If ``m_self`` is not ``NULL``, If ``m_self`` is not ``NULL``,
then this is a no-op: the existing function is returned instead. then this is a no-op: the existing function is returned instead.
#. A new read-only slot ``__objclass__``, represented in the C structure as ``m_objclass``. #. A new read-only attribute ``__parent__``, represented in the C structure as ``m_parent``.
If this attribute exists, it must be a class. If this attribute exists, it represents the defining object.
If so, the function must be called with ``self`` being an instance of that class. For methods of extension types, this is the defining class (``__class__`` in plain Python)
This is mainly meant to support unbound methods of extension types, and for functions of a module, this is the defining module.
replacing ``method_descriptor``. In general, it can be any Python object.
Also ``__qualname__`` will use ``__objclass__`` as namespace If ``__parent__`` is a class, it carries special semantics:
(instead of ``m_self`` before). in that case, the function must be called with ``self`` being an instance of that class.
For methods of extension types, it is automatically assigned as the defining class Finally, ``__qualname__`` and ``__reduce__`` will use ``__parent__``
(``__class__`` in plain Python). as namespace (instead of ``__self__`` before).
#. Argument Clinic [#clinic]_ is not supported. #. A new attribute ``__objclass__`` which equals ``__parent__`` if ``__parent__``
is a class. Otherwise, accessing ``__objclass__`` raises ``AttributeError``.
This is meant to be backwards compatible with ``method_descriptor``.
#. The field ``ml_doc`` and the attributes ``__doc__`` and ``__text_signature__`` #. The field ``ml_doc`` and the attributes ``__doc__`` and
are gone. ``__text_signature__`` (see Argument Clinic [#clinic]_)
are not supported.
#. A new flag ``METH_ARG0_FUNCTION`` for ``ml_flags``. #. A new flag ``METH_PASS_FUNCTION`` for ``ml_flags``.
If this flag is set, the C function stored in ``ml_meth`` will be called with first argument If this flag is set, the C function stored in ``ml_meth`` is called with
equal to the function object instead of ``m_self``. an additional first argument equal to the function object.
#. A new flag ``METH_BINDING`` for ``ml_flags`` which only applies to #. A new flag ``METH_BINDING`` for ``ml_flags`` which only applies to
functions of a module (not methods of a class). functions of a module (not methods of a class).
If this flag is set, then ``m_self`` will be set to ``NULL`` instead If this flag is set, then ``m_self`` is set to ``NULL`` instead
of the module. of the module.
This allows the function to behave more like a Python function This allows the function to behave more like a Python function
as it enables ``__get__``. as it enables ``__get__``.
This flag also enables `Self slicing`_.
#. A new flag ``METH_CALL_UNBOUND`` to disable `self slicing`_.
#. A new flag ``METH_PYTHON`` for ``ml_flags``. #. A new flag ``METH_PYTHON`` for ``ml_flags``.
This flag indicates that this function should be treated as Python function. This flag indicates that this function should be treated as Python function.
Ideally, use of this flag should be avoided because it goes Ideally, use of this flag should be avoided because it goes
against the duck typing philosophy. against the duck typing philosophy.
It is still needed in a few places though, for example `Profiling`_. It is still needed in a few places though, for example `profiling`_.
#. A new flag ``METH_CUSTOM`` for ``ml_flags`` which prevents automatic #. A new flag ``METH_CUSTOM`` for ``ml_flags`` which prevents automatic
generation of a ``builtin_function``, see `Automatic creation of built-in functions`_. generation of a ``builtin_function``, see `automatic creation of built-in functions`_.
The goal of ``base_function`` is that it supports all different ways The goal of ``base_function`` is that it supports all different ways
of calling functions and methods in just one structure. of calling functions and methods in just one structure.
For example, the new flag ``METH_ARG0_FUNCTION`` For example, the new flag ``METH_PASS_FUNCTION``
will be used by the implementation of Python functions. will be used by the implementation of methods.
It is not possible to directly create instances of ``base_function`` It is not possible to directly create instances of ``base_function``
(``tp_new`` is ``NULL``). (``tp_new`` is ``NULL``).
@ -142,7 +155,7 @@ These are the relevant C structures::
PyCFunctionDef *m_ml; /* Description of the C function to call */ PyCFunctionDef *m_ml; /* Description of the C function to call */
PyObject *m_self; /* __self__: anything, can be NULL; readonly */ PyObject *m_self; /* __self__: anything, can be NULL; readonly */
PyObject *m_module; /* __module__: anything (typically str) */ PyObject *m_module; /* __module__: anything (typically str) */
PyTypeObject *m_objclass; /* __objclass__: type or NULL; readonly */ PyObject *m_parent; /* __parent__: anything, can be NULL; readonly */
PyObject *m_weakreflist; /* List of weak references */ PyObject *m_weakreflist; /* List of weak references */
} PyBaseFunctionObject; } PyBaseFunctionObject;
@ -157,8 +170,8 @@ Subclasses may extend ``PyCFunctionDef`` with extra fields.
The Python attribute ``__self__`` returns ``m_self``, The Python attribute ``__self__`` returns ``m_self``,
except if ``METH_STATIC`` is set. except if ``METH_STATIC`` is set.
In that case, ``None`` is returned. In that case or if ``m_self`` is ``NULL``,
If ``m_self`` is ``NULL``, then there is no ``__self__`` attribute at all. then there is no ``__self__`` attribute at all.
For that reason, we write either ``m_self`` or ``__self__`` in this PEP For that reason, we write either ``m_self`` or ``__self__`` in this PEP
with slightly different meanings. with slightly different meanings.
@ -180,19 +193,23 @@ This is a copy of ``base_function``, with the following differences:
} PyMethodDef; } PyMethodDef;
Note that ``PyMethodDef`` is part of the Python Stable ABI [#ABI]_ Note that ``PyMethodDef`` is part of the Python Stable ABI [#ABI]_
and it is used by most extension modules, and it is used by practically all extension modules,
so we absolutely cannot change this structure. so we absolutely cannot change this structure.
#. Argument Clinic [#clinic]_ is supported. #. Argument Clinic [#clinic]_ is supported.
#. ``__self__`` always exists. In the cases where ``base_function.__self__``
would raise ``AttributeError``, instead ``None`` is returned.
The type object is ``PyTypeObject PyCFunction_Type`` The type object is ``PyTypeObject PyCFunction_Type``
and we define ``PyCFunctionObject`` as alias of ``PyBaseFunctionObject``. and we define ``PyCFunctionObject`` as alias of ``PyBaseFunctionObject``
(except for the type of ``m_ml``).
defined_function defined_function
---------------- ----------------
The class ``defined_function`` (a subclass of ``base_function``) adds The class ``defined_function`` (a subclass of ``base_function``) adds
support for various standard attributes which are used in ``inspect``. support for various standard attributes which are used by ``inspect``.
This would be a good class to use for auto-generated C code, for example produced by Cython [#cython]_. This would be a good class to use for auto-generated C code, for example produced by Cython [#cython]_.
The layout of the C structure is as follows:: The layout of the C structure is as follows::
@ -213,9 +230,8 @@ The layout of the C structure is as follows::
PyObject *func_dict; /* __dict__: dict or NULL */ PyObject *func_dict; /* __dict__: dict or NULL */
} PyDefinedFunctionObject; } PyDefinedFunctionObject;
This class adds various slots like ``__doc__`` and ``__code__`` to access the C attributes. The descriptor ``__name__`` returns ``func_name``.
The slot ``__name__`` returns ``func_name``. When setting ``__name__``, also ``base.m_ml->ml_name`` is updated
When setting ``__name__``, also ``base.m_ml.ml_name`` is updated
with the UTF-8 encoded name. with the UTF-8 encoded name.
None of the attributes is required to be meaningful. None of the attributes is required to be meaningful.
@ -226,7 +242,7 @@ And ``__defaults__`` is not required to be used for calling the function.
Apart from adding these extra attributes, Apart from adding these extra attributes,
``defined_function`` behaves exactly the same as ``base_function``. ``defined_function`` behaves exactly the same as ``base_function``.
**TODO**: maybe find a better name for ``defined_function``. **TODO**: find a better name for ``defined_function``.
Other proposals: ``builtout_function`` (a function that is better built out; pun on builtin), Other proposals: ``builtout_function`` (a function that is better built out; pun on builtin),
``generic_function`` (original proposal but conflicts with ``functools.singledispatch`` generic functions), ``generic_function`` (original proposal but conflicts with ``functools.singledispatch`` generic functions),
``user_function`` (defined by the user as opposed to CPython). ``user_function`` (defined by the user as opposed to CPython).
@ -234,8 +250,7 @@ Other proposals: ``builtout_function`` (a function that is better built out; pun
function function
-------- --------
This is the class meant for functions implemented in Python, This is the class meant for functions implemented in Python.
formerly known as ``function``.
Unlike the other function types, Unlike the other function types,
instances of ``function`` can be created from Python code. instances of ``function`` can be created from Python code.
This is not changed, so we do not describe the details in this PEP. This is not changed, so we do not describe the details in this PEP.
@ -264,7 +279,9 @@ which reserves space to be used by ``base.m_ml``.
When constructing an instance of ``function`` from ``code`` and ``globals``, When constructing an instance of ``function`` from ``code`` and ``globals``,
an instance is created with ``base.m_ml = &_ml``, an instance is created with ``base.m_ml = &_ml``,
``base.m_self = NULL`` and with the ``METH_PYTHON`` flag set. ``base.m_self = NULL``.
Instances of ``function`` should always have the flag ``METH_PYTHON`` set.
This is also handled by the constructors.
To make subclassing easier, we also add a copy constructor: To make subclassing easier, we also add a copy constructor:
if ``f`` is an instance of ``defined_function`` with the ``METH_PYTHON`` if ``f`` is an instance of ``defined_function`` with the ``METH_PYTHON``
@ -287,19 +304,19 @@ This may be an already-bound method or simply not an instance of ``base_function
Therefore, in practice there are two kinds of methods: Therefore, in practice there are two kinds of methods:
- For arbitrary callables, we use a single fixed ``PyCFunctionDef`` - For arbitrary callables, we use a single fixed ``PyCFunctionDef``
structure with the ``METH_ARG0_FUNCTION`` flag set. structure with the ``METH_PASS_FUNCTION`` flag set.
The C function then calls ``__func__`` with the correct arguments.
- For methods which bind instances of ``base_function`` - For methods which bind instances of ``base_function``
(more precisely, which have the ``Py_TPFLAGS_BASEFUNCTION`` flag set) (more precisely, which have the ``Py_TPFLAGS_BASEFUNCTION`` flag set)
that allow self slicing, that have `self slicing`_,
we instead use the ``PyCFunctionDef`` from the original function. we instead use the ``PyCFunctionDef`` from the original function.
This way, we don't lose any performance when calling bound methods.
In this case, the ``__func__`` attribute is only used to implement In this case, the ``__func__`` attribute is only used to implement
various attributes but not for calling the method. various attributes but not for calling the method.
When constructing a new method from a ``base_function``, When constructing a new method from a ``base_function``,
we check that the ``self`` object is an instance of ``__objclass__`` we check that the ``self`` object is an instance of ``__objclass__``
(if such a class was specified) and raise a ``TypeError`` otherwise. (if a class was specified as parent) and raise a ``TypeError`` otherwise.
The C structure is:: The C structure is::
@ -317,44 +334,74 @@ Calling base_function instances
We specify the implementation of ``__call__`` for instances of ``base_function``. We specify the implementation of ``__call__`` for instances of ``base_function``.
__objclass__ Checking __objclass__
------------ ---------------------
First of all, if the function has an ``__objclass__`` attribute but First of all, a type check is done if the ``__parent__`` of the function
``m_self`` is ``NULL`` (this is the case for unbound methods of extension types), is a class
(recall that ``__objclass__`` then becomes an alias of ``__parent__``):
if ``m_self`` is ``NULL`` (this is the case for unbound methods of extension types),
then the function must be called with at least one positional argument then the function must be called with at least one positional argument
and the first (typically called ``self``) must be an instance of ``__objclass__``. and the first (typically called ``self``) must be an instance of ``__objclass__``.
If not, a ``TypeError`` is raised. If not, a ``TypeError`` is raised.
Note that bound methods have ``m_self != NULL``, so the ``__objclass__``
is not checked.
Instead, the ``__objclass__`` check is done when constructing the method.
Flags Flags
----- -----
For convenience, we define a new constant: For convenience, we define a new constant:
``METH_CALLSIGNATURE`` combines all flags from ``PyCFunctionDef.ml_flags`` ``METH_CALLFLAGS`` combines all flags from ``PyCFunctionDef.ml_flags``
which specify the signature of the C function to be called. which specify the signature of the C function to be called.
It is equal to :: It is equal to ::
METH_VARARGS | METH_FASTCALL | METH_NOARGS | METH_O | METH_KEYWORDS METH_VARARGS | METH_FASTCALL | METH_NOARGS | METH_O | METH_KEYWORDS | METH_PASS_FUNCTION
Exactly one of the first four flags above must be set Exactly one of the first four flags above must be set
and only ``METH_VARARGS`` and ``METH_FASTCALL`` may be combined with ``METH_KEYWORDS``. and only ``METH_VARARGS`` and ``METH_FASTCALL`` may be combined with ``METH_KEYWORDS``.
Violating these rules is undefined behaviour. Violating these rules is undefined behaviour.
There is one new flag which affects calling functions, There are one new flags which affects calling functions,
namely ``METH_ARG0_FUNCTION``. namely ``METH_PASS_FUNCTION`` and ``METH_CALL_UNBOUND``.
Some flags are already documented in [#methoddoc]_. Some flags are already documented in [#methoddoc]_.
We explain the other two shortly. We explain the others below.
Self slicing Self slicing
------------ ------------
If the function has ``m_self == NULL`` If the function has ``m_self == NULL`` and the flag ``METH_CALL_UNBOUND``
and the flag ``METH_ARG0_FUNCTION`` is not set, is not set, then the first positional argument (if any)
then the first positional argument (if any)
is removed from ``*args`` and instead passed as first argument to the C function. is removed from ``*args`` and instead passed as first argument to the C function.
Effectively, the first positional argument is treated as ``__self__``. Effectively, the first positional argument is treated as ``__self__``.
This process is called "self slicing" and is meant to support unbound methods. This is meant to support unbound methods
This does not affect keyword arguments. such that the C function does not see the difference
between bound and unbound method calls.
This does not affect keyword arguments in any way.
This process is called *self slicing* and a function is said to
*have self slicing* if ``m_self == NULL`` and ``METH_CALL_UNBOUND`` is not set.
Note that a ``METH_NOARGS`` function which has self slicing
effectively has one argument, namely ``self``.
Analogously, a ``METH_O`` function with self slicing has two arguments.
METH_PASS_FUNCTION
------------------
If this flag is set, then the C function is called with an
additional first argument, namely the function itself
(the ``base_function`` instance).
As special case, if the function is a ``bound_method``,
then the underlying function of the method is passed
(but not recursively: if a ``bound_method`` wraps a ``bound_method``,
then ``__func__`` is only applied once).
For example, an ordinary ``METH_VARARGS`` function has signature
``(PyObject *self, PyObject *args)``.
With ``METH_VARARGS | METH_PASS_FUNCTION``, this becomes
``(PyObject *func, PyObject *self, PyObject *args)``.
METH_FASTCALL METH_FASTCALL
------------- -------------
@ -364,13 +411,13 @@ We suggest to officially support and document it.
If the flag ``METH_FASTCALL`` is set without ``METH_KEYWORDS``, If the flag ``METH_FASTCALL`` is set without ``METH_KEYWORDS``,
then the ``ml_meth`` field is of type ``PyCFunctionFast`` then the ``ml_meth`` field is of type ``PyCFunctionFast``
which takes the arguments ``(PyObject *arg0, PyObject *const *args, Py_ssize_t nargs)``. which takes the arguments ``(PyObject *self, PyObject *const *args, Py_ssize_t nargs)``.
Such a function takes only positional arguments and they are passed as plain C array Such a function takes only positional arguments and they are passed as plain C array
``args`` of length ``nargs``. ``args`` of length ``nargs``.
If the flags ``METH_FASTCALL | METH_KEYWORDS`` are set, If the flags ``METH_FASTCALL | METH_KEYWORDS`` are set,
then the ``ml_meth`` field is of type ``PyCFunctionFastWithKeywords`` then the ``ml_meth`` field is of type ``PyCFunctionFastWithKeywords``
which takes the arguments ``(PyObject *arg0, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)``. which takes the arguments ``(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)``.
The positional arguments are passed as C array ``args`` of length ``nargs``. The positional arguments are passed as C array ``args`` of length ``nargs``.
The *values* of the keyword arguments follow in that array, The *values* of the keyword arguments follow in that array,
starting at position ``nargs``. starting at position ``nargs``.
@ -378,14 +425,6 @@ The *keys* (names) of the keyword arguments are passed as a ``tuple`` in ``kwnam
As an example, assume that 3 positional and 2 keyword arguments are given. As an example, assume that 3 positional and 2 keyword arguments are given.
Then ``args`` is an array of length 3 + 2 = 5, ``nargs`` equals 3 and ``kwnames`` is a 2-tuple. Then ``args`` is an array of length 3 + 2 = 5, ``nargs`` equals 3 and ``kwnames`` is a 2-tuple.
METH_ARG0_FUNCTION
------------------
If this flag is set, then the first argument to the C function
is the function itself (the ``base_function`` instance) instead of ``m_self``.
In this case, the C function should deal with ``__self__``
by getting it from the function, for example using ``PyBaseFunction_GET_SELF``.
Automatic creation of built-in functions Automatic creation of built-in functions
======================================== ========================================
@ -402,20 +441,22 @@ This allows an application to customize the creation of functions
in an extension type or module. in an extension type or module.
If ``METH_CUSTOM`` is set, then ``METH_STATIC`` and ``METH_CLASS`` are ignored. If ``METH_CUSTOM`` is set, then ``METH_STATIC`` and ``METH_CLASS`` are ignored.
Built-in unbound methods Unbound methods of extension types
------------------------ ----------------------------------
The type of unbound methods changes from ``method_descriptor`` The type of unbound methods changes from ``method_descriptor``
to ``builtin_function``. to ``builtin_function``.
The object which appears as unbound method is the same object which The object which appears as unbound method is the same object which
appears in the class ``__dict__``. appears in the class ``__dict__``.
Python automatically sets the ``__objclass__`` attribute. Python automatically sets the ``__parent__`` attribute to the defining class.
Built-in functions of a module Built-in functions of a module
------------------------------ ------------------------------
For the case of functions of a module, For the case of functions of a module,
``__self__`` will be set to the module unless the flag ``METH_BINDING`` is set. ``__parent__`` will be set to the module.
Unless the flag ``METH_BINDING`` is given, also ``__self__``
will be set to the module (for backwards compatibility).
An important consequence is that such functions by default An important consequence is that such functions by default
do not become methods when used as attribute do not become methods when used as attribute
@ -438,7 +479,7 @@ A new ``PyTypeObject`` flag (for ``tp_flags``) is added:
``Py_TPFLAGS_BASEFUNCTION`` to indicate that instances of this type are ``Py_TPFLAGS_BASEFUNCTION`` to indicate that instances of this type are
functions which can be called as a ``base_function``. functions which can be called as a ``base_function``.
In other words, subclasses of ``base_function`` In other words, subclasses of ``base_function``
which follow the implementation from `Calling base_function instances`_. which follow the implementation from the section `Calling base_function instances`_.
This is different from flags like ``Py_TPFLAGS_LIST_SUBCLASS`` This is different from flags like ``Py_TPFLAGS_LIST_SUBCLASS``
because it indicates more than just a subclass: because it indicates more than just a subclass:
@ -458,12 +499,12 @@ Some of these are existing (possibly changed) functions, some are new:
- ``int PyBaseFunction_CheckFast(PyObject *op)``: return true if ``op`` - ``int PyBaseFunction_CheckFast(PyObject *op)``: return true if ``op``
is an instance of a class with the ``Py_TPFLAGS_BASEFUNCTION`` set. is an instance of a class with the ``Py_TPFLAGS_BASEFUNCTION`` set.
This is the function that you need to use to determine This is the function that you need to use to determine
whether you can safely access the ``base_function`` internals. whether it is meaningful to access the ``base_function`` internals.
- ``int PyBaseFunction_Check(PyObject *op)``: return true if ``op`` - ``int PyBaseFunction_Check(PyObject *op)``: return true if ``op``
is an instance of ``base_function``. is an instance of ``base_function``.
- ``PyObject *PyBaseFunction_New(PyTypeObject *cls, PyCFunctionDef *ml, PyObject *self, PyObject *module, PyTypeObject *objclass)``: - ``PyObject *PyBaseFunction_New(PyTypeObject *cls, PyCFunctionDef *ml, PyObject *self, PyObject *module, PyObject *parent)``:
create a new instance of ``cls`` (which must be a subclass of ``base_function``) create a new instance of ``cls`` (which must be a subclass of ``base_function``)
from the given data. from the given data.
@ -474,12 +515,13 @@ Some of these are existing (possibly changed) functions, some are new:
create a new instance of ``builtin_function``. create a new instance of ``builtin_function``.
As special case, if ``self`` is ``NULL``, As special case, if ``self`` is ``NULL``,
then set ``self = Py_None`` instead (for backwards compatibility). then set ``self = Py_None`` instead (for backwards compatibility).
If ``self`` is a module, then ``__parent__`` is set to ``self``.
Otherwise, ``__parent__`` is ``NULL``.
- For many existing ``PyCFunction_...`` and ``PyMethod_`` functions, - For many existing ``PyCFunction_...`` and ``PyMethod_`` functions,
we define a new function ``PyBaseFunction_...`` we define a new function ``PyBaseFunction_...``
acting on ``base_function`` instances. acting on ``base_function`` instances.
For backwards compatibility, The old functions are kept as aliases of the new functions.
the old functions are kept as aliases of the new functions.
- ``int PyFunction_Check(PyObject *op)``: return true if ``op`` - ``int PyFunction_Check(PyObject *op)``: return true if ``op``
is an instance of ``defined_function``. is an instance of ``defined_function``.
@ -511,13 +553,13 @@ Two types are added: ``types.BaseFunctionType`` corresponding to
Apart from that, no changes to the ``types`` module are made. Apart from that, no changes to the ``types`` module are made.
In particular, ``types.FunctionType`` refers to ``function``. In particular, ``types.FunctionType`` refers to ``function``.
However, the actual types will change: However, the actual types will change:
for example, ``types.BuiltinFunctionType`` will no longer be the same in particular, ``types.BuiltinFunctionType`` will no longer be the same
as ``types.BuiltinMethodType``. as ``types.BuiltinMethodType``.
Changes to the inspect module Changes to the inspect module
----------------------------- -----------------------------
``inspect.isbasefunction`` checks for an instance of ``base_function``. The new function ``inspect.isbasefunction`` checks for an instance of ``base_function``.
``inspect.isfunction`` checks for an instance of ``defined_function``. ``inspect.isfunction`` checks for an instance of ``defined_function``.
@ -525,7 +567,7 @@ Changes to the inspect module
``inspect.isroutine`` checks ``isbasefunction`` or ``ismethoddescriptor``. ``inspect.isroutine`` checks ``isbasefunction`` or ``ismethoddescriptor``.
Note that bpo-33261 [#bpo33261]_ should be fixed first. **NOTE**: bpo-33261 [#bpo33261]_ should be fixed first.
Profiling Profiling
--------- ---------
@ -564,7 +606,7 @@ The latter two are the only classes which can be instantiated directly
from the Python interpreter. from the Python interpreter.
We require ``base_function`` for consistency but we put no requirements on it: We require ``base_function`` for consistency but we put no requirements on it:
it is acceptable if this is just a copy of ``object``. it is acceptable if this is just a copy of ``object``.
Support for the new ``__objclass__`` attribute is not required. Support for the new ``__parent__`` (and ``__objclass__``) attribute is not required.
If there is no ``defined_function`` type, If there is no ``defined_function`` type,
then ``types.DefinedFunctionType`` should be an alias of ``types.FunctionType``. then ``types.DefinedFunctionType`` should be an alias of ``types.FunctionType``.
@ -588,7 +630,7 @@ Having two independent kinds of ``inspect.signature`` support on the same
class sounds like asking for problems. class sounds like asking for problems.
And this would not fix some of the other differences between built-in functions And this would not fix some of the other differences between built-in functions
and Python functions that were mentioned in the `Motivation`_. and Python functions that were mentioned in the `motivation`_.
Why __text_signature__ is not a solution Why __text_signature__ is not a solution
---------------------------------------- ----------------------------------------
@ -645,7 +687,7 @@ and this is left as a possible future improvement.
Slot wrappers for extension types like ``__init__`` or ``__eq__`` Slot wrappers for extension types like ``__init__`` or ``__eq__``
are quite different from normal methods. are quite different from normal methods.
They are also typically not called directly because you would normally They are also typically not called directly because you would normally
write ``foo[i]`` instead of ``foo.__getitem__(i)`` for example. write ``foo[i]`` instead of ``foo.__getitem__(i)``.
So these are left outside the scope of this PEP. So these are left outside the scope of this PEP.
Python also has an ``instancemethod`` class, which was used in Python 2 Python also has an ``instancemethod`` class, which was used in Python 2
@ -661,14 +703,14 @@ Not treating METH_STATIC and METH_CLASS
--------------------------------------- ---------------------------------------
Almost nothing in this PEP refers to the flags ``METH_STATIC`` and ``METH_CLASS``. Almost nothing in this PEP refers to the flags ``METH_STATIC`` and ``METH_CLASS``.
These flags are checked only by the `Automatic creation of built-in functions`_. These flags are checked only by the `automatic creation of built-in functions`_.
When a ``staticmethod``, ``classmethod`` or ``classmethod_descriptor`` When a ``staticmethod``, ``classmethod`` or ``classmethod_descriptor``
is bound (i.e. ``__get__`` is called), is bound (i.e. ``__get__`` is called),
a ``base_function`` instance is created with ``m_self != NULL``. a ``base_function`` instance is created with ``m_self != NULL``.
For a ``classmethod``, this is obvious since ``m_self`` For a ``classmethod``, this is obvious since ``m_self``
is the class that the method is bound to. is the class that the method is bound to.
For a ``staticmethod``, one can take an arbitrary Python object for ``m_self``. For a ``staticmethod``, one can take an arbitrary Python object for ``m_self``.
For backwards compatibility, we choose ``m_self = __objclass__`` for static methods For backwards compatibility, we choose ``m_self = __parent__`` for static methods
of extension types. of extension types.
__self__ in base_function __self__ in base_function
@ -695,35 +737,42 @@ For ``builtin_function``, the docstring is stored (together with the text signat
as C string in the read-only ``ml_doc`` field of a ``PyMethodDef``. as C string in the read-only ``ml_doc`` field of a ``PyMethodDef``.
For ``defined_function``, the docstring is stored as a writable Python object For ``defined_function``, the docstring is stored as a writable Python object
and it does not actually need to be a string. and it does not actually need to be a string.
This is done like this for backwards compatibility and because It looks hard to unify these two very different ways of dealing with ``__doc__``.
it looks hard to unify these two very different ways of dealing with ``__doc__``. For backwards compatibility, we keep the existing implementations.
Subclassing Subclassing
----------- -----------
We disallow subclassing of ``builtin_function`` and ``bound_method`` We disallow subclassing of ``builtin_function`` and ``bound_method``
to enable fast type checks for ``PyBuiltinFunction_Check`` and ``PyMethod_Check()``. to enable fast type checks for ``PyCFunction_Check`` and ``PyMethod_Check``.
We allow subclassing of the other classes because there is no reason to disallow it. We allow subclassing of the other classes because there is no reason to disallow it.
For Python modules, the only relevant class to subclass is For Python modules, the only relevant class to subclass is
``function`` because the others cannot be instantiated anyway. ``function`` because the others cannot be instantiated anyway.
Replacing tp_call: METH_ARG0_FUNCTION Replacing tp_call: METH_PASS_FUNCTION and METH_CALL_UNBOUND
------------------------------------- -----------------------------------------------------------
The new flag ``METH_ARG0_FUNCTION`` is meant to support cases where The new flags ``METH_PASS_FUNCTION`` and ``METH_CALL_UNBOUND``
formerly a custom ``tp_call`` was used. are meant to support cases where formerly a custom ``tp_call`` was used.
It would reduce the number of special fast paths in ``Python/ceval.c`` It reduces the number of special fast paths in ``Python/ceval.c``
for calling objects: for calling objects:
instead of treating Python functions, built-in functions and methods, instead of treating Python functions, built-in functions and method descriptors
there would only be a single check. separately, there would only be a single check.
The signature of ``tp_call`` is essentially the signature The signature of ``tp_call`` is essentially the signature
of ``PyBaseFunctionObject.m_ml.ml_meth`` with flags of ``PyBaseFunctionObject.m_ml.ml_meth`` with flags
``METH_VARARGS | METH_KEYWORDS | METH_ARG0_FUNCTION``. ``METH_VARARGS | METH_KEYWORDS | METH_PASS_FUNCTION | METH_CALL_UNBOUND``
(the only difference is an added ``self`` argument).
Therefore, it should be easy to change existing ``tp_call`` slots Therefore, it should be easy to change existing ``tp_call`` slots
to use ``METH_ARG0_FUNCTION``. to use the ``base_function`` implementation instead.
There is one extra complication though: ``__self__`` must be handled manually.
It also makes sense to use ``METH_PASS_FUNCTION`` without ``METH_CALL_UNBOUND``
in cases where the C function simply needs access to additional metadata
from the function, such as the ``__parent__``.
This is for example needed to support PEP 573.
Converting existing methods to use ``METH_PASS_FUNCTION`` is trivial:
it only requires adding an extra argument to the C function.
User flags: METH_CUSTOM and METH_USRx User flags: METH_CUSTOM and METH_USRx
------------------------------------- -------------------------------------
@ -801,8 +850,8 @@ New attributes
-------------- --------------
Some objects get new special double-underscore attributes. Some objects get new special double-underscore attributes.
For example, ``__objclass__`` now appears on bound methods too For example, the new attribute ``__parent__`` appears on
and all methods get a ``__func__`` attribute. all built-in functions and all methods get a ``__func__`` attribute.
The fact that ``__self__`` is now a special read-only attribute The fact that ``__self__`` is now a special read-only attribute
for Python functions caused trouble in [#bpo33265]_. for Python functions caused trouble in [#bpo33265]_.
Generally, we expect that not much will break though. Generally, we expect that not much will break though.
@ -810,15 +859,16 @@ Generally, we expect that not much will break though.
method_descriptor and PyDescr_NewMethod method_descriptor and PyDescr_NewMethod
--------------------------------------- ---------------------------------------
The classes ``method_descriptor`` and the constructor ``PyDescr_NewMethod`` The class ``method_descriptor`` and the constructor ``PyDescr_NewMethod``
are deprecated and no longer used by CPython itself. should be deprecated.
For now, they are kept for backwards compatibility. They are no longer used by CPython itself but are still supported.
Two-phase Implementation Two-phase Implementation
======================== ========================
**TODO**: this section is optional. When accepting this PEP, it should **TODO**: this section is optional.
If this PEP is accepted, it should
be decided whether to apply this two-phase implementation or not. be decided whether to apply this two-phase implementation or not.
As mentioned above, the `changes to types and inspect`_ can break some As mentioned above, the `changes to types and inspect`_ can break some
@ -833,12 +883,12 @@ Implement this PEP but duplicate the classes ``bound_method``
and ``builtin_function``. and ``builtin_function``.
Add a new class ``builtin_method`` which is an exact copy of ``builtin_function`` Add a new class ``builtin_method`` which is an exact copy of ``builtin_function``
add a new class ``bound_builtin_method`` which is an exact copy add a new class ``bound_builtin_method`` which is an exact copy
of ``bound_method`` (in both cases, only the name would differ). of ``bound_method`` (in both cases, literally only the name of the class would differ).
The class ``builtin_method`` will be used for unbound methods The class ``builtin_method`` will be used for unbound methods
of extension types. of extension types.
It should be seen as continuation of the existing class It should be seen as continuation of the existing class
``method-descriptor``. ``method_descriptor``.
This ensures 100% backwards compatibility for these objects This ensures 100% backwards compatibility for these objects
(except for added attributes and maybe the ``repr()``). (except for added attributes and maybe the ``repr()``).