164 lines
8.1 KiB
ReStructuredText
164 lines
8.1 KiB
ReStructuredText
PEP: 576
|
|
Title: Rationalize Built-in function classes
|
|
Author: Mark Shannon <mark@hotpy.org>
|
|
BDFL-Delegate: Petr Viktorin
|
|
Status: Withdrawn
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 10-May-2018
|
|
Python-Version: 3.8
|
|
Post-History: 17-May-2018
|
|
23-June-2018
|
|
08-July-2018
|
|
29-Mar-1028
|
|
|
|
Abstract
|
|
========
|
|
|
|
Expose the "FastcallKeywords" convention used internally by CPython to third-party code, and make the ``inspect`` module use duck-typing.
|
|
In combination this will allow third-party C extensions and tools like Cython to create objects that use the same calling conventions as built-in and Python functions, thus gaining performance parity with built-in functions like ``len`` or ``print``.
|
|
|
|
A small improvement in the performance of existing code is expected.
|
|
|
|
Motivation
|
|
==========
|
|
|
|
Currently third-party module authors face a dilemma when implementing
|
|
functions in C. Either they can use one of the pre-existing built-in function
|
|
or method classes or implement their own custom class in C.
|
|
The first choice causes them to lose the ability to access the internals of the callable object.
|
|
The second choice is an additional maintenance burden and, more importantly,
|
|
has a significant negative impact on performance.
|
|
|
|
This PEP aims to allow authors of third-party C modules, and tools like to Cython, to utilize the faster calling convention used internally by CPython for built-in functions and methods, and to do so without a loss of capabilities relative to a function implemented in Python.
|
|
|
|
Introspection
|
|
-------------
|
|
|
|
The inspect module will fully support duck-typing when introspecting callables.
|
|
|
|
The ``inspect.Signature.from_callable()`` function computes the signature of a callable. If an object has a ``__signature__``
|
|
property, then ``inspect.Signature.from_callable()`` simply returns that. To further support duck-typing, if a callable has a ``__text_signature__``
|
|
then the ``__signature__`` will be created from that.
|
|
|
|
This means that 3rd party builtin-functions can implement ``__text_signature__`` if sufficient,
|
|
and the more expensive ``__signature__`` if necessary.
|
|
|
|
Efficient calls to third-party callables
|
|
----------------------------------------
|
|
|
|
Currently the majority of calls are dispatched to ``function``\s and ``method_descriptor``\s in custom code, using the "FastcallKeywords" internal calling convention. This PEP proposes that this calling convention is implemented via a C function pointer. Third-party callables which implement this binary interface will have the potential to be called as fast as a built-in function.
|
|
|
|
Continued prohibition of callable classes as base classes
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Currently any attempt to use ``function``, ``method`` or ``method_descriptor`` as a base class for a new class will fail with a ``TypeError``. This behaviour is desirable as it prevents errors when a subclass overrides the ``__call__`` method. If callables could be sub-classed then any call to a ``function`` or a ``method_descriptor`` would need an additional check that the ``__call__`` method had not been overridden. By exposing an additional call mechanism, the potential for errors becomes greater. As a consequence, any third-party class implementing the addition call interface will not be usable as a base class.
|
|
|
|
|
|
New classes and changes to existing classes
|
|
===========================================
|
|
|
|
Python visible changes
|
|
----------------------
|
|
|
|
#. A new built-in class, ``builtin_function``, will be added.
|
|
|
|
#. ``types.BuiltinFunctionType`` will refer to ``builtin_function`` not ``builtin_function_or_method``.
|
|
|
|
#. Instances of the ``builtin_function`` class will retain the ``__module__`` property of ``builtin_function_or_method`` and gain the ``func_module`` and ``func_globals`` properties. The ``func_module`` allows access to the module to which the function belongs. Note that this is different from the ``__module__`` property which merely returns the name of the module. The ``func_globals`` property is equivalent to ``func_module.__dict__`` and is provided to mimic the Python function property of the same name.
|
|
|
|
#. When binding a ``method_descriptor`` instance to an instance of its owning class, a ``bound_method`` will be created instead of a ``builtin_function_or_method``. This means that the ``method_descriptors`` now mimic the behaviour of Python functions more closely. In other words, ``[].append`` becomes a ``bound_method`` instead of a ``builtin_function_or_method``.
|
|
|
|
|
|
C API changes
|
|
-------------
|
|
|
|
#. A new function ``PyBuiltinFunction_New(PyMethodDef *ml, PyObject *module)`` is added to create built-in functions.
|
|
|
|
#. ``PyCFunction_NewEx()`` and ``PyCFunction_New()`` are deprecated and will return a ``PyBuiltinFunction`` if able, otherwise a ``builtin_function_or_method``.
|
|
|
|
Retaining backwards compatibility in the C API and ABI
|
|
======================================================
|
|
|
|
The proposed changes are fully backwards and forwards compatible at both the API and ABI level.
|
|
|
|
|
|
Internal C changes
|
|
------------------
|
|
|
|
Two new flags will be allowed for the ``typeobject.tp_flags`` field.
|
|
These are ``Py_TPFLAGS_EXTENDED_CALL`` and ``Py_TPFLAGS_FUNCTION_DESCRIPTOR``
|
|
|
|
Py_TPFLAGS_EXTENDED_CALL
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
For any built-in class that sets ``Py_TPFLAGS_EXTENDED_CALL``
|
|
The C struct corresponding to this built-in class must begin with the struct ``PyExtendedCallable`` which is defined as follows::
|
|
|
|
typedef PyObject *(*extended_call_ptr)(PyObject *callable, PyObject** args,
|
|
int positional_argcount, PyTupleObject* kwnames);
|
|
|
|
typedef struct {
|
|
PyObject_HEAD
|
|
extended_call_ptr ext_call;
|
|
} PyExtendedCallable;
|
|
|
|
Any class that sets the ``Py_TPFLAGS_EXTENDED_CALL`` cannot be used as a base class and a TypeError will be raised if any Python code tries to use it a base class.
|
|
|
|
|
|
Py_TPFLAGS_FUNCTION_DESCRIPTOR
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
If this flag is set for a built-in class ``F``, then instances of that class are expected to behave the same as a Python function when used as a class attribute.
|
|
Specifically, this mean that the value of ``c.m`` where ``C.m`` is an instanceof the built-in class ``F`` (and ``c`` is an instance of ``C``) must be a bound-method binding ``C.m`` and ``c``.
|
|
Without this flag, it would be impossible for custom callables to behave like Python functions *and* be efficient as Python or built-in functions.
|
|
|
|
|
|
|
|
Changes to existing C structs
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The ``function``, ``method_descriptor`` and ``method`` classes will have their corresponding structs changed to
|
|
start with the ``PyExtendedCallable`` struct.
|
|
|
|
Third-party built-in classes using the new extended call interface
|
|
------------------------------------------------------------------
|
|
|
|
To enable call performance on a par with Python functions and built-in functions, third-party callables should set the ``Py_TPFLAGS_EXTENDED_CALL`` bit of ``tp_flags`` and ensure that the corresponding C struct starts with the ``PyExtendedCallable``.
|
|
Any built-in class that has the ``Py_TPFLAGS_EXTENDED_CALL`` bit set must also implement the ``tp_call`` function and make sure its behaviour is consistent with the ``ext_call`` function.
|
|
|
|
Performance implications of these changes
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Adding a function pointer to each callable, rather than each class of callable, enables the choice of dispatching function (the code to shuffle arguments about and do error checking) to be made when the callable object is created rather than when it is called. This should reduce the number of instructions executed between the call-site in the interpreter and the execution of the callee.
|
|
|
|
|
|
Alternative Suggestions
|
|
=======================
|
|
|
|
:pep:`580` is an alternative approach to solving the same problem as this PEP.
|
|
|
|
|
|
|
|
Reference implementation
|
|
========================
|
|
|
|
A draft implementation can be found at https://github.com/markshannon/cpython/tree/pep-576-minimal
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|
|
End:
|