From 7d1b83a0d3e6aff960ebe3015207898751281500 Mon Sep 17 00:00:00 2001 From: jdemeyer Date: Thu, 12 Apr 2018 18:13:50 +0200 Subject: [PATCH] Update PEP 575 (#619) --- pep-0575.rst | 125 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 80 insertions(+), 45 deletions(-) diff --git a/pep-0575.rst b/pep-0575.rst index b3b71e411..fedadf4c3 100644 --- a/pep-0575.rst +++ b/pep-0575.rst @@ -6,7 +6,7 @@ Type: Standards Track Content-Type: text/x-rst Created: 27-Mar-2018 Python-Version: 3.8 -Post-History: 31-Mar-2018 +Post-History: 31-Mar-2018, 12-Apr-2018 Abstract @@ -66,7 +66,7 @@ This is the new class hierarchy for functions and methods:: / | defined_function / | \ builtin_function (*) | \ - | python_function + | function | bound_method (*) @@ -224,13 +224,13 @@ And ``__defaults__`` is not required to be used for calling the function. Apart from adding these extra attributes, ``defined_function`` behaves exactly the same as ``base_function``. -python_function ---------------- +function +-------- This is the class meant for functions implemented in Python, formerly known as ``function``. Unlike the other function types, -instances of ``python_function`` can be created from Python code. +instances of ``function`` can be created from Python code. This is not changed, so we do not describe the details in this PEP. The layout of the C structure is almost the same as ``defined_function``:: @@ -255,11 +255,11 @@ The layout of the C structure is almost the same as ``defined_function``:: The only difference is an ``_ml`` field which reserves space to be used by ``base.m_ml``. -When constructing an instance of ``python_function`` from ``code`` and ``globals``, +When constructing an instance of ``function`` from ``code`` and ``globals``, an instance is created with ``base.m_ml = &_ml``, ``base.m_self = NULL`` and with the ``METH_PYTHON`` flag set. -To make subclassing easier, we also add a copying constructor: +To make subclassing easier, we also add a copy constructor: if ``f`` is an instance of ``defined_function`` with the ``METH_PYTHON`` flag set, then ``types.FunctionType(f)`` copies ``f``. @@ -445,12 +445,18 @@ because that would not be safe (heap types can be changed dynamically). C API functions --------------- -We add and change some Python/C API functions: +We list some relevant Python/C API macros and functions. +Some of these are existing (possibly changed) functions, some are new: + +- ``int PyBaseFunction_CheckFast(PyObject *op)``: return true if ``op`` + is an instance of a class with the ``Py_TPFLAGS_BASEFUNCTION`` set. + This is the function that you need to use to determine + whether you can safely access the ``base_function`` internals. - ``int PyBaseFunction_Check(PyObject *op)``: return true if ``op`` - is an instance of a type with the ``Py_TPFLAGS_BASEFUNCTION`` set. + is an instance of ``base_function``. -- ``PyObject* PyBaseFunction_New(PyTypeObject *cls, PyCFunctionDef *ml, PyObject *self, PyObject *module, PyTypeObject *objclass)``: +- ``PyObject *PyBaseFunction_New(PyTypeObject *cls, PyCFunctionDef *ml, PyObject *self, PyObject *module, PyTypeObject *objclass)``: create a new instance of ``cls`` (which must be a subclass of ``base_function``) from the given data. @@ -462,25 +468,32 @@ We add and change some Python/C API functions: As special case, if ``self`` is ``NULL``, then set ``self = Py_None`` instead (for backwards compatibility). -- ``int PyFunction_Check(PyObject *op)``: return true if ``op`` - is an instance of ``defined_function``. - -- ``PyPythonFunction_New(PyTypeObject *cls, PyObject *code, PyObject *globals, PyObject *name, PyObject *qualname)``: - create a new instance of ``cls`` (which must be a sublass of ``python_function``) - from the given data. - -- ``PyObject* PyFunction_New(PyObject *code, PyObject *globals)``: - create a new instance of ``python_function``. - -- ``PyObject* PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname)``: - create a new instance of ``python_function``. - - For many existing ``PyCFunction_...`` and ``PyMethod_`` functions, we define a new function ``PyBaseFunction_...`` acting on ``base_function`` instances. For backwards compatibility, the old functions are kept as aliases of the new functions. +- ``int PyFunction_Check(PyObject *op)``: return true if ``op`` + is an instance of ``defined_function``. + +- ``PyObject *PyFunction_NewPython(PyTypeObject *cls, PyObject *code, PyObject *globals, PyObject *name, PyObject *qualname)``: + create a new instance of ``cls`` (which must be a sublass of ``function``) + from the given data. + +- ``PyObject *PyFunction_New(PyObject *code, PyObject *globals)``: + create a new instance of ``function``. + +- ``PyObject *PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname)``: + create a new instance of ``function``. + +- ``PyObject *PyFunction_Copy(PyTypeObject *cls, PyObject *func)``: + create a new instance of ``cls`` (which must be a sublass of ``function``) + by copying a given ``defined_function``. + +- All other existing ``PyFunction_...`` functions now act on ``defined_function`` + instances (instead of ``function``). + Changes to the types module --------------------------- @@ -489,7 +502,7 @@ Two types are added: ``types.BaseFunctionType`` corresponding to ``defined_function``. Apart from that, no changes to the ``types`` module are made. -In particular, ``types.FunctionType`` refers to ``python_function``. +In particular, ``types.FunctionType`` refers to ``function``. However, the actual types will change: for example, ``types.BuiltinFunctionType`` will no longer be the same as ``types.BuiltinMethodType``. @@ -539,7 +552,7 @@ Non-CPython implementations =========================== For other implementations of Python apart from CPython, -only the classes ``base_function``, ``bound_method`` and ``python_function`` are required. +only the classes ``base_function``, ``bound_method`` and ``function`` are required. The latter two are the only classes which can be instantiated directly from the Python interpreter. We require ``base_function`` for consistency but we put no requirements on it: @@ -583,28 +596,21 @@ And even if ``__text_signature__`` would allow arbitrary signatures somehow, that is only one piece of introspection: it does not help with ``inspect.getsourcefile`` for example. -defined_function versus python_function ---------------------------------------- - -The names ``defined_function`` and ``python_function`` -were chosen to be different from ``function`` -because none of the two classes ``defined_function``/``python_function`` -is an obvious candidate to receive the ``function`` name. -It also allows to use the word "function" informally without referring -to a specific class. +defined_function versus function +-------------------------------- In many places, a decision needs to be made whether the old ``function`` class -should be replaced by ``defined_function`` or ``python_function``. +should be replaced by ``defined_function`` or the new ``function`` class. This is done by thinking of the most likely use case: -1. ``types.FunctionType`` refers to ``python_function`` because that +1. ``types.FunctionType`` refers to ``function`` because that type might be used to construct instances using ``types.FunctionType(...)``. 2. ``inspect.isfunction()`` refers to ``defined_function`` because this is the class where introspection is supported. 3. The C API functions ``PyFunction_New...`` - refer to ``python_function`` simply because one cannot create instances + refer to ``function`` simply because one cannot create instances of ``defined_function``. 4. The C API functions ``PyFunction_Check`` and ``PyFunction_Get/Set...`` @@ -678,7 +684,7 @@ to enable fast type checks for ``PyBuiltinFunction_Check`` and ``PyMethod_Check( We allow subclassing of the other classes because there is no reason to disallow it. For Python modules, the only relevant class to subclass is -``python_function`` because the others cannot be instantiated anyway. +``function`` because the others cannot be instantiated anyway. Replacing tp_call: METH_ARG0_FUNCTION ------------------------------------- @@ -726,6 +732,11 @@ For Python functions, essentially nothing changes. The attributes that existed before still exist and Python functions can be initialized, called and turned into methods as before. +The name ``function`` is kept for backwards compatibility. +While it might make sense to change the name to something more +specific like ``python_function``, +that would require a lot of annoying changes in documentation and testsuites. + Built-in functions of a module ------------------------------ @@ -744,22 +755,28 @@ was specifically designed to be backwards compatible. All attributes which existed before (like ``__objclass__`` and ``__self__``) still exist. -New classes ------------ +New and changed classes +----------------------- Tools which take various kinds of functions as input will need to deal with the new function hieararchy and the possibility of custom function classes. -If those tools use ``inspect`` properly, there should be few -backwards compatibility problems. +The proposed changes to ``types`` and ``inspect`` +are meant to minimize changes in behaviour. +However, it is unavoidable that some things change +and this can cause code to break. +In the Python standard library, +changes are needed in the ``doctest`` module because of this. New attributes -------------- -Some objects get new attributes. +Some objects get new special double-underscore attributes. For example, ``__objclass__`` now appears on bound methods too and all methods get a ``__func__`` attribute. -We expect that this will not cause problems. +The fact that ``__self__`` is now a special read-only attribute +for Python functions caused trouble in [#bpo33265]_. +Generally, we expect that not much will break though. method_descriptor and PyDescr_NewMethod --------------------------------------- @@ -772,9 +789,24 @@ They are kept for backwards compatibility though. Reference Implementation ======================== -The implementation in CPython is being developed at +Most of this PEP has been implemented for CPython at https://github.com/jdemeyer/cpython/tree/pep575 +There are four steps, corresponding to the commits on that branch. +After each step, CPython is in a mostly working state. + +1. Add the ``base_function`` class and make it a subclass for ``builtin_function``. + This is by far the biggest step as the complete ``__call__`` protocol + is implemented in this step. + +2. Rename ``method`` to ``bound_method`` and make it a subclass of ``base_function``. + Change unbound methods of extension types to be instances of ``builtin_function`` + such that bound methods of extension types are also instances of ``bound_method``. + +3. Implement ``defined_function`` and ``function``. + +4. Changes to other parts of Python, such as the standard library and testsuite. + Appendix: current situation =========================== @@ -902,6 +934,9 @@ References .. [#bpo33261] Python bug 33261, inspect.isgeneratorfunction fails on hand-created methods (https://bugs.python.org/issue33261 and https://github.com/python/cpython/pull/6448) +.. [#bpo33265] Python bug 33265, contextlib.ExitStack abuses __self__ + (https://bugs.python.org/issue33265 and https://github.com/python/cpython/pull/6456) + .. [#ABI] PEP 384, Defining a Stable ABI, Löwis (https://www.python.org/dev/peps/pep-0384) .. [#clinic] PEP 436, The Argument Clinic DSL, Hastings (https://www.python.org/dev/peps/pep-0436)