From 591d466b92a7e39aa3fe1540f09dbb011924f734 Mon Sep 17 00:00:00 2001 From: Jeroen Demeyer Date: Mon, 1 Apr 2019 21:55:23 +0200 Subject: [PATCH] PEP 580: new flag CCALL_DEFARG instead of CCALL_FUNCARG (#967) --- pep-0580.rst | 79 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 54 insertions(+), 25 deletions(-) diff --git a/pep-0580.rst b/pep-0580.rst index 86adcd197..50fd84c70 100644 --- a/pep-0580.rst +++ b/pep-0580.rst @@ -131,8 +131,8 @@ It must be a strictly positive integer. At that offset, a ``PyCCallRoot`` structure appears:: typedef struct { - PyCCallDef *cr_ccall; - PyObject *cr_self; /* __self__ argument for methods */ + const PyCCallDef *cr_ccall; + PyObject *cr_self; /* __self__ argument for methods */ } PyCCallRoot; The ``PyCCallDef`` structure contains everything needed to describe how @@ -255,32 +255,22 @@ signature flags: - ``CCALL_O``: ``cc_func(PyObject *self, PyObject *arg)`` -The flag ``CCALL_FUNCARG`` may be combined with any of these. +The flag ``CCALL_DEFARG`` may be combined with any of these. If so, the C function takes an additional argument -as first argument before ``self``. -This argument is used to pass the function object (see NOTE 1 below). +as first argument before ``self``, +namely a const pointer to the ``PyCCallDef`` structure used for this call. For example, we have the following signature: -- ``CCALL_FUNCARG | CCALL_VARARGS``: - ``cc_func(PyObject *func, PyObject *self, PyObject *args)`` +- ``CCALL_DEFARG | CCALL_VARARGS``: + ``cc_func(const PyCCallDef *def, PyObject *self, PyObject *args)`` -One exception is ``CCALL_FUNCARG | CCALL_NOARGS``: +One exception is ``CCALL_DEFARG | CCALL_NOARGS``: the ``unused`` argument is dropped, so the signature becomes -- ``CCALL_FUNCARG | CCALL_NOARGS``: - ``cc_func(PyObject *func, PyObject *self)`` +- ``CCALL_DEFARG | CCALL_NOARGS``: + ``cc_func(const PyCCallDef *def, PyObject *self)`` -**NOTE 1**: with "function object", we mean the ``self`` in ``__call__``. -In the case of bound methods, it is currently unspecified -whether this refers -to the bound method or the original function (which is wrapped by the bound method). -In the reference implementation, the bound method is passed. -In the future, this may change to the wrapped function. -Despite this ambiguity, the implementation of bound methods -guarantees that ``PyCCall_CCALLDEF(func)`` -points to the ``PyCCallDef`` of the original function. - -**NOTE 2**: unlike the existing ``METH_...`` flags, +**NOTE**: unlike the existing ``METH_...`` flags, the ``CCALL_...`` constants do not necessarily represent single bits. So checking ``if (cc_flags & CCALL_VARARGS)`` is not a valid way for checking the signature. @@ -405,13 +395,13 @@ In other words, ``PyCCall_Check(func)`` must be true. Macros to access the ``PyCCallRoot`` and ``PyCCallDef`` structures: -- ``PyCCallRoot *PyCCall_CCALLROOT(PyObject *func)``: +- ``const PyCCallRoot *PyCCall_CCALLROOT(PyObject *func)``: pointer to the ``PyCCallRoot`` structure inside ``func``. -- ``PyCCallDef *PyCCall_CCALLDEF(PyObject *func)``: +- ``const PyCCallDef *PyCCall_CCALLDEF(PyObject *func)``: shorthand for ``PyCCall_CCALLROOT(func)->cr_ccall``. -- ``PyCCallDef *PyCCall_FLAGS(PyObject *func)``: +- ``uint32_t PyCCall_FLAGS(PyObject *func)``: shorthand for ``PyCCall_CCALLROOT(func)->cr_ccall->cc_flags``. - ``PyObject *PyCCall_SELF(PyOject *func)``: @@ -452,7 +442,11 @@ the implementation becomes very similar, but they remain separate classes The ``PyCCallDef`` structure is simply stored as part of the object structure. Both classes use ``PyCFunctionObject`` as object structure. -This is the new layout for both classes:: +This is the new layout for both classes: + +.. _PyCFunctionObject: + +:: typedef struct { PyObject_HEAD @@ -537,6 +531,8 @@ performance improvements are discussed: - https://mail.python.org/pipermail/python-dev/2018-July/154775.html +- https://mail.python.org/pipermail/python-dev/2019-April/156954.html + Stable ABI ========== @@ -678,6 +674,31 @@ since it cannot distinguish at runtime between a function (without ``self`` argument) and a bound method (with ``self`` argument). The ``CCALL_SELFARG`` flag makes this difference explicit. +Why CCALL_DEFARG? +----------------- + +The flag ``CCALL_DEFARG`` gives the callee access to the ``PyCCallDef *``. +There are various use cases for this: + +1. The callee can use the ``cc_parent`` field, which is useful for PEP 573. + +2. Applications are free to extend the ``PyCCallDef`` structure with user-defined + fields, which can then be accessed analogously. + +3. In the case where the ``PyCCallDef`` structure + is part of the object structure + (this is true for example for `PyCFunctionObject`_), + an appropriate offset can be subtracted from the ``PyCCallDef`` pointer + to get a pointer to the callable object defining that ``PyCCallDef``. + +An earlier version of this PEP defined a flag ``CCALL_FUNCARG`` +instead of ``CCALL_DEFARG`` which would pass the callable object +to the callee. +This had similar use cases, but there was some ambiguity for +bound methods: should the "callable object" be the bound method +object or the original function wrapped by the method? +By passing the ``PyCCallDef *`` instead, this ambiguity is gone +since the bound method uses the ``PyCCallDef *`` from the wrapped function. Replacing tp_print ------------------ @@ -717,6 +738,10 @@ where this PEP has been discussed: - https://mail.python.org/pipermail/python-dev/2018-October/155403.html +- https://mail.python.org/pipermail/python-dev/2019-March/156853.html + +- https://mail.python.org/pipermail/python-dev/2019-March/156879.html + Reference implementation ======================== @@ -724,6 +749,10 @@ Reference implementation The reference implementation can be found at https://github.com/jdemeyer/cpython/tree/pep580 +For an example of using the C call protocol, +the following branch implements ``functools.lru_cache`` using PEP 580: +https://github.com/jdemeyer/cpython/tree/lru580 + References ==========