iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update PEP 579 and 580 (#678)

This commit is contained in:
jdemeyer 2018-06-23 00:02:29 +02:00 committed by Brett Cannon
parent a5ac5e105b
commit c2716b76f4
2 changed files with 150 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
pep-0579.rst
View File

@ -5,7 +5,7 @@ Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 04-Jun-2018
Post-History: 19-Jun-2018
Post-History: 20-Jun-2018
Abstract
@ -242,8 +242,8 @@ all variations of methods.
This is not a problem by itself, but a compounding issue.
For ordinary Python classes, the table below gives the classes
for various kinds of methods, where columns
refer to the class in the class ``__dict__``,
for various kinds of methods.
The columns refer to the class in the class ``__dict__``,
the class for unbound methods (bound to the class)
and the class for bound methods (bound to the instance):
@ -279,7 +279,7 @@ Since CPython has optimizations for calls to most of these objects,
the code for dealing with them can also become complex.
A good example of this is the ``call_function`` function in ``Python/ceval.c``.
**Solution**: all these class should implement the C call protocol.
**Solution**: all these classes should implement the C call protocol.
Then the complexity in the code can mostly be fixed by
checking for the C call protocol (``tp_ccalloffset != 0``)
instead of doing type checks.
@ -355,9 +355,46 @@ Such an entry could look like ::
{"__init__", NULL, METH_SLOTDOC, "pointer to __init__ doc goes here"}
11. Static methods and class methods should be callable
-------------------------------------------------------
Instances of ``staticmethod`` and ``classmethod`` should be callable.
Admittedly, there is no strong use case for this,
but it has occasionally been requested (see for example [#bpo20309]_).
Making static/class methods callable would increase consistency.
First of all, function decorators typically add functionality or modify
a function, but the result remains callable. This is not true for
``@staticmethod`` and ``@classmethod``.
Second, class methods of extension types are already callable::
>>> fromhex = float.__dict__["fromhex"]
>>> type(fromhex)
<class 'classmethod_descriptor'>
>>> fromhex(float, "0xff")
255.0
Third, one can see ``function``, ``staticmethod`` and ``classmethod``
as different kinds of unbound methods:
they all become ``method`` when bound, but the implementation of ``__get__``
is slightly different.
From this point of view, it looks strange that ``function`` is callable
but the others are not.
**Solution**:
when changing the implementation of ``staticmethod``, ``classmethod``,
we should consider making instances callable.
Even if this is not a goal by itself, it may happen naturally
because of the implementation.
References
==========
.. [#bpo20309] Not all method descriptors are callable
(https://bugs.python.org/issue20309)
.. [#bpo30071] Duck-typing inspect.isfunction()
(https://bugs.python.org/issue30071)

158
pep-0580.rst
View File

@ -6,7 +6,7 @@ Type: Standards Track
Content-Type: text/x-rst
Created: 14-Jun-2018
Python-Version: 3.8
Post-History: 19-Jun-2018
Post-History: 20-Jun-2018, 22-Jun-2018
Abstract
@ -61,12 +61,12 @@ and a new flag ``Py_TPFLAGS_HAVE_CCALL``.
If this flag is set, then ``tp_ccalloffset`` is assumed to be a valid
offset inside the object structure (similar to ``tp_weaklistoffset``).
It must be a strictly positive integer.
At that offset, a ``PyCMethodDef`` structure appears::
At that offset, a ``PyCCallRoot`` structure appears::
typedef struct {
PyCCallDef *cm_ccall;
PyObject *cm_self; /* __self__ argument for methods */
} PyCMethodDef;
PyCCallDef *cr_ccall;
PyObject *cr_self; /* __self__ argument for methods */
} PyCCallRoot;
The ``PyCCallDef`` structure contains everything needed to describe how
the function can be called::
@ -89,19 +89,6 @@ for ``tp_ccalloffset`` to mean counting from the end.
There does not seem to be a use case for it and it would only complicate
the implementation.
**NOTE**: in the reference implementation, ``tp_ccalloffset`` actually
replaces ``tp_print`` and ``Py_TPFLAGS_HAVE_CCALL`` is *not*
added to ``Py_TPFLAGS_DEFAULT``.
The latter ensures full backwards compatibility for existing
extension modules setting ``tp_print``.
It also means that we can require that ``tp_ccalloffset`` is a valid
offset when ``Py_TPFLAGS_HAVE_CCALL`` is specified:
we do not need to check ``tp_ccalloffset != 0``.
In future Python versions, we may decide that ``tp_print``
becomes ``tp_ccalloffset`` unconditionally,
drop the ``Py_TPFLAGS_HAVE_CCALL`` flag and instead check for
``tp_ccalloffset != 0``.
Parent
------
@ -129,6 +116,22 @@ The parent would also typically be used to implement ``__qualname__``.
Custom classes are free to set ``cc_parent`` to whatever they want.
It is only used by the C call protocol if the ``CCALL_OBJCLASS`` flag is set.
Using tp_print
--------------
We propose to replace the existing unused field ``tp_print``
by ``tp_ccalloffset``.
Since ``Py_TPFLAGS_HAVE_CCALL`` would *not* be added to
``Py_TPFLAGS_DEFAULT``, this ensures full backwards compatibility for
existing extension modules setting ``tp_print``.
It also means that we can require that ``tp_ccalloffset`` is a valid
offset when ``Py_TPFLAGS_HAVE_CCALL`` is specified:
we do not need to check ``tp_ccalloffset != 0``.
In future Python versions, we may decide that ``tp_print``
becomes ``tp_ccalloffset`` unconditionally,
drop the ``Py_TPFLAGS_HAVE_CCALL`` flag and instead check for
``tp_ccalloffset != 0``.
The C call protocol
===================
@ -155,7 +158,8 @@ signature flags:
- ``CCALL_FASTCALL | CCALL_KEYWORDS``: ``cc_func(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)``
- ``CCALL_NOARGS``: ``cc_func(PyObject *self, PyObject *unused)``
- ``CCALL_NULLARG``: ``cc_func(PyObject *self, PyObject *null)``
(the function takes no arguments but a ``NULL`` is passed to the C function)
- ``CCALL_O``: ``cc_func(PyObject *self, PyObject *arg)``
@ -174,7 +178,7 @@ for checking the signature.
Checking __objclass__
---------------------
If the ``CCALL_OBJCLASS`` flag is set and if ``cm_self`` is NULL
If the ``CCALL_OBJCLASS`` flag is set and if ``cr_self`` is NULL
(this is the case for unbound methods of extension types),
then a type check is done:
the function must be called with at least one positional argument
@ -185,11 +189,11 @@ If not, a ``TypeError`` is raised.
Self slicing
------------
If ``cm_self`` is not NULL or if the flag ``CCALL_SLICE_SELF``
If ``cr_self`` is not NULL or if the flag ``CCALL_SLICE_SELF``
is not set in ``cc_flags``, then the argument passed as ``self``
is simply ``cm_self``.
is simply ``cr_self``.
If ``cm_self`` is NULL and the flag ``CCALL_SLICE_SELF`` is set,
If ``cr_self`` is NULL and the flag ``CCALL_SLICE_SELF`` is set,
then the first positional argument (if any) is removed from
``args`` and instead passed as first argument to the C function.
Effectively, the first positional argument is treated as ``__self__``.
@ -198,9 +202,9 @@ not see the difference between bound and unbound method calls.
This does not affect keyword arguments in any way.
This process is called self slicing and a function is said to have self
slicing if ``cm_self`` is NULL and ``CCALL_SLICE_SELF`` is set.
slicing if ``cr_self`` is NULL and ``CCALL_SLICE_SELF`` is set.
Note that a ``METH_NOARGS`` function with self slicing effectively has
Note that a ``METH_NULLARG`` function with self slicing effectively has
one argument, namely ``self``.
Analogously, a ``METH_O`` function with self slicing has two arguments.
@ -213,18 +217,19 @@ This is required to correctly deal with the ``LOAD_METHOD``/``CALL_METHOD`` opti
If ``func`` supports the C call protocol, then ``func.__get__``
must behave as follows:
- If ``cm_self`` is not NULL, then ``__get__`` must be a no-op
- If ``cr_self`` is not NULL, then ``__get__`` must be a no-op
in the sense that ``func.__get__(obj, cls)(*args, **kwds)``
behaves exactly the same as ``func(*args, **kwds)``.
It is also allowed for ``__get__`` to be not implemented at all.
- If ``cm_self`` is NULL, then ``func.__get__(obj, cls)(*args, **kwds)``
- If ``cr_self`` is NULL, then ``func.__get__(obj, cls)(*args, **kwds)``
(with ``obj`` not None)
must be equivalent to ``func(obj, *args, **kwds)``.
In particular, ``__get__`` must be implemented in this case.
Note that this is unrelated to self slicing: ``obj`` may be passed
as ``self`` argument to the C function or it may be the first positional argument.
- If ``cm_self`` is NULL, then ``func.__get__(None, cls)(*args, **kwds)``
- If ``cr_self`` is NULL, then ``func.__get__(None, cls)(*args, **kwds)``
must be equivalent to ``func(*args, **kwds)``.
There are no restrictions on the object ``func.__get__(obj, cls)``.
@ -232,7 +237,7 @@ The latter is not required to implement the C call protocol for example.
It only specifies what ``func.__get__(obj, cls).__call__`` does.
For classes that do not care about ``__self__`` and ``__get__`` at all,
the easiest solution is to assign ``cm_self = Py_None``
the easiest solution is to assign ``cr_self = Py_None``
(or any other non-NULL value).
Generic API functions
@ -257,6 +262,27 @@ The following C API functions are added:
In the latter case, the keyword values are stored in the ``args``
array, starting at ``args[nargs]``.
The following four functions are generic getters,
meant to be put into the ``tp_getset`` array:
- ``PyObject * PyCCall_GenericGetName(PyObject *func, void *ignored)``:
return ``cc_name`` for any instance supporting the C call protocol.
- ``PyObject * PyCCall_GenericGetParent(PyObject *func, void *ignored)``:
return ``cc_parent`` for any instance supporting the C call protocol.
Raise ``AttributeError`` if ``cc_parent`` is NULL.
- ``PyObject * PyCCall_GenericGetQualname(PyObject *func, void *ignored)``:
return a string suitable for using as ``__qualname__``.
This uses the ``__qualname__`` of ``cc_parent`` if possible.
Otherwise, this returns ``cc_name``.
- ``PyObject * PyCCall_GenericGetSelf(PyObject *func, void *ignored)``:
return ``cr_self`` for any instance supporting the C call protocol.
Raise ``AttributeError`` if ``cr_self`` is NULL.
None of the functions in this section is added to the stable ABI [#pep384]_.
Profiling
---------
@ -264,7 +290,7 @@ A flag ``CCALL_PROFILE`` is added to control profiling [#setprofile]_.
If this flag is set, then the profiling events
``c_call``, ``c_return`` and ``c_exception`` are generated.
When an unbound method is called
(``cm_self`` is NULL and ``CCALL_SLICE_SELF`` is set),
(``cr_self`` is NULL and ``CCALL_SLICE_SELF`` is set),
the argument to the profiling function is the corresponding bound method
(obtained by calling ``__get__``).
This is meant for backwards compatibility and to simplify
@ -277,37 +303,47 @@ Changes to built-in functions and methods
The reference implementation of this PEP changes
the existing classes ``builtin_function_or_method`` and ``method_descriptor``
to use the C call protocol.
In both cases, the ``PyCCallDef`` structure is simply stored
In fact, those two classes are almost merged:
the implementation becomes very similar, but they remain separate classes
(mostly for backwards compatibility).
The ``PyCCallDef`` structure is simply stored
as part of the object structure.
So, these are the new layouts of ``PyCFunctionObject`` and ``PyMethodDescrObject``::
Both classes use ``PyCFunctionObject`` as object structure.
This is the new layout::
typedef struct {
PyObject_HEAD
PyCCallDef *m_ccall;
PyObject *m_self;
PyObject *m_module;
PyObject *m_weakreflist;
PyCCallDef _ccalldef;
PyObject *m_module;
const char *m_doc;
PyObject *m_weakreflist;
} PyCFunctionObject;
typedef struct {
PyObject_HEAD
PyCCallDef *md_ccall;
PyObject *md_self; /* Always NULL */
PyObject *md_qualname;
PyCCallDef _ccalldef;
} PyMethodDescrObject;
For functions of a module, ``m_ccall`` would point to the ``_ccalldef``
field.
For bound methods, ``m_ccall`` would point to the ``PyCCallDef``
For functions of a module, ``m_ccall`` points to the ``_ccalldef`` field.
For bound methods, ``m_ccall`` points to the ``PyCCallDef``
of the unbound method.
**NOTE**: the new layout of ``PyMethodDescrObject`` changes it
**NOTE**: the new layout of ``method_descriptor`` changes it
such that it no longer starts with ``PyDescr_COMMON``.
This is really an implementation detail and it should cause few (if any)
compatibility problems.
C API functions
---------------
The following function is added:
- ``PyObject * PyCFunction_ClsNew(PyTypeObject *cls, PyMethodDef *ml, PyObject *self, PyObject *module, PyObject *parent)``:
create a new object with object structure ``PyCFunctionObject`` and class ``cls``.
This is called in turn by ``PyCFunction_NewEx`` and ``PyDescr_NewMethod``.
The undocumented functions ``PyCFunction_GetFlags``
and ``PyCFunction_GET_FLAGS``
are removed because it would be non-trivial to support them
in a backwards-compatible way.
Inheritance
===========
@ -327,8 +363,18 @@ There should be no difference at all for the Python interface,
and neither for the documented C API
(in the sense that all functions remain supported with the same functionality).
So the only potential breakage is with C code accessing the
internals of ``PyCFunctionObject`` and ``PyMethodDescrObject``.
The removed function ``PyCFunction_GetFlags``,
is officially part of the stable ABI [#pep384]_.
However, this is probably an oversight:
first of all, it is not even documented.
Second, the flag ``METH_FASTCALL``
is not part of the stable ABI but it is very common
(because of Argument Clinic).
So, if one cannot support ``METH_FASTCALL``,
it is hard to imagine a use case for ``PyCFunction_GetFlags``.
Concluding: the only potential breakage is with C code
which accesses the internals of ``PyCFunctionObject`` and ``PyMethodDescrObject``.
We expect very few problems because of this.
@ -369,16 +415,30 @@ However, the current protocol makes it easy to support the case
where the same C function is called for all instances:
just use a single static ``PyCCallDef`` structure for every instance.
Replacing tp_print
------------------
We re-purpose ``tp_print`` as ``tp_ccalloffset`` because this makes
it easier for external projects to backport the C call protocol
to earlier Python versions.
In particular, the Cython project has shown interest in doing that
(see https://mail.python.org/pipermail/python-dev/2018-June/153927.html).
Reference implementation
========================
Work in progress.
A draft implementation can be found at
https://github.com/jdemeyer/cpython/tree/pep580
References
==========
.. [#pep384] Löwis, PEP 384 – Defining a Stable ABI,
https://www.python.org/dev/peps/pep-0384/
.. [#setprofile] ``sys.setprofile`` documentation,
https://docs.python.org/3.8/library/sys.html#sys.setprofile