PEP 674: List fixed projects; change title (#2284)
This commit is contained in:
Victor Stinner
GitHub
parent
e647fcedbd
commit
61e93c33be
194
pep-0674.rst
194
pep-0674.rst
|
|
@ -1,5 +1,5 @@
|
|||
PEP: 674
|
||||
Title: Disallow using macros as an l-value
|
||||
Title: Disallow using Py_TYPE() and Py_SIZE() macros as l-values
|
||||
Author: Victor Stinner <vstinner@python.org>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
|
|
@ -10,23 +10,24 @@ Python-Version: 3.11
|
|||
Abstract
|
||||
========
|
||||
|
||||
Incompatible C API change disallowing using macros as an l-value to:
|
||||
Incompatible C API change disallowing using macros, especially
|
||||
``Py_TYPE()`` and ``Py_SIZE()``, as l-values to:
|
||||
|
||||
* Allow evolving CPython internals;
|
||||
* Allow evolving CPython internals (change the ``PyObject`` structure);
|
||||
* Ease the C API implementation on other Python implementation;
|
||||
* Help migrating existing C extensions to the HPy API.
|
||||
|
||||
Only 9 out of the top 5000 PyPI projects (0.2%) are affected by 2 macro
|
||||
changes. An additional 22 projects just have to regenerate their Cython
|
||||
Only 7 out of the top 5000 PyPI projects (0.1%) are affected by this
|
||||
PEP. An additional 22 projects just have to regenerate their Cython
|
||||
code.
|
||||
|
||||
In practice, the majority of affected projects only have to make two
|
||||
changes:
|
||||
|
||||
* Replace ``Py_TYPE(obj) = new_type;``
|
||||
with ``Py_SET_TYPE(obj, new_type);``.
|
||||
* Replace ``Py_SIZE(obj) = new_size;``
|
||||
with ``Py_SET_SIZE(obj, new_size);``.
|
||||
* Replace ``Py_TYPE(obj) = new_type``
|
||||
with ``Py_SET_TYPE(obj, new_type)``.
|
||||
* Replace ``Py_SIZE(obj) = new_size``
|
||||
with ``Py_SET_SIZE(obj, new_size)``.
|
||||
|
||||
|
||||
Rationale
|
||||
|
|
@ -119,8 +120,10 @@ These statements are endorsed by Tim Felgentreff (GraalVM Python developer).
|
|||
Specification
|
||||
=============
|
||||
|
||||
Disallow using macros as an l-value
|
||||
-----------------------------------
|
||||
Disallow using macros as l-values
|
||||
----------------------------------
|
||||
|
||||
The following 65 macros are modified to disallow using them as l-values.
|
||||
|
||||
PyObject and PyVarObject macros
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
|
@ -128,8 +131,8 @@ PyObject and PyVarObject macros
|
|||
* ``Py_TYPE()``: ``Py_SET_TYPE()`` must be used instead
|
||||
* ``Py_SIZE()``: ``Py_SET_SIZE()`` must be used instead
|
||||
|
||||
"GET" macros
|
||||
^^^^^^^^^^^^
|
||||
GET macros
|
||||
^^^^^^^^^^
|
||||
|
||||
* ``PyByteArray_GET_SIZE()``
|
||||
* ``PyBytes_GET_SIZE()``
|
||||
|
|
@ -162,8 +165,8 @@ PyObject and PyVarObject macros
|
|||
* ``PyUnicode_GET_SIZE()``
|
||||
* ``PyWeakref_GET_OBJECT()``
|
||||
|
||||
"AS" macros
|
||||
^^^^^^^^^^^
|
||||
AS macros
|
||||
^^^^^^^^^
|
||||
|
||||
* ``PyByteArray_AS_STRING()``
|
||||
* ``PyBytes_AS_STRING()``
|
||||
|
|
@ -185,8 +188,8 @@ PyUnicode macros
|
|||
* ``PyUnicode_READ()``
|
||||
* ``PyUnicode_READ_CHAR()``
|
||||
|
||||
PyDateTime "GET" macros
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
PyDateTime GET macros
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* ``PyDateTime_DATE_GET_FOLD()``
|
||||
* ``PyDateTime_DATE_GET_HOUR()``
|
||||
|
|
@ -213,10 +216,10 @@ Port C extensions to Python 3.11
|
|||
In practice, the majority of projects affected by these PEP only have to
|
||||
make two changes:
|
||||
|
||||
* Replace ``Py_TYPE(obj) = new_type;``
|
||||
with ``Py_SET_TYPE(obj, new_type);``.
|
||||
* Replace ``Py_SIZE(obj) = new_size;``
|
||||
with ``Py_SET_SIZE(obj, new_size);``.
|
||||
* Replace ``Py_TYPE(obj) = new_type``
|
||||
with ``Py_SET_TYPE(obj, new_type)``.
|
||||
* Replace ``Py_SIZE(obj) = new_size``
|
||||
with ``Py_SET_SIZE(obj, new_size)``.
|
||||
|
||||
The `pythoncapi_compat project
|
||||
<https://github.com/pythoncapi/pythoncapi_compat>`_ can be used to
|
||||
|
|
@ -225,37 +228,6 @@ losing support with older Python versions. The project provides a header
|
|||
file which provides ``Py_SET_REFCNT()``, ``Py_SET_TYPE()`` and
|
||||
``Py_SET_SIZE()`` functions to Python 3.8 and older.
|
||||
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
The proposed C API changes are backward incompatible on purpose.
|
||||
|
||||
On January 26, 2022, a code search on the top 5000 PyPI projects (4762
|
||||
projects in practice; others don't have a source archive) found that
|
||||
only 9 projects are affected (0.2%):
|
||||
|
||||
* datatable (1.0.0)
|
||||
* guppy3 (3.1.2)
|
||||
* pickle5 (0.0.12)
|
||||
* psycopg2 (2.9.3)
|
||||
* pysha3 (1.0.2)
|
||||
* python-snappy (0.6.0)
|
||||
* recordclass (0.17.1)
|
||||
* scipy (1.7.3)
|
||||
* zodbpickle (2.2.0)
|
||||
|
||||
Of these 9 projects, only 2 macros are used as an l-value:
|
||||
``Py_TYPE()`` and ``Py_SIZE()``.
|
||||
|
||||
An additional 22 projects just have to regenerate their Cython code to
|
||||
use ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``.
|
||||
|
||||
This change does not follow the :pep:`387` deprecation process. There is
|
||||
no known way to emit a deprecation warning only when a macro is used as
|
||||
an l-value, but not when it's used differently (ex: as a r-value).
|
||||
|
||||
PyTuple_GET_ITEM() and PyList_GET_ITEM() are left unchanged
|
||||
-----------------------------------------------------------
|
||||
|
||||
|
|
@ -288,6 +260,121 @@ the PyDescrObject structure opaque: PyDescr_NAME() and PyDescr_TYPE()"
|
|||
issue for more details.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
The implementation is tracked by `bpo-45476: [C API] PEP 674: Disallow
|
||||
using macros as l-values <https://bugs.python.org/issue45476>`_.
|
||||
|
||||
Py_TYPE() and Py_SIZE() macros
|
||||
------------------------------
|
||||
|
||||
In May 2020, the ``Py_TYPE()`` and ``Py_SIZE()`` macros have been
|
||||
modified to disallow using them as l-values (`Py_TYPE
|
||||
<https://github.com/python/cpython/commit/ad3252bad905d41635bcbb4b76db30d570cf0087>`_,
|
||||
`Py_SIZE
|
||||
<https://github.com/python/cpython/commit/fe2978b3b940fe2478335e3a2ca5ad22338cdf9c>`_).
|
||||
|
||||
In November 2020, the change was `reverted
|
||||
<https://github.com/python/cpython/commit/0e2ac21dd4960574e89561243763eabba685296a>`__,
|
||||
since it broke too many third party projects.
|
||||
|
||||
In June 2021, once most third party projects were updated, a `second
|
||||
attempt
|
||||
<https://github.com/python/cpython/commit/f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970>`_
|
||||
was done, but had to be `reverted again
|
||||
<https://github.com/python/cpython/commit/6d518bb3a11f9b16098f45b21a13ebe8f537f045>`__
|
||||
, since it broke test_exceptions on Windows.
|
||||
|
||||
In September 2021, once `test_exceptions has been fixed
|
||||
<https://github.com/python/cpython/commit/fb305092a5d7894b41f122c1a1117b3abf4c567e>`_,
|
||||
Py_TYPE() and Py_SIZE() were finally `changed
|
||||
<https://github.com/python/cpython/commit/f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970>`_.
|
||||
|
||||
In November 2021, this backward incompatible change got a
|
||||
`Steering Council exception
|
||||
<https://github.com/python/steering-council/issues/79#issuecomment-981153173>`_.
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
The proposed C API changes are backward incompatible on purpose.
|
||||
|
||||
This change does not follow the :pep:`387` deprecation process. There is
|
||||
no known way to emit a deprecation warning only when a macro is used as
|
||||
an l-value, but not when it's used differently (ex: as a r-value).
|
||||
|
||||
The following 4 macros are left unchanged to reduce the number of
|
||||
affected projects: ``PyDescr_NAME()``, ``PyDescr_TYPE()``,
|
||||
``PyList_GET_ITEM()`` and ``PyTuple_GET_ITEM()``.
|
||||
|
||||
Top 5000 PyPI
|
||||
-------------
|
||||
|
||||
On January 27, 2022, a code search on the top 5000 PyPI projects found
|
||||
that only 7 projects are affected (0.1%):
|
||||
|
||||
* datatable (1.0.0):
|
||||
`pending PR <https://github.com/h2oai/datatable/pull/3231>`__
|
||||
* guppy3 (3.1.2): `fixed <https://github.com/zhuyifei1999/guppy3/commit/4cb9fcb5d75327544a6875b6caabfdffb70a7e29>`__
|
||||
* pickle5 (0.0.12): backport for Python <= 3.7
|
||||
* pysha3 (1.0.2): backport for Python <= 3.5
|
||||
* python-snappy (0.6.0):
|
||||
`fixed <https://github.com/andrix/python-snappy/commit/1a539d71d5b1ceaf9a2291f21f686cf53a46d707>`__
|
||||
* recordclass (0.17.1):
|
||||
`fixed <https://bitbucket.org/intellimath/recordclass/commits/d20d72fa3cdbdcf96c72941560041460adeecff1>`__
|
||||
* scipy (1.7.3): fixed (need to update boost)
|
||||
|
||||
All of those projects are fixed (but not released), have a pending fix,
|
||||
or should not be fixed (backports).
|
||||
|
||||
An additional 23 projects only have to regenerate their Cython code to
|
||||
use ``Py_SET_TYPE()`` and ``Py_SET_SIZE()``.
|
||||
|
||||
Only 2 macros are used as an l-value: ``Py_TYPE()`` and ``Py_SIZE()``.
|
||||
|
||||
Projects already fixed
|
||||
----------------------
|
||||
|
||||
Fixed projects but not released yet (4):
|
||||
|
||||
* SWIG:
|
||||
`commit <https://github.com/swig/swig/commit/7246cfa6c6e1877cd679a23970da477cad039a28>`__
|
||||
* M2Crypto: need SWIG release
|
||||
* mecab-python3: need SWIG release
|
||||
|
||||
Fixed projects with a release (13):
|
||||
|
||||
* bitarray (1.6.2):
|
||||
`commit <https://github.com/ilanschnell/bitarray/commit/a0cca9f2986ec796df74ca8f42aff56c4c7103ba>`__
|
||||
* boost (1.78.0): `commit
|
||||
<https://github.com/boostorg/python/commit/500194edb7833d0627ce7a2595fec49d0aae2484>`__
|
||||
* breezy (3.2.1): `bug report <https://bugs.launchpad.net/brz/+bug/1904868>`__
|
||||
* Cython (0.29.20): `commit <https://github.com/cython/cython/commit/d8e93b332fe7d15459433ea74cd29178c03186bd>`__
|
||||
* duplicity (0.8.18):
|
||||
`commit <https://git.launchpad.net/duplicity/commit/duplicity/_librsyncmodule.c?id=bbaae91b5ac6ef7e295968e508522884609fbf84>`__
|
||||
* gobject-introspection (1.70.0):
|
||||
`MR <https://gitlab.gnome.org/GNOME/gobject-introspection/-/merge_requests/243>`__
|
||||
* immutables (0.15):
|
||||
`commit <https://github.com/MagicStack/immutables/commit/45105ecd8b56a4d88dbcb380fcb8ff4b9cc7b19c>`__
|
||||
* mercurial (5.7):
|
||||
`commit <https://www.mercurial-scm.org/repo/hg/rev/e92ca942ddca>`__,
|
||||
`bug report <https://bz.mercurial-scm.org/show_bug.cgi?id=6451>`__
|
||||
* mypy (v0.930):
|
||||
`commit <https://github.com/python/mypy/commit/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa>`__
|
||||
* numpy (1.22.1):
|
||||
`commit <https://github.com/numpy/numpy/commit/a96b18e3d4d11be31a321999cda4b795ea9eccaa>`__,
|
||||
`commit 2 <https://github.com/numpy/numpy/commit/f1671076c80bd972421751f2d48186ee9ac808aa>`__
|
||||
* pycurl (7.44.1):
|
||||
`commit <https://github.com/pycurl/pycurl/commit/e633f9a1ac4df5e249e78c218d5fbbd848219042>`__
|
||||
* PyGObject (3.42.0)
|
||||
* pyside2 (5.15.1):
|
||||
`bug report <https://bugreports.qt.io/browse/PYSIDE-1436>`__
|
||||
* python-zstd (1.5.0.3):
|
||||
`commit <https://github.com/sergey-dryabzhinsky/python-zstd/commit/8aa6d7a4b250e1f0a4e27b4107c39dc516c87f96>`__
|
||||
|
||||
|
||||
Relationship with the HPy project
|
||||
=================================
|
||||
|
||||
|
|
@ -419,9 +506,6 @@ References
|
|||
* `Python C API: Add functions to access PyObject
|
||||
<https://vstinner.github.io/c-api-abstract-pyobject.html>`_ (October
|
||||
2021) article by Victor Stinner
|
||||
* `[C API] Disallow using PyFloat_AS_DOUBLE() as l-value
|
||||
<https://bugs.python.org/issue45476>`_
|
||||
(October 2021)
|
||||
* `[capi-sig] Py_TYPE() and Py_SIZE() become static inline functions
|
||||
<https://mail.python.org/archives/list/capi-sig@python.org/thread/WGRLTHTHC32DQTACPPX36TPR2GLJAFRB/>`_
|
||||
(September 2021)
|
||||
|
|
|
|||
Loading…
Reference in New Issue