PEP 674: Backward compatibility (#2170)

pep-0674.rst

@@ -10,12 +10,18 @@ Python-Version: 3.11
 Abstract
 ========
 
-Incompatible C API change disallowing using macros as l-value to allow
-evolving CPython internals and to ease the C API implementation on other
-Python implementation.
+Incompatible C API change disallowing using macros as l-value to:
 
-In practice, the majority of projects impacted by these incompatible
-changes should only have to make two changes:
+* Allow evolving CPython internals;
+* Ease the C API implementation on other Python implementation;
+* Help migrating existing C extensions to the HPy API.
+
+On the PyPI top 5000 projects, only 14 projects (0.3%) are affected by 4
+macro changes. Moreover, 24 projects just have to regenerate their
+Cython code to use ``Py_SET_TYPE()``.
+
+In practice, the majority of affected projects only have to make two
+changes:
 
 * Replace ``Py_TYPE(obj) = new_type;``
   with ``Py_SET_TYPE(obj, new_type);``.
@@ -194,8 +200,8 @@ PyDescr macros
 Port C extensions to Python 3.11
 --------------------------------
 
-In practice, the majority of projects impacted by these PEP incompatible
-changes should only have to make two changes:
+In practice, the majority of projects affected by these PEP only have to
+make two changes:
 
 * Replace ``Py_TYPE(obj) = new_type;``
   with ``Py_SET_TYPE(obj, new_type);``.
@@ -219,43 +225,47 @@ The code pattern ``&PyTuple_GET_ITEM(tuple, 0)`` and
 ``&PyList_GET_ITEM(list, 0)`` is still commonly used to get access to
 the inner ``PyObject**`` array.
 
-Changing these macros would require to add a new API to get access to
-the inner array which is out of the scope of this PEP.
+Changing these macros is out of the scope of this PEP.
 
 
 Backwards Compatibility
 =======================
 
-The proposed C API changes are backward incompatible on purpose.  In
-practice, only a minority of third party projects are affected (16
-projects are known to be broken) and `most of them have already been
-updated for these changes
-<https://bugs.python.org/issue39573#msg401378>`__ (12 on 16).
+The proposed C API changes are backward incompatible on purpose.
 
-Most projects are broken by ``Py_TYPE()`` and ``Py_SIZE()`` changes.
-These two macros have been converted to static inline macro in Python
-3.10 alpha versions, but the change had to be reverted since it broke
-too many projects. In the meanwhile, many projects, like Cython, have
-been prepared for this change by using ``Py_SET_TYPE()`` and
-``Py_SET_SIZE()``. For example, projects using Cython only have to
-regenerate their outdated generated C code to become compatible.
+At December 1, 2021, a code search on the PyPI top 5000 projects (4760
+projects in practice, others don't have a source achive) found that
+`only 14 projects are affected
+<https://bugs.python.org/issue45476#msg407456>`_ (0.3%):
 
-For the "GET" functions like ``PyDict_GET_SIZE()``, no project in the PyPI
-top 5000 projects use these functions as l-value.
+* datatable (1.0.0)
+* frozendict (2.1.1)
+* guppy3 (3.1.2)
+* M2Crypto (0.38.0)
+* mecab (python3 (1.0.4)
+* mypy (0.910)
+* Naked (0.1.31)
+* pickle5 (0.0.12)
+* pysha3 (1.0.2)
+* python-snappy (0.6.0)
+* recordclass (0.16.3)
+* scipy (1.7.3)
+* zodbpickle (2.2.0)
+* zstd (1.5.0.2)
 
-The ``PyFloat_AS_DOUBLE()`` function is not used as a l-value in the
-PyPI top 5000 projects.
+These 14 projects only use 4 macros as l-value:
 
-The ``PyBytes_AS_STRING()`` and ``PyByteArray_AS_STRING()`` are used as
-l-value but only to modify string characters, not to override the
-``PyBytesObject.ob_sval`` or ``PyByteArrayObject.ob_start`` member.
-For example, Cython uses the following code which remains valid::
+* ``PyDescr_NAME()`` and ``PyDescr_TYPE()`` (2 projects)
+* ``Py_SIZE()`` (8 projects)
+* ``Py_TYPE()`` (4 projects)
 
-    PyByteArray_AS_STRING(string)[i] = (char) v;
+Moreover, `24 projects just have to regenerate their Cython code
+<https://bugs.python.org/issue45476#msg407416>`_ to use
+``Py_SET_TYPE()``.
 
 This change does not follow the PEP 387 deprecation process. There is no
-known way to emit a deprecation warning when a macro is used as a
-l-value, but not when it's used differently (ex: r-value).
+known way to emit a deprecation warning only when a macro is used as a
+l-value, but not when it's used differently (ex: as a r-value).
 
 
 Rejected Idea: Leave the macros as they are