PEP 743: Add Py_COMPAT_API_VERSION to the Python C API (#3715)
This commit is contained in:
Victor Stinner
GitHub
parent
e749257eb8
commit
17cd307bd1
|
|
@ -621,6 +621,7 @@ peps/pep-0738.rst @encukou
|
|||
peps/pep-0740.rst @dstufft
|
||||
peps/pep-0741.rst @vstinner
|
||||
peps/pep-0742.rst @JelleZijlstra
|
||||
peps/pep-0743.rst @vstinner
|
||||
# ...
|
||||
# peps/pep-0754.rst
|
||||
# ...
|
||||
|
|
|
|||
|
|
@ -0,0 +1,212 @@
|
|||
PEP: 743
|
||||
Title: Add Py_COMPAT_API_VERSION to the Python C API
|
||||
Author: Victor Stinner <vstinner@python.org>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Created: 11-Mar-2024
|
||||
Python-Version: 3.13
|
||||
|
||||
.. highlight:: c
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
Add ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX`` macros
|
||||
to opt-in for planned incompatible C API changes in a C extension.
|
||||
Maintainers can decide when they make their C extension compatible
|
||||
and also decide which future Python version they want to be compatible
|
||||
with.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
Python releases enforce C API changes
|
||||
-------------------------------------
|
||||
|
||||
Every Python 3.x release has a long list of C API changes, including
|
||||
incompatible changes. C extensions have to be updated to work on the
|
||||
newly released Python.
|
||||
|
||||
Some incompatible changes are driven by new features: they cannot be
|
||||
avoided, unless we decide to not add these features. Other reasons:
|
||||
|
||||
* Remove deprecated API (see :pep:`387`).
|
||||
* Ease the implementation of another change.
|
||||
* Change or remove error-prone API.
|
||||
|
||||
Currently, there is no middle ground between "not change the C API" and
|
||||
"incompatible C API changes impact everybody". Either a C extension is
|
||||
updated or the new Python version cannot be used. Such all-or-nothing
|
||||
deal does not satisfy C extension maintainers nor C extensions users.
|
||||
|
||||
|
||||
Limited C API
|
||||
-------------
|
||||
|
||||
The limited C API is versioned: the ``Py_LIMITED_API`` macro can be set
|
||||
to a Python version to select which API is available. On the Python
|
||||
side, it allows introducing incompatible changes at a specific
|
||||
``Py_LIMITED_API`` version. For example, if ``Py_LIMITED_API`` is set to
|
||||
Python 3.11 or newer, the ``<stdio.h>`` is no longer included by
|
||||
``Python.h``, whereas C extensions targeting Python 3.10 are not
|
||||
affected.
|
||||
|
||||
The difference here is that upgrading Python does not change if
|
||||
``<stdio.h>`` is included or not, but updating ``Py_LIMITED_API`` does.
|
||||
Updating ``Py_LIMITED_API`` is an deliberate action made by the C
|
||||
extension maintainer. It gives more freedom to decide **when** the
|
||||
maintainer is ready to deal with the latest batch of incompatible
|
||||
changes.
|
||||
|
||||
A similar version can be used with the regular (non-limited) C API.
|
||||
|
||||
|
||||
Deprecation and compiler warnings
|
||||
---------------------------------
|
||||
|
||||
Deprecated functions are marked with ``Py_DEPRECATED()``. Using a
|
||||
deprecated function emits a compiler warning.
|
||||
|
||||
The problem is that ``pip`` and ``build`` tools hide compiler logs by
|
||||
default, unless a build fails. Moreover, it's easy to miss a single
|
||||
warning in the middle of hundred lines of logs.
|
||||
|
||||
Schedule changes
|
||||
----------------
|
||||
|
||||
Currently, there is no way to schedule a C API change: announce it but
|
||||
also provide a way to maintainers to test their C extensions with the
|
||||
change. Either a change is not made, or everybody must update their code
|
||||
if they want to update Python.
|
||||
|
||||
|
||||
Specification
|
||||
=============
|
||||
|
||||
New macros
|
||||
----------
|
||||
|
||||
Add new ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX``
|
||||
macros. They can be set to test if a C extension is prepared for future
|
||||
C API changes: compatible with future Python versions.
|
||||
|
||||
The ``Py_COMPAT_API_VERSION`` macro can be set to a specific Python
|
||||
version. For example, ``Py_COMPAT_API_VERSION=0x030e0000`` tests C API
|
||||
changes scheduled in Python 3.14.
|
||||
|
||||
If the ``Py_COMPAT_API_VERSION`` macro is set to
|
||||
``Py_COMPAT_API_VERSION_MAX``, all scheduled C API changes are tested at
|
||||
once.
|
||||
|
||||
If the ``Py_COMPAT_API_VERSION`` macro is not set, it is to
|
||||
``PY_VERSION_HEX`` by default.
|
||||
|
||||
The ``Py_COMPAT_API_VERSION`` macro can be set in a single C file or for
|
||||
a whole project in compiler flags. The macro does not affected other
|
||||
projects or Python itself.
|
||||
|
||||
|
||||
Example in Python
|
||||
-----------------
|
||||
|
||||
For example, the ``PyImport_ImportModuleNoBlock()`` function is
|
||||
deprecated in Python 3.13 and scheduled for removal in Python 3.15. The
|
||||
function can be declared in the Python C API with the following
|
||||
declaration:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
#if Py_COMPAT_API_VERSION < 0x030f0000
|
||||
Py_DEPRECATED(3.13) PyAPI_FUNC(PyObject *) PyImport_ImportModuleNoBlock(
|
||||
const char *name /* UTF-8 encoded string */
|
||||
);
|
||||
#endif
|
||||
|
||||
If ``if Py_COMPAT_API_VERSION`` is equal to or greater than Python 3.15
|
||||
(``0x030f0000``), the ``PyImport_ImportModuleNoBlock()`` function is not
|
||||
declared, and so using it fails with a build error.
|
||||
|
||||
Goals
|
||||
-----
|
||||
|
||||
* Reduce the number of C API changes affecting C extensions when
|
||||
updating Python.
|
||||
* When testing C extensions (for example, optional CI test),
|
||||
``Py_COMPAT_API_VERSION`` can be set to ``Py_COMPAT_API_VERSION_MAX``
|
||||
to detect future incompatibilities. For mandatory tests, it is
|
||||
recommended to set ``Py_COMPAT_API_VERSION`` to a specific Python
|
||||
version.
|
||||
* For core developers, make sure that the C API can still evolve
|
||||
without being afraid of breaking an unknown number of C extensions.
|
||||
|
||||
Non-goals
|
||||
---------
|
||||
|
||||
* Freeze the API forever: this is not the stable ABI. For example,
|
||||
deprecated functions will continue to be removed on a regular basis.
|
||||
* C extensions maintainers not using ``Py_COMPAT_API_VERSION`` will
|
||||
still be affected by C API changes when updating Python.
|
||||
* Provide a stable ABI: the macro only impacts the regular (non-limited)
|
||||
API.
|
||||
* Silver bullet solving all C API issues.
|
||||
|
||||
|
||||
Examples of ``Py_COMPAT_API_VERSION`` usages
|
||||
============================================
|
||||
|
||||
* Remove deprecated functions.
|
||||
* Remove deprecated structure members, such as
|
||||
``PyBytesObject.ob_shash``.
|
||||
* Remove a standard ``#include``, such as ``#include <string.h>``,
|
||||
from ``<Python.h>``.
|
||||
* Change the behavior of a function or a macro. For example, calling
|
||||
``PyObject_SetAttr(obj, name, NULL)`` can fail, to enforce the usage
|
||||
of the ``PyObject_DelAttr()`` function instead to delete an attribute.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
* `Issue gh-116587 <https://github.com/python/cpython/issues/116587>`_
|
||||
* PR: `Add Py_COMPAT_API_VERSION and Py_COMPAT_API_VERSION_MAX macros
|
||||
<https://github.com/python/cpython/pull/116588>`_
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
There is no impact on backward compatibility.
|
||||
|
||||
Adding ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX``
|
||||
macros has no effect on backward compatibility. Only developers setting
|
||||
the ``Py_COMPAT_API_VERSION`` macro in their project will be impacted by
|
||||
effects of this macro which is the expected behavior.
|
||||
|
||||
|
||||
Discussions
|
||||
===========
|
||||
|
||||
* C API Evolutions: `Macro to hide deprecated functions
|
||||
<https://github.com/capi-workgroup/api-evolution/issues/24>`_
|
||||
(October 2023)
|
||||
* C API Problems: `Opt-in macro for a new clean API? Subset of functions
|
||||
with no known issues
|
||||
<https://github.com/capi-workgroup/problems/issues/54>`_
|
||||
(June 2023)
|
||||
|
||||
|
||||
Prior Art
|
||||
=========
|
||||
|
||||
* ``Py_LIMITED_API`` macro of :pep:`384` "Defining a Stable ABI".
|
||||
* Rejected :pep:`606` "Python Compatibility Version" which has a global
|
||||
scope.
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document is placed in the public domain or under the
|
||||
CC0-1.0-Universal license, whichever is more permissive.
|
||||
Loading…
Reference in New Issue