213 lines
6.8 KiB
ReStructuredText
213 lines
6.8 KiB
ReStructuredText
PEP: 743
|
|
Title: Add Py_COMPAT_API_VERSION to the Python C API
|
|
Author: Victor Stinner <vstinner@python.org>
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Created: 11-Mar-2024
|
|
Python-Version: 3.13
|
|
|
|
.. highlight:: c
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
Add ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX`` macros
|
|
to opt-in for planned incompatible C API changes in a C extension.
|
|
Maintainers can decide when they make their C extension compatible
|
|
and also decide which future Python version they want to be compatible
|
|
with.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
Python releases enforce C API changes
|
|
-------------------------------------
|
|
|
|
Every Python 3.x release has a long list of C API changes, including
|
|
incompatible changes. C extensions have to be updated to work on the
|
|
newly released Python.
|
|
|
|
Some incompatible changes are driven by new features: they cannot be
|
|
avoided, unless we decide to not add these features. Other reasons:
|
|
|
|
* Remove deprecated API (see :pep:`387`).
|
|
* Ease the implementation of another change.
|
|
* Change or remove error-prone API.
|
|
|
|
Currently, there is no middle ground between "not change the C API" and
|
|
"incompatible C API changes impact everybody". Either a C extension is
|
|
updated or the new Python version cannot be used. Such all-or-nothing
|
|
deal does not satisfy C extension maintainers nor C extensions users.
|
|
|
|
|
|
Limited C API
|
|
-------------
|
|
|
|
The limited C API is versioned: the ``Py_LIMITED_API`` macro can be set
|
|
to a Python version to select which API is available. On the Python
|
|
side, it allows introducing incompatible changes at a specific
|
|
``Py_LIMITED_API`` version. For example, if ``Py_LIMITED_API`` is set to
|
|
Python 3.11 or newer, the ``<stdio.h>`` is no longer included by
|
|
``Python.h``, whereas C extensions targeting Python 3.10 are not
|
|
affected.
|
|
|
|
The difference here is that upgrading Python does not change if
|
|
``<stdio.h>`` is included or not, but updating ``Py_LIMITED_API`` does.
|
|
Updating ``Py_LIMITED_API`` is an deliberate action made by the C
|
|
extension maintainer. It gives more freedom to decide **when** the
|
|
maintainer is ready to deal with the latest batch of incompatible
|
|
changes.
|
|
|
|
A similar version can be used with the regular (non-limited) C API.
|
|
|
|
|
|
Deprecation and compiler warnings
|
|
---------------------------------
|
|
|
|
Deprecated functions are marked with ``Py_DEPRECATED()``. Using a
|
|
deprecated function emits a compiler warning.
|
|
|
|
The problem is that ``pip`` and ``build`` tools hide compiler logs by
|
|
default, unless a build fails. Moreover, it's easy to miss a single
|
|
warning in the middle of hundred lines of logs.
|
|
|
|
Schedule changes
|
|
----------------
|
|
|
|
Currently, there is no way to schedule a C API change: announce it but
|
|
also provide a way to maintainers to test their C extensions with the
|
|
change. Either a change is not made, or everybody must update their code
|
|
if they want to update Python.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
New macros
|
|
----------
|
|
|
|
Add new ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX``
|
|
macros. They can be set to test if a C extension is prepared for future
|
|
C API changes: compatible with future Python versions.
|
|
|
|
The ``Py_COMPAT_API_VERSION`` macro can be set to a specific Python
|
|
version. For example, ``Py_COMPAT_API_VERSION=0x030e0000`` tests C API
|
|
changes scheduled in Python 3.14.
|
|
|
|
If the ``Py_COMPAT_API_VERSION`` macro is set to
|
|
``Py_COMPAT_API_VERSION_MAX``, all scheduled C API changes are tested at
|
|
once.
|
|
|
|
If the ``Py_COMPAT_API_VERSION`` macro is not set, it is to
|
|
``PY_VERSION_HEX`` by default.
|
|
|
|
The ``Py_COMPAT_API_VERSION`` macro can be set in a single C file or for
|
|
a whole project in compiler flags. The macro does not affected other
|
|
projects or Python itself.
|
|
|
|
|
|
Example in Python
|
|
-----------------
|
|
|
|
For example, the ``PyImport_ImportModuleNoBlock()`` function is
|
|
deprecated in Python 3.13 and scheduled for removal in Python 3.15. The
|
|
function can be declared in the Python C API with the following
|
|
declaration:
|
|
|
|
.. code-block:: c
|
|
|
|
#if Py_COMPAT_API_VERSION < 0x030f0000
|
|
Py_DEPRECATED(3.13) PyAPI_FUNC(PyObject *) PyImport_ImportModuleNoBlock(
|
|
const char *name /* UTF-8 encoded string */
|
|
);
|
|
#endif
|
|
|
|
If ``if Py_COMPAT_API_VERSION`` is equal to or greater than Python 3.15
|
|
(``0x030f0000``), the ``PyImport_ImportModuleNoBlock()`` function is not
|
|
declared, and so using it fails with a build error.
|
|
|
|
Goals
|
|
-----
|
|
|
|
* Reduce the number of C API changes affecting C extensions when
|
|
updating Python.
|
|
* When testing C extensions (for example, optional CI test),
|
|
``Py_COMPAT_API_VERSION`` can be set to ``Py_COMPAT_API_VERSION_MAX``
|
|
to detect future incompatibilities. For mandatory tests, it is
|
|
recommended to set ``Py_COMPAT_API_VERSION`` to a specific Python
|
|
version.
|
|
* For core developers, make sure that the C API can still evolve
|
|
without being afraid of breaking an unknown number of C extensions.
|
|
|
|
Non-goals
|
|
---------
|
|
|
|
* Freeze the API forever: this is not the stable ABI. For example,
|
|
deprecated functions will continue to be removed on a regular basis.
|
|
* C extensions maintainers not using ``Py_COMPAT_API_VERSION`` will
|
|
still be affected by C API changes when updating Python.
|
|
* Provide a stable ABI: the macro only impacts the regular (non-limited)
|
|
API.
|
|
* Silver bullet solving all C API issues.
|
|
|
|
|
|
Examples of ``Py_COMPAT_API_VERSION`` usages
|
|
============================================
|
|
|
|
* Remove deprecated functions.
|
|
* Remove deprecated structure members, such as
|
|
``PyBytesObject.ob_shash``.
|
|
* Remove a standard ``#include``, such as ``#include <string.h>``,
|
|
from ``<Python.h>``.
|
|
* Change the behavior of a function or a macro. For example, calling
|
|
``PyObject_SetAttr(obj, name, NULL)`` can fail, to enforce the usage
|
|
of the ``PyObject_DelAttr()`` function instead to delete an attribute.
|
|
|
|
|
|
Implementation
|
|
==============
|
|
|
|
* `Issue gh-116587 <https://github.com/python/cpython/issues/116587>`_
|
|
* PR: `Add Py_COMPAT_API_VERSION and Py_COMPAT_API_VERSION_MAX macros
|
|
<https://github.com/python/cpython/pull/116588>`_
|
|
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
There is no impact on backward compatibility.
|
|
|
|
Adding ``Py_COMPAT_API_VERSION`` and ``Py_COMPAT_API_VERSION_MAX``
|
|
macros has no effect on backward compatibility. Only developers setting
|
|
the ``Py_COMPAT_API_VERSION`` macro in their project will be impacted by
|
|
effects of this macro which is the expected behavior.
|
|
|
|
|
|
Discussions
|
|
===========
|
|
|
|
* C API Evolutions: `Macro to hide deprecated functions
|
|
<https://github.com/capi-workgroup/api-evolution/issues/24>`_
|
|
(October 2023)
|
|
* C API Problems: `Opt-in macro for a new clean API? Subset of functions
|
|
with no known issues
|
|
<https://github.com/capi-workgroup/problems/issues/54>`_
|
|
(June 2023)
|
|
|
|
|
|
Prior Art
|
|
=========
|
|
|
|
* ``Py_LIMITED_API`` macro of :pep:`384` "Defining a Stable ABI".
|
|
* Rejected :pep:`606` "Python Compatibility Version" which has a global
|
|
scope.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|