python-peps/peps/pep-0741.rst

PEP: 741
Title: Python Configuration C API
Author: Victor Stinner <vstinner@python.org>
Discussions-To: https://discuss.python.org/t/pep-741-python-configuration-c-api-second-version/45403
Status: Draft
Type: Standards Track
Created: 18-Jan-2024
Python-Version: 3.14
Post-History: `19-Jan-2024 <https://discuss.python.org/t/pep-741-python-configuration-c-api/43637>`__,
              `08-Feb-2024 <https://discuss.python.org/t/pep-741-python-configuration-c-api-second-version/45403>`__,

Abstract
========

Add a C API to configure the Python initialization without relying on C
structures and the ability to make ABI-compatible changes in the future.

Complete :pep:`587` API by adding ``PyInitConfig_AddModule()`` which can be
used to add a built-in extension module; feature previously referred to
as the "inittab".

Add ``PyConfig_Get()`` and ``PyConfig_Set()`` functions to
get and set the current runtime configuration.

:pep:`587` "Python Initialization Configuration" unified all the ways to
configure the Python **initialization**. This PEP unifies also the
configuration of the Python **preinitialization** and the Python
**initialization** in a single API. Moreover, this PEP only provides a
single choice to embed Python, instead of having two "Python" and
"Isolated" choices (:pep:`587`), to simplify the API further.

The lower level :pep:`587` ``PyConfig`` API remains available for use
cases with an intentionally higher level of coupling to CPython
implementation details (such as emulating the full functionality of
CPython's CLI, including its configuration mechanisms).


Rationale
=========

Get the runtime configuration
-----------------------------

:pep:`587` has no API to **get** the **current** runtime configuration,
only to **configure** the Python **initialization**.

For example, the global configuration variable
``Py_UnbufferedStdioFlag`` was deprecated in Python 3.12 and using
``PyConfig.buffered_stdio`` is recommended instead. It only works to
configure Python, there is no public API to get
``PyConfig.buffered_stdio``.

Users of the limited C API are asking for a public API to get the
current runtime configuration.

Cython needs to get the ``optimization_level`` configuration option:
`issue <https://github.com/python/cpython/issues/99872>`_.

When global configuration variables were deprecated in 2022, `Marc-André
Lemburg requested
<https://github.com/python/cpython/issues/93103#issuecomment-1136462708>`__
a C API to access these configuration variables at runtime (not only
during Python initialization).


Security fix
------------

To fix `CVE-2020-10735
<https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10735>`_,
a denial-of-service when converting a very large string to an integer (in base
10), it was discussed to add a new ``PyConfig`` member to stable
branches which affects the ABI.

Gregory P. Smith proposed a different API using text based configuration
file to not be limited by ``PyConfig`` members: `FR: Allow private
runtime config to enable extending without breaking the PyConfig ABI
<https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004>`__
(August 2022).

In the end, it was decided to not add a new ``PyConfig`` member to
stable branches, but only add a new ``PyConfig.int_max_str_digits``
member to the development branch (which became Python 3.12). A dedicated
private global variable (unrelated to ``PyConfig``) is used in stable
branches.


Redundancy between PyPreConfig and PyConfig
-------------------------------------------

The Python preinitialization uses the ``PyPreConfig`` structure and the
Python initialization uses the ``PyConfig`` structure. Both structures
have four duplicated members: ``dev_mode``, ``parse_argv``, ``isolated``
and ``use_environment``.

The redundancy is caused by the fact that the two structures are
separated, whereas some ``PyConfig`` members are needed by the
preinitialization.


Embedding Python
----------------

Applications embedding Python
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Examples:

* `Blender 3D graphics <https://www.blender.org/>`_.
* `fontforge <https://fontforge.org/>`_ font editor.
* `Gimp <https://www.gimp.org/>`_.
* `LibreOffice <https://www.libreoffice.org/>`_.
* `OBS Studio <https://obsproject.com/>`_.
* `Tiled <https://www.mapeditor.org/>`_.
* `vim <https://www.vim.org/>`_ text editor.

On Linux, FreeBSD and macOS, applications are usually either statically
linked to a ``libpython``, or load dynamically a ``libpython`` . The
``libpython`` shared library is versioned, example:
``libpython3.12.so`` for Python 3.12 on Linux.

The vim project can target the stable ABI. Usually, the "system Python"
version is used. It's not currently possible to select which Python
version to use. Users would like the ability to select a newer Python
on demand.

On Linux, another approach to deploy an application embedding Python,
such as GIMP, is to include Python in Flatpack, AppImage or Snap
"container". In this case, the application brings its own copy of Python
version with the container.

Libraries embedding Python
^^^^^^^^^^^^^^^^^^^^^^^^^^

Examples:

* `Apache mod_wsgi <https://modwsgi.readthedocs.io/>`_
  (`source <https://github.com/GrahamDumpleton/mod_wsgi/blob/f54eadd6da8e3da0faccd497d4165de435b97242/src/server/wsgi_interp.c#L2367-L2404>`__).
* `nimpy <https://github.com/yglukhov/nimpy>`_:
  Nim - Python bridge.
* `PyO3 <https://github.com/PyO3/pyo3>`__:
  Rust bindings for the Python interpreter.

Utilities creating standalone applications
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* `py2app <https://py2app.readthedocs.io/>`_ for macOS.
* `py2exe <http://www.py2exe.org/>`_ for Windows.
* `pyinstaller <https://pyinstaller.org/>`_.
* `PyOxidizer <https://github.com/indygreg/PyOxidizer>`_:
  it uses the PEP 587 PyConfig API.

These utilities create standalone applications, they are not linked to
libpython.

Set the runtime configuration
-----------------------------

`Marc-André Lemburg requested
<https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004/34>`__
a C API to **set** the value of some configuration options at runtime:

* ``optimization_level``
* ``verbose``
* ``parser_debug``
* ``inspect``
* ``write_bytecode``

Previously, it was possible to set directly global configuration
variables:

* ``Py_OptimizeFlag``
* ``Py_VerboseFlag``
* ``Py_DebugFlag``
* ``Py_InspectFlag``
* ``Py_DontWriteBytecodeFlag``

But these configuration flags were deprecated in Python 3.12 and are
scheduled for removal in Python 3.14.


Specification
=============

Add C API functions and structure to configure the Python
initialization:

* Create config:

  * ``PyInitConfig`` opaque structure.
  * ``PyInitConfig_Create()``.
  * ``PyInitConfig_Free(config)``.

* Get options:

  * ``PyInitConfig_HasOption(config, name)``.
  * ``PyInitConfig_GetInt(config, name, &value)``.
  * ``PyInitConfig_GetStr(config, name, &value)``.
  * ``PyInitConfig_GetStrList(config, name, &length, &items)``.
  * ``PyInitConfig_FreeStrList()``.

* Set options:

  * ``PyInitConfig_SetInt(config, name, value)``.
  * ``PyInitConfig_SetStr(config, name, value)``.
  * ``PyInitConfig_SetStrList(config, name, length, items)``.
  * ``PyInitConfig_AddModule(config, name, initfunc)``

* Initialize:

  * ``Py_InitializeFromInitConfig(config)``.

* Error handling:

  * ``PyInitConfig_GetError(config, &err_msg)``.
  * ``PyInitConfig_GetExitcode(config, &exitcode)``.

Add C API functions to get and set the current runtime configuration:

* ``PyConfig_Get(name)``.
* ``PyConfig_GetInt(name, &value)``.
* ``PyConfig_Set(name)``.
* ``PyConfig_Names()``.

The C API uses null-terminated UTF-8 encoded strings to refer to a
configuration option name.

These C API functions are excluded from the limited C API.

PyInitConfig structure
----------------------

The ``PyInitConfig`` structure is implemented by combining the three
structures of the ``PyConfig`` API and has an ``inittab`` member as
well:

* ``PyPreConfig preconfig``
* ``PyConfig config``
* ``PyStatus status``
* ``struct _inittab *inittab`` for ``PyInitConfig_AddModule()``

The ``PyStatus`` status is no longer separated, but part of the unified
``PyInitConfig`` structure, which makes the API easier to use.


Configuration Options
---------------------

Configuration options are named after ``PyPreConfig`` and ``PyConfig``
structure members. See the `PyPreConfig documentation
<https://docs.python.org/dev/c-api/init_config.html#pypreconfig>`_ and
the `PyConfig documentation
<https://docs.python.org/dev/c-api/init_config.html#pyconfig>`_.

Deprecating and removing configuration options is out of the scope of
the PEP and should be discussed on a case by case basis.

Public configuration options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Following options can be get by ``PyConfig_Get()`` and set and
``PyConfig_Set()``.

.. list-table::
   :widths: 20 20 50
   :header-rows: 1

   * - Option
     - Type
     - Comment
   * - ``argv``
     - ``list[str]``
     - API: ``sys.argv``.
   * - ``base_exec_prefix``
     - ``str``
     - API: ``sys.base_exec_prefix``.
   * - ``base_executable``
     - ``str``
     - API: ``sys.base_executable``.
   * - ``base_prefix``
     - ``str``
     - API: ``sys.base_prefix``.
   * - ``bytes_warning``
     - ``int``
     - API: ``sys.flags.bytes_warning``.
   * - ``exec_prefix``
     - ``str``
     - API: ``sys.base_prefix``.
   * - ``executable``
     - ``str``
     - API: ``sys.executable``.
   * - ``inspect``
     - ``bool``
     - API: ``sys.flags.inspect`` (``int``).
   * - ``int_max_str_digits``
     - ``int``
     - API: ``sys.flags.int_max_str_digits``,
       ``sys.get_int_max_str_digits()`` and
       ``sys.set_int_max_str_digits()``.
   * - ``interactive``
     - ``bool``
     - API: ``sys.flags.interactive``.
   * - ``module_search_paths``
     - ``list[str]``
     - API: ``sys.path``.
   * - ``optimization_level``
     - ``int``
     - API: ``sys.flags.optimize``.
   * - ``parser_debug``
     - ``bool``
     - API: ``sys.flags.debug`` (``int``).
   * - ``platlibdir``
     - ``str``
     - API: ``sys.platlibdir``.
   * - ``prefix``
     - ``str``
     - API: ``sys.base_prefix``.
   * - ``pycache_prefix``
     - ``str``
     - API: ``sys.pycache_prefix``.
   * - ``quiet``
     - ``bool``
     - API: ``sys.flags.quiet`` (``int``).
   * - ``stdlib_dir``
     - ``str``
     - API: ``sys._stdlib_dir``.
   * - ``use_environment``
     - ``bool``
     - API: ``sys.flags.ignore_environment`` (``int``).
   * - ``verbose``
     - ``int``
     - API: ``sys.flags.verbose``.
   * - ``warnoptions``
     - ``list[str]``
     - API: ``sys.warnoptions``.
   * - ``write_bytecode``
     - ``bool``
     - API: ``sys.flags.dont_write_bytecode`` (``int``) and ``sys.dont_write_bytecode`` (``bool``).
   * - ``xoptions``
     - ``dict[str, str]``
     - API: ``sys._xoptions``.

Some option names are different than ``sys`` attributes, such as
``optimization_level`` option and ``sys.flags.optimize`` attribute.
``PyConfig_Set()`` sets the corresponding ``sys`` attribute.

The ``xoptions`` is a list of strings in ``PyInitConfig`` where each
string has the format ``key`` (*value* is ``True`` implicitly) or
``key=value``. In the current runtime configuration, it becomes a
dictionary (``key: str`` → ``value: str | True``).

Read-only configuration options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Following options can be get by ``PyConfig_Get()``, but cannot be set by
``PyConfig_Set()``.

.. list-table::
   :widths: 20 20 50
   :header-rows: 1

   * - Option
     - Type
     - Comment
   * - ``allocator``
     - ``int``
     -
   * - ``buffered_stdio``
     - ``bool``
     -
   * - ``check_hash_pycs_mode``
     - ``str``
     - API: ``imp.check_hash_pycs_mode``.
   * - ``code_debug_ranges``
     - ``bool``
     -
   * - ``coerce_c_locale``
     - ``bool``
     -
   * - ``coerce_c_locale_warn``
     - ``bool``
     -
   * - ``configure_c_stdio``
     - ``bool``
     -
   * - ``configure_locale``
     - ``bool``
     -
   * - ``cpu_count``
     - ``int``
     - API: ``os.cpu_count()`` (``int | None``).
   * - ``dev_mode``
     - ``bool``
     - API: ``sys.flags.dev_mode``.
   * - ``dump_refs``
     - ``bool``
     -
   * - ``dump_refs_file``
     - ``str``
     -
   * - ``faulthandler``
     - ``bool``
     - API: ``faulthandler.is_enabled()``.
   * - ``filesystem_encoding``
     - ``str``
     - API: ``sys.getfilesystemencoding()``.
   * - ``filesystem_errors``
     - ``str``
     - API: ``sys.getfilesystemencodeerrors()``.
   * - ``hash_seed``
     - ``int``
     -
   * - ``home``
     - ``str``
     -
   * - ``import_time``
     - ``bool``
     -
   * - ``install_signal_handlers``
     - ``bool``
     -
   * - ``isolated``
     - ``bool``
     - API: ``sys.flags.isolated`` (``int``).
   * - ``legacy_windows_fs_encoding``
     - ``bool``
     -
   * - ``legacy_windows_stdio``
     - ``bool``
     - Windows only.
   * - ``malloc_stats``
     - ``bool``
     -
   * - ``module_search_paths_set``
     - ``bool``
     -
   * - ``orig_argv``
     - ``list[str]``
     - API: ``sys.orig_argv``.
   * - ``pathconfig_warnings``
     - ``bool``
     -
   * - ``parse_argv``
     - ``bool``
     -
   * - ``perf_profiling``
     - ``bool``
     - API: ``sys.is_stack_trampoline_active()``.
   * - ``program_name``
     - ``str``
     -
   * - ``pythonpath_env``
     - ``str``
     -
   * - ``run_command``
     - ``str``
     -
   * - ``run_filename``
     - ``str``
     -
   * - ``run_module``
     - ``str``
     -
   * - ``run_presite``
     - ``str``
     - need a debug build.
   * - ``safe_path``
     - ``bool``
     -
   * - ``show_ref_count``
     - ``bool``
     -
   * - ``site_import``
     - ``bool``
     - API: ``sys.flags.no_site`` (``int``).
   * - ``skip_source_first_line``
     - ``bool``
     -
   * - ``stdio_encoding``
     - ``str``
     - API: ``sys.stdin.encoding``, ``sys.stdout.encoding`` and
       ``sys.stderr.encoding``.
   * - ``stdio_errors``
     - ``str``
     - API: ``sys.stdin.errors``, ``sys.stdout.errors`` and
       ``sys.stderr.errors``.
   * - ``sys_path_0``
     - ``str``
     -
   * - ``tracemalloc``
     - ``int``
     - API: ``tracemalloc.is_tracing()`` (``bool``).
   * - ``use_frozen_modules``
     - ``bool``
     -
   * - ``use_hash_seed``
     - ``bool``
     -
   * - ``utf8_mode``
     - ``bool``
     -
   * - ``user_site_directory``
     - ``bool``
     - API: ``sys.flags.no_user_site`` (``int``).
   * - ``warn_default_encoding``
     - ``bool``
     -
   * - ``_install_importlib``
     - ``bool``
     -
   * - ``_init_main``
     - ``bool``
     -
   * - ``_is_python_build``
     - ``bool``
     -
   * - ``_pystats``
     - ``bool``
     - API: ``sys._stats_on()``, ``sys._stats_off()``.
       Need a ``Py_STATS`` build.


Create Config
-------------

``PyInitConfig`` structure:
    Opaque structure to configure the Python preinitialization and the
    Python initialization.

``PyInitConfig* PyInitConfig_Create(void)``:
    Create a new initialization configuration using default values
    of the `Isolated Configuration
    <https://docs.python.org/dev/c-api/init_config.html#isolated-configuration>`_.

    It must be freed with ``PyInitConfig_Free()``.

    Return ``NULL`` on memory allocation failure.

``void PyInitConfig_Free(PyInitConfig *config)``:
    Free memory of an initialization configuration.

Get Options
-----------

The configuration option *name* parameter must be a non-NULL
null-terminated UTF-8 encoded string.

``int PyInitConfig_HasOption(PyInitConfig *config, const char *name)``:
    Test if the configuration has an option called *name*.

    Return ``1`` if the option exists, or return ``0`` otherwise.

``int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value)``:
    Get an integer configuration option.

    * Set *\*value*, and return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

``int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value)``:
    Get a string configuration option as a null-terminated UTF-8
    encoded string.

    * Set *\*value*, and return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

    On success, the string must be released with ``free(value)``.

``int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items)``:
    Get a string list configuration option as an array of
    null-terminated UTF-8 encoded strings.

    * Set *\*length* and *\*value*, and return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

    On success, the string list must be released with
    ``PyInitConfig_FreeStrList(length, items)``.

``void PyInitConfig_FreeStrList(size_t length, char **items)``:
    Free memory of a string list created by
    ``PyInitConfig_GetStrList()``.


Set Options
-----------

The configuration option *name* parameter must be a non-NULL
null-terminated UTF-8 encoded string.

Some configuration options have side effects on other options. This
logic is only implemented when ``Py_InitializeFromInitConfig()`` is
called, not by the "Set" functions below. For example, setting
``dev_mode`` to ``1`` does not set ``faulthandler`` to ``1``.

``int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value)``:
    Set an integer configuration option.

    * Return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

``int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value)``:
    Set a string configuration option from a null-terminated UTF-8
    encoded string. The string is copied.

    * Return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

``int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items)``:
    Set a string list configuration option from an array of
    null-terminated UTF-8 encoded strings. The string list is copied.

    * Return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

``int PyInitConfig_AddModule(PyInitConfig *config, const char *name, PyObject* (*initfunc)(void))``:
    Add a built-in extension module to the table of built-in modules.

    The new module can be imported by the name *name*, and uses the
    function *initfunc* as the initialization function called on the
    first attempted import.

    * Return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.

    If Python is initialized multiple times,
    ``PyInitConfig_AddModule()`` must be called at each Python
    initialization.

    Similar to the ``PyImport_AppendInittab()`` function.


Initialize Python
-----------------

``int Py_InitializeFromInitConfig(PyInitConfig *config)``:
    Initialize Python from the initialization configuration.

    * Return ``0`` on success.
    * Set an error in *config* and return ``-1`` on error.
    * Set an exit code in *config* and return ``-1`` if Python wants to
      exit.

    See ``PyInitConfig_GetExitcode()`` for the exitcode case.


Error Handling
--------------

``int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg)``:
   Get the *config* error message.

   * Set *\*err_msg* and return ``1`` if an error is set.
   * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise.

   An error message is an UTF-8 encoded string.

   If *config* has an exit code, format the exit code as an error
   message.

   The error message remains valid until another ``PyInitConfig``
   function is called with *config*. The caller doesn't have to free the
   error message.

``int PyInitConfig_GetExitcode(PyInitConfig* config, int *exitcode)``:
   Get the *config* exit code.

   * Set *\*exitcode* and return ``1`` if Python wants to exit.
   * Return ``0`` if *config* has no exit code set.

   Only the ``Py_InitializeFromInitConfig()`` function can set an exit
   code if the ``parse_argv`` option is non-zero.

   An exit code can be set when parsing the command line failed (exit
   code 2) or when a command line option asks to display the command
   line help (exit code 0).


Get and Set the Runtime Configuration
-------------------------------------

The configuration option *name* parameter must be a non-NULL
null-terminated UTF-8 encoded string.

``PyObject* PyConfig_Get(const char *name)``:
    Get the current runtime value of a configuration option as a Python
    object.

    * Return a new reference on success.
    * Set an exception and return ``NULL`` on error.

    The object type depends on the option: see `Configuration Options`_
    tables.

    Other options are get from internal ``PyPreConfig`` and ``PyConfig`` structures.

    The caller must hold the GIL. The function cannot be called before
    Python initialization nor after Python finalization.

``int PyConfig_GetInt(const char *name, int *value)``:
    Similar to ``PyConfig_Get()``, but get the value as an integer.

    * Set ``*value`` and return ``0`` success.
    * Set an exception and return ``-1`` on error.

``PyObject* PyConfig_Names(void)``:
    Get all configuration option names as a ``frozenset``.

    Set an exception and return ``NULL`` on error.

    The caller must hold the GIL.

``PyObject* PyConfig_Set(const char *name, PyObject *value)``:
    Set the current runtime value of a configuration option.

    * Raise a ``ValueError`` if there is no option *name*.
    * Raise a ``ValueError`` if *value* is an invalid value.
    * Raise a ``ValueError`` if the option is read-only: cannot be set.
    * Raise a ``TypeError`` if *value* has not the proper type.

    `Read-only configuration options`_ cannot be set.

    The caller must hold the GIL. The function cannot be called before
    Python initialization nor after Python finalization.


Stability
---------

The behavior of options, the default option values, and the Python
behavior can change at each Python version: they are not "stable".

Moreover, configuration options can be added, deprecated and removed
following the usual :pep:`387` deprecation process.


Interaction with the PyPreConfig and PyConfig APIs
--------------------------------------------------

The lower level :pep:`587` ``PyPreConfig`` and ``PyConfig`` APIs  remain
available and fully supported. As noted in the Abstract, they remain the
preferred approach for embedding use cases that are aiming to closely
emulate the behaviour of the full CPython CLI, rather than just making a
Python runtime available as part of a larger application.

The ``PyPreConfig`` APIs may be used in combination with the
initialization API in this PEP. In such cases, the read-only vs
read/write restrictions for preconfiguration settings apply to
``PyInitConfig_SetInt`` in addition to ``PyConfig_Set`` once the
interpreter has been preconfigured (specifically, only
``use_environment`` may be updated, attempting to update any of the
other preconfiguration variables will report an error).


Examples
========

Initialize Python
-----------------

Example initializing Python, set configuration options of various types,
return ``-1`` on error:

.. code-block:: c

    int init_python(void)
    {
        PyInitConfig *config = PyInitConfig_Create();
        if (config == NULL) {
            printf("PYTHON INIT ERROR: memory allocation failed\n");
            return -1;
        }

        // Set an integer (dev mode)
        if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) {
            goto error;
        }

        // Set a list of UTF-8 strings (argv)
        char *argv[] = {"my_program", "-c", "pass"};
        if (PyInitConfig_SetStrList(config, "argv",
                                     Py_ARRAY_LENGTH(argv), argv) < 0) {
            goto error;
        }

        // Set a UTF-8 string (program name)
        if (PyInitConfig_SetStr(config, "program_name", L"my_program") < 0) {
            goto error;
        }

        // Initialize Python with the configuration
        if (Py_InitializeFromInitConfig(config) < 0) {
            goto error;
        }
        PyInitConfig_Free(config);
        return 0;

    error:
        // Display the error message
        const char *err_msg;
        (void)PyInitConfig_GetError(config, &err_msg);
        printf("PYTHON INIT ERROR: %s\n", err_msg);
        PyInitConfig_Free(config);

        return -1;
    }


Increase initialization bytes_warning option
--------------------------------------------

Example increasing the ``bytes_warning`` option of an initialization
configuration:

.. code-block:: c

    int config_bytes_warning(PyInitConfig *config)
    {
        int64_t bytes_warning;
        if (PyInitConfig_GetInt(config, "bytes_warning", &bytes_warning)) {
            return -1;
        }
        bytes_warning += 1;
        if (PyInitConfig_SetInt(config, "bytes_warning", bytes_warning)) {
            return -1;
        }
        return 0;
    }


Get the runtime verbose option
------------------------------

Example getting the current runtime value of the configuration option
``verbose``:

.. code-block:: c

    int get_verbose(void)
    {
        int verbose;
        if (PyConfig_GetInt("verbose", &verbose) < 0) {
            // Silently ignore the error
            PyErr_Clear();
            return -1;
        }
        return verbose;
    }

On error, the function silently ignores the error and returns ``-1``. In
practice, getting the ``verbose`` option cannot fail, unless a future
Python version removes the option.


Implementation
==============

* Issue: `No limited C API to customize Python initialization
  <https://github.com/python/cpython/issues/107954>`_
* PR: `Add PyInitConfig C API
  <https://github.com/python/cpython/pull/110176>`_
* PR: `Add PyConfig_Get() function
  <https://github.com/python/cpython/pull/112609>`_


Backwards Compatibility
=======================

Changes are fully backward compatible. Only new APIs are added.

Existing API such as the ``PyConfig`` C API (PEP 587) are left
unchanged.


Rejected Ideas
==============

Configuration as text
---------------------

It was proposed to provide the configuration as text to make the API
compatible with the stable ABI and to allow custom options.

Example::

    # integer
    bytes_warning = 2

    # string
    filesystem_encoding = "utf8"   # comment

    # list of strings
    argv = ['python', '-c', 'code']

The API would take the configuration as a string, not as a file. Example
with a hypothetical ``PyInit_SetConfig()`` function:

.. code-block:: c

    void stable_abi_init_demo(int set_path)
    {
        PyInit_SetConfig(
            "isolated = 1\n"
            "argv = ['python', '-c', 'code']\n"
            "filesystem_encoding = 'utf-8'\n"
        );
        if (set_path) {
            PyInit_SetConfig("pythonpath = '/my/path'");
        }
    }

The example ignores error handling to make it easier to read.

The problem is that generating such configuration text requires adding
quotes to strings and to escape quotes in strings. Formatting an array
of strings becomes non-trivial.

Providing an API to format a string or an array of strings is not really
worth it, whereas Python can provide directly an API to set a
configuration option where the value is passed directly as a string or
an array of strings. It avoids giving special meaning to some
characters, such as newline characters, which would have to be escaped.


Refer to an option with an integer
----------------------------------

Using strings to refer to a configuration option requires comparing
strings which can be slower than comparing integers.

Use integers, similar to type "slots" such as ``Py_tp_doc``, to refer to
a configuration option. The ``const char *name`` parameter is replaced
with ``int option``.

Accepting custom options is more likely to cause conflicts when using
integers, since it's harder to maintain "namespaces" (ranges) for
integer options. Using strings, a simple prefix with a colon separator
can be used.

Integers also requires maintaining a list of integer constants and so
make the C API and the Python API larger.

Python 3.13 only has around 62 configuration options, and so performance
is not really a blocker issue. If better performance is needed later, a
hash table can be used to get an option by its name.

If getting a configuration option is used in hot code, the value can be
read once and cached. By the way, most configuration options cannot be
changed at runtime.


Multi-phase initialization (similar to PEP 432)
-----------------------------------------------

`Eric Snow expressed concerns <https://discuss.python.org/t/pep-741-python-configuration-c-api-second-version/45403/27>`_
that this proposal might reinforce with embedders the idea that
initialization is a single monolithic step.  He argued that initialization
involves 5 distinct phases and even suggested that the API should
reflect this explicitly.  Eric proposed that, at the very least, the
implementation of initialization should reflect the phases, in part
for improved code health.  Overall, his explanation has some
similarities with :pep:`432` and :pep:`587`.

Another of Eric's key points relevant to this PEP was that, ideally,
the config passed to ``Py_InitializeFromConfig()`` should be complete
before that function is called, whereas currently initialization
actually modifies the config.

While Eric wasn't necessarily suggesting an alternative to PEP 741,
any proposal to add a granular initialization API around phases is
effectively the opposite of what this PEP is trying to accomplish.
Such API is more complicated, it requires adding new public structures
and new public functions. It makes the Python initialization more
complicated, rather than this PEP tries to unify existing APIs and make
them simpler (the opposite). Having multiple structures for similar
purpose can lead to duplicate members, similar issue than duplicated
members between existing ``PyPreConfig`` and ``PyConfig`` structures.


Locale encoding and wide strings
--------------------------------

Accepting strings encoded to the locale encoding and accepting wide
strings (``wchar_t*``) in the ``PyInitConfig`` API was deferred to keep
the ``PyInitConfig`` API simple and avoid the complexity of the Python
preinitialization. These features are also mostly needed when emulating
the full CPython CLI behaviour, and hence better served by the lower
level :pep:`587` API.


Discussions
===========

* `PEP 741: Python Configuration C API (second version)
  <https://discuss.python.org/t/pep-741-python-configuration-c-api-second-version/45403>`_
  (February 2024).
* `PEP 741: Python Configuration C API
  <https://discuss.python.org/t/pep-741-python-configuration-c-api/43637>`_
  (January 2024).
* `FR: Allow private runtime config to enable extending without breaking
  the PyConfig ABI
  <https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004>`__
  (August 2022).


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.