From 4352a4a56cb9d45c6f0df3bafcc9f98453b12c51 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Sun, 10 Mar 2024 00:54:18 +0100 Subject: [PATCH] PEP 741: Remove the Python API (#3711) --- peps/pep-0741.rst | 446 ++++++++++++++++++++-------------------------- 1 file changed, 194 insertions(+), 252 deletions(-) diff --git a/peps/pep-0741.rst b/peps/pep-0741.rst index fc1fc484e..9549461ec 100644 --- a/peps/pep-0741.rst +++ b/peps/pep-0741.rst @@ -16,15 +16,13 @@ Add a C API to the limited C API to configure the Python initialization, and to get the current configuration. It can be used with the stable ABI. -Add ``sys.get_config(name)`` and ``sys.get_config_names()`` functions to -get the current configuration. Add ``sys.set_config(name, value)`` -function to set a configuration option. Unified API to access all -configuration options. - Complete :pep:`587` API by adding ``PyInitConfig_AddModule()`` which can be used to add a built-in extension module; feature previously referred to as the "inittab". +Add ``PyConfig_Get()`` and ``PyConfig_Set()`` functions to +get and set the current configuration at runtime. + :pep:`587` "Python Initialization Configuration" unified all the ways to configure the Python **initialization**. This PEP (almost fully) unifies also the configuration of the Python **preinitialization** and the @@ -123,7 +121,7 @@ Cython needs to access the ``optimization_level`` configuration option: When global configuration variables were deprecated in 2022, `Marc-André Lemburg requested `__ -an API to access these configuration variables at runtime (not only +a C API to access these configuration variables at runtime (not only during Python initialization). @@ -366,7 +364,7 @@ Set the current configuration `Marc-André Lemburg requested `__ -an API to **set** the value of some configuration options at runtime: +a C API to **set** the value of some configuration options at runtime: * ``optimization_level`` * ``verbose`` @@ -374,22 +372,6 @@ an API to **set** the value of some configuration options at runtime: * ``inspect`` * ``write_bytecode`` -Moreover, currently, when a new configuration option is added, usually -"get" and "set" functions should be added as well. It makes the ``sys`` -module bigger and bigger, whereas not all options deserves dedicated -functions. For example, when the ``int_max_str_digits`` option was -added, ``sys.get_int_max_str_digits()`` and -``sys.set_int_max_str_digits()`` functions were added as well. - -Some options are exposed directly as ``sys`` attributes. For example, -the ``module_search_paths`` option is expoed as ``sys.path``. The -problem is that there is no validation when the attribute is modified -which can lead to various bugs or even crashes. The attribute can even -be removed: ``del sys.path``. - -Having a single unified API with generic "get" and "set" functions would -avoid these issues and give access to all configuration options. - Specification ============= @@ -397,39 +379,50 @@ Specification Add C API functions and structure to configure the Python initialization: -* ``PyInitConfig`` opaque structure. -* ``PyInitConfig_CreatePython()``. -* ``PyInitConfig_CreateIsolated()``. -* ``PyInitConfig_Free(config)``. -* ``PyInitConfig_HasOption(config, name)``. -* ``PyInitConfig_GetInt(config, name, &value)``. -* ``PyInitConfig_GetStr(config, name, &value)``. -* ``PyInitConfig_GetWStr(config, name, &value)``. -* ``PyInitConfig_GetStrList(config, name, &length, &items)``. -* ``PyInitConfig_FreeStrList()``. -* ``PyInitConfig_GetWStrList(config, name, &length, &items)``. -* ``PyInitConfig_FreeWStrList()``. -* ``PyInitConfig_SetInt(config, name, value)``. -* ``PyInitConfig_SetStr(config, name, value)``. -* ``PyInitConfig_SetStrLocale(config, name, value)``. -* ``PyInitConfig_SetWStr(config, name, value)``. -* ``PyInitConfig_SetStrList(config, name, length, items)``. -* ``PyInitConfig_SetStrLocaleList(config, name, length, items)``. -* ``PyInitConfig_SetWStrList(config, name, length, items)``. -* ``PyInitConfig_AddModule(config, name, initfunc)`` -* ``Py_PreInitializeFromInitConfig(config)``. -* ``Py_InitializeFromInitConfig(config)``. -* ``PyInitConfig_GetError(config, &err_msg)``. +* Create config: -Add C API and Python functions to get the current configuration: + * ``PyInitConfig`` opaque structure. + * ``PyInitConfig_CreatePython()``. + * ``PyInitConfig_CreateIsolated()``. + * ``PyInitConfig_Free(config)``. -* ``PyConfig_Get(name)``. +* Get options: + + * ``PyInitConfig_HasOption(config, name)``. + * ``PyInitConfig_GetInt(config, name, &value)``. + * ``PyInitConfig_GetStr(config, name, &value)``. + * ``PyInitConfig_GetWStr(config, name, &value)``. + * ``PyInitConfig_GetStrList(config, name, &length, &items)``. + * ``PyInitConfig_FreeStrList()``. + * ``PyInitConfig_GetWStrList(config, name, &length, &items)``. + * ``PyInitConfig_FreeWStrList()``. + +* Set optons: + + * ``PyInitConfig_SetInt(config, name, value)``. + * ``PyInitConfig_SetStr(config, name, value)``. + * ``PyInitConfig_SetStrLocale(config, name, value)``. + * ``PyInitConfig_SetWStr(config, name, value)``. + * ``PyInitConfig_SetStrList(config, name, length, items)``. + * ``PyInitConfig_SetStrLocaleList(config, name, length, items)``. + * ``PyInitConfig_SetWStrList(config, name, length, items)``. + * ``PyInitConfig_AddModule(config, name, initfunc)`` + +* Initialize: + + * ``Py_PreInitializeFromInitConfig(config)``. + * ``Py_InitializeFromInitConfig(config)``. + +* Error handling: + + * ``PyInitConfig_GetError(config, &err_msg)``. + +Add C API functions to get and set the current configuration: + +* ``PyConfig_Get(name)`` → ``object``. * ``PyConfig_GetInt(name, &value)``. * ``PyConfig_Set(name)``. -* ``PyConfig_Keys()``. -* ``sys.get_config(name)``. -* ``sys.get_config_names()``. -* ``sys.set_config(name, value)``. +* ``PyConfig_Names()`` → ``frozenset``. The C API uses null-terminated UTF-8 encoded strings to refer to a configuration option. @@ -464,8 +457,8 @@ the PEP and should be discussed on a case by case basis. Public configuration options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Following options can be get by ``sys.get_config()`` and set and -``sys.set_config()``. +Following options can be get by ``PyConfig_Get()`` and set and +``PyConfig_Set()``. .. list-table:: :widths: 20 20 50 @@ -474,88 +467,87 @@ Following options can be get by ``sys.get_config()`` and set and * - Option - Type - Comment - * - ``"argv"`` + * - ``argv`` - ``list[str]`` - - API: ``sys.argv`` (``list[str]``). - * - ``"base_exec_prefix"`` + - API: ``sys.argv``. + * - ``base_exec_prefix`` - ``str`` - - API: ``sys.base_exec_prefix`` (``str``). - * - ``"base_executable"`` + - API: ``sys.base_exec_prefix``. + * - ``base_executable`` - ``str`` - - API: ``sys.base_executable`` (``str``). - * - ``"base_prefix"`` + - API: ``sys.base_executable``. + * - ``base_prefix`` - ``str`` - - API: ``sys.base_prefix`` (``str``). - * - ``"bytes_warning"`` + - API: ``sys.base_prefix``. + * - ``bytes_warning`` - ``int`` - - API: ``sys.flags.bytes_warning`` (``int``). - * - ``"exec_prefix"`` + - API: ``sys.flags.bytes_warning``. + * - ``exec_prefix`` - ``str`` - - API: ``sys.base_prefix`` (``str``). - * - ``"executable"`` + - API: ``sys.base_prefix``. + * - ``executable`` - ``str`` - - API: ``sys.executable`` (``str``). - * - ``"inspect"`` + - API: ``sys.executable``. + * - ``inspect`` - ``bool`` - API: ``sys.flags.inspect`` (``int``). - * - ``"int_max_str_digits"`` + * - ``int_max_str_digits`` - ``int`` - - API: ``sys.flags.int_max_str_digits`` (``int``), + - API: ``sys.flags.int_max_str_digits``, ``sys.get_int_max_str_digits()`` and ``sys.set_int_max_str_digits()``. - * - ``"interactive"`` + * - ``interactive`` - ``bool`` - - API: ``sys.flags.interactive`` (``int``). - * - ``"module_search_paths"`` + - API: ``sys.flags.interactive``. + * - ``module_search_paths`` - ``list[str]`` - - API: ``sys.path`` (``list[str]``). - * - ``"optimization_level"`` + - API: ``sys.path``. + * - ``optimization_level`` - ``int`` - - API: ``sys.flags.optimize`` (``int``). - * - ``"parser_debug"`` + - API: ``sys.flags.optimize``. + * - ``parser_debug`` - ``bool`` - API: ``sys.flags.debug`` (``int``). - * - ``"platlibdir"`` + * - ``platlibdir`` - ``str`` - - API: ``sys.platlibdir`` (``str``). - * - ``"prefix"`` + - API: ``sys.platlibdir``. + * - ``prefix`` - ``str`` - - API: ``sys.base_prefix`` (``str``). - * - ``"pycache_prefix"`` + - API: ``sys.base_prefix``. + * - ``pycache_prefix`` - ``str`` - - API: ``sys.pycache_prefix`` (``str``). - * - ``"quiet"`` + - API: ``sys.pycache_prefix``. + * - ``quiet`` - ``bool`` - API: ``sys.flags.quiet`` (``int``). - * - ``"stdlib_dir"`` + * - ``stdlib_dir`` - ``str`` - - API: ``sys._stdlib_dir`` (``str``). - * - ``"use_environment"`` + - API: ``sys._stdlib_dir``. + * - ``use_environment`` - ``bool`` - API: ``sys.flags.ignore_environment`` (``int``). - * - ``"verbose"`` + * - ``verbose`` - ``int`` - - API: ``sys.flags.verbose`` (``int``). - * - ``"warnoptions"`` + - API: ``sys.flags.verbose``. + * - ``warnoptions`` - ``list[str]`` - - API: ``sys.warnoptions`` (``list[str]``). - * - ``"write_bytecode"`` + - API: ``sys.warnoptions``. + * - ``write_bytecode`` - ``bool`` - API: ``sys.flags.dont_write_bytecode`` (``int``) and ``sys.dont_write_bytecode`` (``bool``). - * - ``"xoptions"`` + * - ``xoptions`` - ``dict[str, str]`` - - API: ``sys._xoptions`` (``dict[str, str]``). + - API: ``sys._xoptions``. Some option names are different than ``sys`` attributes, such as -``"optimization_level"`` option and ``sys.flags.optimize`` attribute. -``sys.set_config(name, value)`` sets the corresponding ``sys`` -attribute. +``optimization_level`` option and ``sys.flags.optimize`` attribute. +``PyConfig_Set()`` sets the corresponding ``sys`` attribute. Read-only configuration options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Following options can be get ``sys.get_config()``, but cannot be set by -``sys.set_config()``. +Following options can be get ``PyConfig_Get()``, but cannot be set by +``PyConfig_Set()``. .. list-table:: :widths: 20 20 50 @@ -564,180 +556,166 @@ Following options can be get ``sys.get_config()``, but cannot be set by * - Option - Type - Comment - * - ``"check_hash_pycs_mode"`` + * - ``allocator`` + - ``int`` + - + * - ``buffered_stdio`` + - ``bool`` + - + * - ``check_hash_pycs_mode`` - ``str`` - - API: ``imp.check_hash_pycs_mode`` (``str``). - * - ``"code_debug_ranges"`` + - API: ``imp.check_hash_pycs_mode``. + * - ``code_debug_ranges`` - ``bool`` - - * - ``"coerce_c_locale"`` + * - ``coerce_c_locale`` - ``bool`` - - * - ``"coerce_c_locale_warn"`` + * - ``coerce_c_locale_warn`` - ``bool`` - - * - ``"configure_locale"`` + * - ``configure_c_stdio`` - ``bool`` - - * - ``"cpu_count"`` + * - ``configure_locale`` + - ``bool`` + - + * - ``cpu_count`` - ``int`` - API: ``os.cpu_count()`` (``int | None``). - * - ``"dev_mode"`` + * - ``dev_mode`` - ``bool`` - - API: ``sys.flags.dev_mode`` (``bool``). - * - ``"filesystem_encoding"`` - - ``str`` - - API: ``sys.getfilesystemencoding()`` (``str``). - * - ``"filesystem_errors"`` - - ``str`` - - API: ``sys.getfilesystemencodeerrors()`` (``str``). - * - ``"import_time"`` + - API: ``sys.flags.dev_mode``. + * - ``dump_refs`` - ``bool`` - - * - ``"isolated"`` + * - ``dump_refs_file`` + - ``str`` + - + * - ``faulthandler`` + - ``bool`` + - API: ``faulthandler.is_enabled()``. + * - ``filesystem_encoding`` + - ``str`` + - API: ``sys.getfilesystemencoding()``. + * - ``filesystem_errors`` + - ``str`` + - API: ``sys.getfilesystemencodeerrors()``. + * - ``hash_seed`` + - ``int`` + - + * - ``home`` + - ``str`` + - + * - ``import_time`` + - ``bool`` + - + * - ``install_signal_handlers`` + - ``bool`` + - + * - ``isolated`` - ``bool`` - API: ``sys.flags.isolated`` (``int``). - * - ``"legacy_windows_fs_encoding"`` + * - ``legacy_windows_fs_encoding`` - ``bool`` - - * - ``"orig_argv"`` - - ``list[str]`` - - API: ``sys.orig_argv`` (``list[str]``). - * - ``"perf_profiling"`` - - ``bool`` - - API: ``sys.is_stack_trampoline_active()``. - * - ``"site_import"`` - - ``bool`` - - API: ``sys.flags.no_site`` (``int``). - * - ``"utf8_mode"`` - - ``bool`` - - - -Initialization-only configuration options -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Following options are only to initialize Python, are not in -``sys.get_config_names()``, and cannot be get with ``sys.get_config()``. - -.. list-table:: - :widths: 20 20 50 - :header-rows: 1 - - * - Option - - Type - - Comment - * - ``"allocator"`` - - ``int`` - - - * - ``"buffered_stdio"`` - - ``bool`` - - - * - ``"configure_c_stdio"`` - - ``bool`` - - - * - ``"dump_refs"`` - - ``bool`` - - - * - ``"dump_refs_file"`` - - ``str`` - - - * - ``"faulthandler"`` - - ``bool`` - - API: ``faulthandler.is_enabled()`` (``bool``). - * - ``"hash_seed"`` - - ``int`` - - - * - ``"home"`` - - ``str`` - - - * - ``"install_signal_handlers"`` - - ``bool`` - - - * - ``"legacy_windows_stdio"`` + * - ``legacy_windows_stdio`` - ``bool`` - Windows only - * - ``"malloc_stats"`` + * - ``malloc_stats`` - ``bool`` - - * - ``"module_search_paths_set"`` + * - ``module_search_paths_set`` - ``bool`` - - * - ``"pathconfig_warnings"`` + * - ``orig_argv`` + - ``list[str]`` + - API: ``sys.orig_argv``. + * - ``pathconfig_warnings`` - ``bool`` - - * - ``"parse_argv"`` + * - ``parse_argv`` - ``bool`` - - * - ``"program_name"`` + * - ``perf_profiling`` + - ``bool`` + - API: ``sys.is_stack_trampoline_active()``. + * - ``program_name`` - ``str`` - - * - ``"pythonpath_env"`` + * - ``pythonpath_env`` - ``str`` - - * - ``"run_command"`` + * - ``run_command`` - ``str`` - - * - ``"run_filename"`` + * - ``run_filename`` - ``str`` - - * - ``"run_module"`` + * - ``run_module`` - ``str`` - - * - ``"run_presite"`` + * - ``run_presite`` - ``str`` - need a debug build. - * - ``"safe_path"`` + * - ``safe_path`` - ``bool`` - - * - ``"show_ref_count"`` + * - ``show_ref_count`` - ``bool`` - - * - ``"skip_source_first_line"`` + * - ``site_import`` + - ``bool`` + - API: ``sys.flags.no_site`` (``int``). + * - ``skip_source_first_line`` - ``bool`` - - * - ``"stdio_encoding"`` + * - ``stdio_encoding`` - ``str`` - API: ``sys.stdin.encoding``, ``sys.stdout.encoding`` and - ``sys.stderr.encoding`` (``str``). - * - ``"stdio_errors"`` + ``sys.stderr.encoding``. + * - ``stdio_errors`` - ``str`` - API: ``sys.stdin.errors``, ``sys.stdout.errors`` and - ``sys.stderr.errors`` (``str``). - * - ``"sys_path_0"`` + ``sys.stderr.errors``. + * - ``sys_path_0`` - ``str`` - - * - ``"tracemalloc"`` + * - ``tracemalloc`` - ``int`` - API: ``tracemalloc.is_tracing()`` (``bool``). - * - ``"use_frozen_modules"`` + * - ``use_frozen_modules`` - ``bool`` - - * - ``"use_hash_seed"`` + * - ``use_hash_seed`` - ``bool`` - - * - ``"user_site_directory"`` + * - ``utf8_mode`` + - ``bool`` + - + * - ``user_site_directory`` - ``bool`` - API: ``sys.flags.no_user_site`` (``int``). - * - ``"warn_default_encoding"`` + * - ``warn_default_encoding`` - ``bool`` - - * - ``"_install_importlib"`` + * - ``_install_importlib`` - ``bool`` - - * - ``"_init_main"`` + * - ``_init_main`` - ``bool`` - - * - ``"_is_python_build"`` + * - ``_is_python_build`` - ``bool`` - - * - ``"_pystats"`` + * - ``_pystats`` - ``bool`` - API: ``sys._stats_on()``, ``sys._stats_off()``. Need a ``Py_STATS`` build. -Preconfiguration ----------------- +Preinitialization +----------------- Calling ``Py_PreInitializeFromInitConfig()`` preinitializes Python. For example, it sets the memory allocator, and can configure the @@ -747,18 +725,19 @@ example, it sets the memory allocator, and can configure the The following option names can only be set during the Python preconfiguration: -* ``"allocator"``, -* ``"coerce_c_locale"``, -* ``"coerce_c_locale_warn"``, -* ``"configure_locale"``, -* ``"legacy_windows_fs_encoding"``, -* ``"utf8_mode"``. +* ``allocator``, +* ``coerce_c_locale``, +* ``coerce_c_locale_warn``, +* ``configure_locale``, +* ``legacy_windows_fs_encoding``, +* ``utf8_mode``. Trying to set these options after Python preinitialization fails with an error. ``PyInitConfig_SetStrLocale()`` and ``PyInitConfig_SetStrLocaleList()`` -functions cannot be called before the Python preinitialization. +functions call ``Py_PreInitializeFromInitConfig()`` if Python is not +already preinitialized. Create PyInitConfig @@ -857,7 +836,7 @@ null-terminated UTF-8 encoded string. Some configuration options have side effects on other options. This logic is only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the "Set" functions below. For example, setting -``"dev_mode"`` to ``1`` does not set ``"faulthandler"`` to ``1``. +``dev_mode`` to ``1`` does not set ``faulthandler`` to ``1``. ``int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value)``: Set an integer configuration option. @@ -971,37 +950,14 @@ Get and set the current configuration The configuration option *name* parameter must be a non-NULL null-terminated UTF-8 encoded string. -Not all configuration options are accessible at runtime: see -`Configuration Options`_. For example, ``"module_search_paths_set"`` is -not relevant and cannot be accessed. Some options are read-only and can -only be read, such as ``"utf8_mode"``. - ``PyObject* PyConfig_Get(const char *name)``: Get the current value of a configuration option as an object. * Return a new reference on success. * Set an exception and return ``NULL`` on error. - The object type depends on the option. - - The following options are read from the ``sys`` modules. - - * ``"argv"``: ``sys.argv``. - * ``"base_exec_prefix"``: ``sys.base_exec_prefix``. - * ``"base_executable"``: ``sys._base_executable``. - * ``"base_prefix"``: ``sys.base_prefix``. - * ``"exec_prefix"``: ``sys.exec_prefix``. - * ``"executable"``: ``sys.executable``. - * ``"module_search_paths"``: ``sys.path``. - * ``"orig_argv"``: ``sys.orig_argv``. - * ``"platlibdir"``: ``sys.platlibdir``. - * ``"prefix"``: ``sys.prefix``. - * ``"pycache_prefix"``: ``sys.pycache_prefix``. - * ``"stdlib_dir"``: ``sys._stdlib_dir``. - * ``"warnoptions"``: ``sys.warnoptions``. - * ``"write_bytecode"``: ``not sys.dont_write_bytecode`` - (opposite value). - * ``"xoptions"``: ``sys._xoptions``. + The object type depends on the option: see `Configuration Options`_ + tables. Other options are get from internal ``PyPreConfig`` and ``PyConfig`` structures. @@ -1014,7 +970,7 @@ only be read, such as ``"utf8_mode"``. * Set ``*value`` and return ``0`` success. * Set an exception and return ``-1`` on error. -``PyObject* PyConfig_Keys(void)``: +``PyObject* PyConfig_Names(void)``: Get all configuration option names as a ``frozenset``. Set an exception and return ``NULL`` on error. @@ -1029,26 +985,12 @@ only be read, such as ``"utf8_mode"``. * Raise a ``ValueError`` if *value* is an invalid value. * Raise a ``ValueError`` if the option is read-only: cannot be set. + `Read-only configuration options`_ cannot be set. + The caller must hold the GIL. The function cannot be called before Python initialization nor after Python finalization. -Add sys functions ------------------ - -* Add ``sys.get_config(name: str)`` function which calls - ``PyConfig_Get()``: - - * Return the configuration option value on success. - * Raise an exception on error. - -* Add ``sys.get_config_names()`` function which gets all configuration - option names as a ``frozenset``. - -* Add ``sys.get_config(name: str, value)`` function which calls - ``PyConfig_Set(name, value)``. Raise an exception on error. - - Scope of the stable ABI ----------------------- @@ -1087,7 +1029,7 @@ return -1 on error: } // Set a list of wide strings (argv) - wchar_t *argv[] = {L"my_program"", L"-c", L"pass"}; + wchar_t *argv[] = {L"my_program", L"-c", L"pass"}; if (PyInitConfig_SetWStrList(config, "argv", Py_ARRAY_LENGTH(argv), argv) < 0) { goto error;