iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 587: Init API v2 (#1018)

This commit is contained in:
Victor Stinner 2019-05-02 15:14:06 -04:00 committed by GitHub
parent 7519d3280b
commit 8763930784
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 312 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

447
pep-0587.rst
View File

@ -55,13 +55,20 @@ New structures (4):
* ``PyConfig``
* ``PyInitError``
* ``PyPreConfig``
* ``PyWideCharList``
* ``PyWideStringList``
New functions (9):
New functions (16):
* ``Py_PreInitialize(config)``
* ``Py_PreInitializeFromArgs(config, argc, argv)``
* ``Py_PreInitializeFromWideArgs(config, argc, argv)``
* ``PyWideStringList_Append(list, item)``
* ``PyConfig_DecodeLocale(config_str, str)``
* ``PyConfig_SetString(config_str, str)``
* ``PyConfig_Read(config)``
* ``PyConfig_SetArgv(config, argc, argv)``
* ``PyConfig_SetWideArgv(config, argc, argv)``
* ``PyConfig_Clear(config)``
* ``Py_InitializeFromConfig(config)``
* ``Py_InitializeFromArgs(config, argc, argv)``
* ``Py_InitializeFromWideArgs(config, argc, argv)``
@ -69,27 +76,29 @@ New functions (9):
* ``Py_ExitInitError(err)``
* ``Py_RunMain()``
New macros (9):
New macros (6):
* ``PyPreConfig_INIT``
* ``PyConfig_INIT``
* ``Py_INIT_OK()``
* ``Py_INIT_ERR(MSG)``
* ``Py_INIT_USER_ERR(MSG)``
* ``Py_INIT_NO_MEMORY()``
* ``Py_INIT_EXIT(EXITCODE)``
* ``Py_INIT_IS_ERROR(err)``
* ``Py_INIT_IS_EXIT(err)``
* ``Py_INIT_FAILED(err)``
This PEP also adds ``_PyRuntimeState.preconfig`` (``PyPreConfig``) and
``PyInterpreterState.config`` (``PyConfig``) fields to these internal
structures. ``PyInterpreterState.config`` becomes the new reference
configuration, replacing global configuration variables (and a few other
private variables).
This PEP also adds ``_PyRuntimeState.preconfig`` (``PyPreConfig`` type)
and ``PyInterpreterState.config`` (``PyConfig`` type) fields to these
internal structures. ``PyInterpreterState.config`` becomes the new
reference configuration, replacing global configuration variables and
other private variables.
PyWideCharList
--------------
PyWideStringList
----------------
``PyWideCharList`` is a list of ``wchar_t*`` strings.
``PyWideStringList`` is a list of ``wchar_t*`` strings.
Example to initialize a string from C static array::
@ -97,27 +106,29 @@ Example to initialize a string from C static array::
L"-c",
L"pass",
};
PyWideCharList config_argv = PyWideCharList_INIT;
PyWideStringList config_argv = PyWideStringList_INIT;
config_argv.length = Py_ARRAY_LENGTH(argv);
config_argv.items = argv;
``PyWideCharList`` structure fields:
``PyWideStringList`` structure fields:
* ``length`` (``Py_ssize_t``)
* ``items`` (``wchar_t**``)
Methods:
* ``PyInitError PyWideStringList_Append(PyWideStringList *list, const wchar_t *item)``:
Append *item* to *list*.
If *length* is non-zero, *items* must be non-NULL and all strings must
be non-NULL.
.. note::
The "WideChar" name comes from the existing Python C API. Example:
``PyUnicode_FromWideChar(const wchar_t *str, Py_ssize_t size)``.
PyInitError
-----------
``PyInitError`` is a structure to store an error message or an exit code
for the Python Initialization.
for the Python Initialization. For an error, it stores the C function
name which created the error.
Example::
@ -143,29 +154,31 @@ Example::
``PyInitError`` fields:
* ``exitcode`` (``int``): if greater or equal to zero, argument passed to
``exit()``
* ``msg`` (``const char*``): error message
* ``prefix`` (``const char*``): string displayed before the message,
usually rendered as ``prefix: msg``.
* ``user_err`` (int): if non-zero, the error is caused by the user
configuration, otherwise it's an internal Python error.
* ``exitcode`` (``int``):
argument passed to ``exit()`` on Unix and to ``ExitProcess()`` on
Windows. Only set by ``Py_INIT_EXIT()``.
* ``err_msg`` (``const char*``): error message
* private ``_func`` field: used by ``Py_INIT_ERR()`` to store the C
function name which created the error.
* private ``_type`` field: for internal usage only.
Macro to create an error:
* ``Py_INIT_OK()``: success
* ``Py_INIT_ERR(err_msg)``: initialization error with a message
* ``Py_INIT_NO_MEMORY()``: memory allocation failure (out of memory)
* ``Py_INIT_ERR(MSG)``: Python internal error
* ``Py_INIT_USER_ERR(MSG)``: error caused by user configuration
* ``Py_INIT_EXIT(STATUS)``: exit Python with the specified status
* ``Py_INIT_EXIT(exitcode)``: exit Python with the specified exit code
Other macros and functions:
* ``Py_INIT_FAILED(err)``: Is the result an error or an exit?
* ``Py_ExitInitError(err)``: call ``exit(status)`` for
an error created by ``Py_INIT_EXIT(status)``,
call ``Py_FatalError(msg)`` for other errors. Must not be called for
an error created by ``Py_INIT_OK()``.
* ``Py_INIT_IS_ERROR(err)``: Is the result an error?
* ``Py_INIT_IS_EXIT(err)``: Is the result an exit?
* ``Py_INIT_FAILED(err)``: Is the result an error or an exit? Similar
to ``Py_INIT_IS_ERROR(err) || Py_INIT_IS_EXIT(err)``.
* ``Py_ExitInitError(err)``: Call ``exit(exitcode)`` on Unix or
``ExitProcess(exitcode)`` if the result is an exit, call
``Py_FatalError(err_msg)`` if the result is an error. Must not be
called if the result is a success.
Pre-Initialization with PyPreConfig
-----------------------------------
@ -195,26 +208,36 @@ Example using the pre-initialization to enable the UTF-8 Mode::
Functions to pre-initialize Python:
* ``PyInitError Py_PreInitialize(const PyPreConfig *config)``
* ``PyInitError Py_PreInitializeFromArgs( const PyPreConfig *config, int argc, char **argv)``
* ``PyInitError Py_PreInitializeFromWideArgs( const PyPreConfig *config, int argc, wchar_t **argv)``
* ``PyInitError Py_PreInitializeFromArgs(const PyPreConfig *config, int argc, char **argv)``
* ``PyInitError Py_PreInitializeFromWideArgs(const PyPreConfig *config, int argc, wchar_t **argv)``
If Python should be pre-initialized explicitly first and then
initialized with command line arguments, it is possible to pass these
command line arguments to the pre-initialization since they impact the
encodings. For example, ``-X utf8`` enables the UTF-8 Mode.
These functions can be called with *config* set to ``NULL``. The caller
is responsible to handler error using ``Py_INIT_FAILED()`` and
is responsible to handle error using ``Py_INIT_FAILED()`` and
``Py_ExitInitError()``.
``PyPreConfig`` fields:
* ``allocator``: name of the memory allocator (ex: ``"malloc"``)
* ``coerce_c_locale_warn``: if non-zero, emit a warning if the C locale
* ``allocator`` (``char*``): name of the memory allocator (ex: ``"malloc"``)
* ``coerce_c_locale_warn`` (``int``): if non-zero, emit a warning if the C locale
is coerced.
* ``coerce_c_locale``: if equals to 2, coerce the C locale; if equals to
* ``coerce_c_locale`` (``int``): if equals to 2, coerce the C locale; if equals to
1, read the LC_CTYPE to decide if it should be coerced.
* ``dev_mode``: see ``PyConfig.dev_mode``
* ``isolated``: see ``PyConfig.isolated``
* ``legacy_windows_fs_encoding`` (Windows only): if non-zero, set the
* ``dev_mode`` (``int``): see ``PyConfig.dev_mode``
* ``isolated`` (``int``): see ``PyConfig.isolated``
* ``legacy_windows_fs_encoding`` (``int``, Windows only): if non-zero, set the
Python filesystem encoding to ``"mbcs"``.
* ``use_environment``: see ``PyConfig.use_environment``
* ``utf8_mode``: if non-zero, enable the UTF-8 mode
* ``use_environment`` (``int``): see ``PyConfig.use_environment``
* ``utf8_mode`` (``int``): if non-zero, enable the UTF-8 mode
There is also a private field which is for internal-usage only:
* ``_config_version`` (``int``): Configuration version, used for ABI
compatibility
The C locale coercion (PEP 538) and the UTF-8 Mode (PEP 540) are
disabled by default in ``PyPreConfig``. Set ``coerce_c_locale``,
@ -226,6 +249,188 @@ Initialization with PyConfig
The ``PyConfig`` structure contains all parameters to configure Python.
Example::
PyInitError err;
PyConfig config = PyConfig_INIT;
err = PyConfig_SetString(&config.program_name, L"my_program");
if (_Py_INIT_FAILED(err)) {
Py_ExitInitError(err);
}
err = Py_InitializeFromConfig(&config);
PyConfig_Clear(&config);
if (Py_INIT_FAILED(err)) {
Py_ExitInitError(err);
}
``PyConfig`` methods:
* ``PyInitError PyConfig_SetString(wchar_t **config_str, const wchar_t *str)``:
Set a config wide string field from *str* (copy the string)
* ``PyInitError PyConfig_DecodeLocale(wchar_t **config_str, const char *str)``:
Decode *str* using ``Py_DecodeLocale()`` and set the result into
``*config_str``. Pre-initialize Python if needed to ensure that
encodings are properly configured.
* ``PyInitError PyConfig_SetArgv(PyConfig *config, int argc, char **argv)``:
Set command line arguments (decode bytes). Pre-initialize Python if
needed to ensure that encodings are properly configured.
* ``PyInitError PyConfig_SetWideArgv(PyConfig *config, int argc, wchar_t **argv)``:
Set command line arguments (wide characters).
* ``PyInitError PyConfig_Read(PyConfig *config)``:
Read all Python configuration
* ``void PyConfig_Clear(PyConfig *config)``:
Release memory
Functions to initialize Python:
* ``PyInitError Py_InitializeFromConfig(const PyConfig *config)``
These functions can be called with *config* set to ``NULL``. The caller
is responsible to handler error using ``Py_INIT_FAILED()`` and
``Py_ExitInitError()``.
PyConfig fields:
* ``argv`` (``PyWideStringList``): ``sys.argv``
* ``base_exec_prefix`` (``wchar_t*``): ``sys.base_exec_prefix``
* ``base_prefix`` (``wchar_t*``): ``sys.base_prefix``
* ``buffered_stdio`` (``int``): if equals to 0, enable unbuffered mode,
make stdout and stderr streams to be unbuffered.
* ``bytes_warning`` (``int``): if equals to 1, issue a warning when
comparing ``bytes`` or ``bytearray`` with ``str``, or comparing
``bytes`` with ``int``. If equal or greater to 2, raise a
``BytesWarning`` exception.
* ``check_hash_pycs_mode`` (``wchar_t*``): ``--check-hash-based-pycs``
command line option value (see PEP 552)
* ``dev_mode`` (``int``): Development mode
* ``dll_path`` (``wchar_t*``, Windows only): Windows DLL path
* ``dump_refs`` (``int``): if non-zero, display all objects still alive
at exit
* ``exec_prefix`` (``wchar_t*``): ``sys.exec_prefix``
* ``executable`` (``wchar_t*``): ``sys.executable``
* ``faulthandler`` (``int``): if non-zero, call
``faulthandler.enable()``
* ``filesystem_encoding`` (``wchar_t*``): Filesystem encoding,
``sys.getfilesystemencoding()``
* ``filesystem_errors`` (``wchar_t*``): Filesystem encoding errors,
``sys.getfilesystemencodeerrors()``
* ``use_hash_seed`` (``int``), ``hash_seed`` (``unsigned long``):
randomized hash function seed
* ``home`` (``wchar_t*``): Python home
* ``import_time`` (``int``): if non-zero, profile import time
* ``inspect`` (``int``): enter interactive mode after executing a script or a
command
* ``install_signal_handlers`` (``int``): install signal handlers?
* ``interactive`` (``int``): interactive mode
* ``legacy_windows_stdio`` (``int``, Windows only): if non-zero, use
``io.FileIO`` instead of ``WindowsConsoleIO`` for ``sys.stdin``,
``sys.stdout`` and ``sys.stderr``.
* ``malloc_stats`` (``int``): if non-zero, dump memory allocation
statistics at exit
* ``module_search_path_env`` (``wchar_t*``): ``PYTHONPATH`` environment variale value
* ``use_module_search_paths`` (``int``), ``module_search_paths``
(``PyWideStringList``): ``sys.path``
* ``optimization_level`` (``int``): compilation optimization level
* ``parser_debug`` (``int``): if non-zero, turn on parser debugging output (for
expert only, depending on compilation options).
* ``prefix`` (``wchar_t*``): ``sys.prefix``
* ``program_name`` (``wchar_t*``): Program name
* ``program`` (``wchar_t*``): ``argv[0]`` or an empty string
* ``pycache_prefix`` (``wchar_t*``): ``.pyc`` cache prefix
* ``quiet`` (``int``): quiet mode (ex: don't display the copyright and version
messages even in interactive mode)
* ``run_command`` (``wchar_t*``): ``-c COMMAND`` argument
* ``run_filename`` (``wchar_t*``): ``python3 SCRIPT`` argument
* ``run_module`` (``wchar_t*``): ``python3 -m MODULE`` argument
* ``show_alloc_count`` (``int``): show allocation counts at exit?
* ``show_ref_count`` (``int``): show total reference count at exit?
* ``site_import`` (``int``): import the ``site`` module at startup?
* ``skip_source_first_line`` (``int``): skip the first line of the source
* ``stdio_encoding`` (``wchar_t*``), ``stdio_errors`` (``wchar_t*``): encoding and encoding errors of
``sys.stdin``, ``sys.stdout`` and ``sys.stderr``
* ``tracemalloc`` (``int``): if non-zero, call
``tracemalloc.start(value)``
* ``user_site_directory`` (``int``): if non-zero, add user site directory to
``sys.path``
* ``verbose`` (``int``): if non-zero, enable verbose mode
* ``warnoptions`` (``PyWideStringList``): options of the ``warnings`` module to build filters
* ``write_bytecode`` (``int``): if non-zero, write ``.pyc`` files
* ``xoptions`` (``PyWideStringList``): ``sys._xoptions``
There are also private fields which are for internal-usage only:
* ``_config_version`` (``int``): Configuration version, used for ABI
compatibility
* ``_frozen`` (``int``): Emit warning when computing the path
configuration?
* ``_install_importlib`` (``int``): Install importlib?
More complete commented example modifying the configuration before
calling ``PyConfig_Read()`` and then modify the read configuration::
PyInitError init_python(const char *program_name)
{
PyInitError err;
PyConfig config = PyConfig_INIT;
/* Set the program name before reading the configuraton
(decode byte string from the locale encoding) */
err = PyConfig_DecodeLocale(&config.program_name,
program_name);
if (_Py_INIT_FAILED(err)) {
goto fail;
}
/* Read all configuration at once */
err = PyConfig_Read(&config);
if (_Py_INIT_FAILED(err)) {
goto fail;
}
/* Append our custom search path to sys.path */
err = PyWideStringList_Append(&config.module_search_paths,
L"/path/to/more/modules");
if (_Py_INIT_FAILED(err)) {
goto fail;
}
/* Override executable computed by PyConfig_Read() */
err = PyConfig_SetString(&config.executable, L"my_executable");
if (_Py_INIT_FAILED(err)) {
goto fail;
}
err = Py_InitializeFromConfig(&config);
/* Py_InitializeFromConfig() copied config which must now be
cleared to release memory */
PyConfig_Clear(&config);
return err;
fail:
PyConfig_Clear(&config);
Py_ExitInitError(err);
}
.. note::
``PyConfig`` does not have any field for extra inittab functions:
``PyImport_AppendInittab()`` and ``PyImport_ExtendInittab()``
functions are still relevant.
Initialization with static PyConfig
-----------------------------------
When no ``PyConfig`` method is used but only
``Py_InitializeFromConfig()``, the caller is responsible for managing
``PyConfig`` memory which means that static strings and static string
lists can be used rather than using dynamically allocated memory. It
can be used for most simple configurations.
Example of Python initialization enabling the isolated mode::
PyConfig config = PyConfig_INIT;
@ -238,82 +443,50 @@ Example of Python initialization enabling the isolated mode::
/* ... use Python API here ... */
Py_Finalize();
In this example, ``PyConfig_Clear()`` is not needed since ``config``
does not contain any dynamically allocated string:
``Py_InitializeFromConfig`` is responsible for filling other fields
and manage the memory.
Functions to initialize Python:
For convenience, two other functions are provided:
* ``PyInitError Py_InitializeFromConfig(const PyConfig *config)``
* ``PyInitError Py_InitializeFromArgs(const PyConfig *config, int argc, char **argv)``
* ``PyInitError Py_InitializeFromWideArgs(const PyConfig *config, int argc, wchar_t **argv)``
These functions can be called with *config* set to ``NULL``. The caller
is responsible to handler error using ``Py_INIT_FAILED()`` and
``Py_ExitInitError()``.
These functions can be used with static ``PyConfig``.
PyConfig fields:
Pseudo-code of ``Py_InitializeFromArgs()``::
* ``argv``: ``sys.argv``
* ``base_exec_prefix``: ``sys.base_exec_prefix``
* ``base_prefix``: ``sys.base_prefix``
* ``buffered_stdio``: if equals to 0, enable unbuffered mode, make
stdout and stderr streams to be unbuffered.
* ``bytes_warning``: if equals to 1, issue a warning when comparing
``bytes`` or ``bytearray`` with ``str``, or comparing ``bytes`` with
``int``. If equal or greater to 2, raise a ``BytesWarning`` exception.
* ``dll_path`` (Windows only): Windows DLL path
* ``dump_refs``: if non-zero, display all objects still alive at exit
* ``exec_prefix``: ``sys.exec_prefix``
* ``executable``: ``sys.executable``
* ``faulthandler``: if non-zero, call ``faulthandler.enable()``
* ``filesystem_encoding``: Filesystem encoding,
``sys.getfilesystemencoding()``
* ``filesystem_errors``: Filesystem encoding errors,
``sys.getfilesystemencodeerrors()``
* ``hash_seed``, ``use_hash_seed``: randomized hash function seed
* ``home``: Python home
* ``import_time``: if non-zero, profile import time
* ``inspect``: enter interactive mode after executing a script or a
command
* ``install_signal_handlers``: install signal handlers?
* ``interactive``: interactive mode
* ``legacy_windows_stdio`` (Windows only): if non-zero, use
``io.FileIO`` instead of ``WindowsConsoleIO`` for ``sys.stdin``,
``sys.stdout`` and ``sys.stderr``.
* ``malloc_stats``: if non-zero, dump memory allocation statistics
at exit
* ``module_search_path_env``: ``PYTHONPATH`` environment variale value
* ``module_search_paths``, ``use_module_search_paths``: ``sys.path``
* ``optimization_level``: compilation optimization level
* ``parser_debug``: if non-zero, turn on parser debugging output (for
expert only, depending on compilation options).
* ``prefix``: ``sys.prefix``
* ``program_name``: Program name
* ``program``: ``argv[0]`` or an empty string
* ``pycache_prefix``: ``.pyc`` cache prefix
* ``quiet``: quiet mode (ex: don't display the copyright and version
messages even in interactive mode)
* ``run_command``: ``-c COMMAND`` argument
* ``run_filename``: ``python3 SCRIPT`` argument
* ``run_module``: ``python3 -m MODULE`` argument
* ``show_alloc_count``: show allocation counts at exit
* ``show_ref_count``: show total reference count at exit
* ``site_import``: import the ``site`` module at startup?
* ``skip_source_first_line``: skip the first line of the source
* ``stdio_encoding``, ``stdio_errors``: encoding and encoding errors of
``sys.stdin``, ``sys.stdout`` and ``sys.stderr``
* ``tracemalloc``: if non-zero, call ``tracemalloc.start(value)``
* ``user_site_directory``: if non-zero, add user site directory to
``sys.path``
* ``verbose``: if non-zero, enable verbose mode
* ``warnoptions``: options of the ``warnings`` module to build filters
* ``write_bytecode``: if non-zero, write ``.pyc`` files
* ``xoptions``: ``sys._xoptions``
PyInitError init_with_args(const PyConfig *src_config, int argc, char **argv)
{
PyInitError err;
PyConfig config = PyConfig_INIT;
There are also private fields which are for internal-usage only:
/* Copy strings and string lists
* (memory dynamically allocated on the heap) */
err = _PyConfig_Copy(&config, src_config);
if (Py_INIT_FAILED(err)) {
goto exit;
}
/* Set config.argv: decode argv bytes. Pre-initialize Python
if needed to ensure that the encodings are properly
configured. */
err = PyConfig_SetArgv(&config, argc, argv);
if (Py_INIT_FAILED(err)) {
goto exit;
}
err = Py_InitializeFromConfig(&config);
exit:
PyConfig_Clear(&config);
return err;
}
where ``_PyConfig_Copy()`` is an internal function. The actual
implementation of ``Py_InitializeFromArgs()`` is more complex.
* ``_check_hash_pycs_mode``
* ``_frozen``
* ``_init_main``
* ``_install_importlib``
Py_UnixMain()
-------------
@ -363,33 +536,21 @@ discussed in PEP 432.
Memory allocations and Py_DecodeLocale()
----------------------------------------
New pre-initialization and initialization APIs use constant
``PyPreConfig`` or ``PyConfig`` structures. If memory is allocated
dynamically, the caller is responsible to release it. Using static
strings is just fine.
Python memory allocation functions like ``PyMem_RawMalloc()`` must not
be used before Python pre-initialization. Using ``malloc()`` and
``free()`` is always safe.
be used before Python pre-initialization. Calling directly ``malloc()``
and ``free()`` is always safe.
``Py_DecodeLocale()`` must only be used after the pre-initialization.
For ``PyPreConfig`` and static ``PyConfig``, the caller is responsible
to manage dynamically allocated strings, but static strings and static
string lists are fine.
Dynamic ``PyConfig`` requires to call ``PyConfig_Clear()`` to release
memory.
XXX Open Questions
==================
``Py_DecodeLocale()`` must not be called before the pre-initialization.
This PEP is still a draft with open questions which should be answered:
* Do we need to add an API for import ``inittab``?
* What about the stable ABI? Should we add a version into
``PyPreConfig`` and ``PyConfig`` structures somehow? The Windows API
is known for its ABI stability and it stores the structure size into
the structure directly. Do the same?
* The PEP 432 stores ``PYTHONCASEOK`` into the config. Do we need
to add something for that into ``PyConfig``? How would it be exposed
at the Python level for ``importlib``? Passed as an argument to
``importlib._bootstrap._setup()`` maybe? It can be added later if
needed.
When using dynanic configuration, ``PyConfig_DecodeLocale()`` must be
used instead of ``Py_DecodeLocale()``.
Backwards Compatibility
@ -519,6 +680,15 @@ Usage::
python3 [options] SCRIPT
Command line options mapped to pseudo-action on ``PyConfig`` fields:
================================ ================================
Option ``PyPreConfig`` field
================================ ================================
``-X dev`` ``dev_mode = 1``
``-X utf8=N`` ``utf8_mode = N``
================================ ================================
Command line options mapped to pseudo-action on ``PyConfig`` fields:
================================ ================================
@ -544,6 +714,13 @@ Option ``PyConfig`` field
``-W WARNING`` add ``WARNING`` to ``warnoptions``
``-x`` ``skip_source_first_line = 1``
``-X XOPTION`` add ``XOPTION`` to ``xoptions``
``-X dev`` ``dev_mode = 1``
``-X faulthandler`` ``faulthandler = 1``
``-X importtime`` ``import_time = 1``
``-X pycache_prefix=PREFIX`` ``pycache_prefix = PREFIX``
``-X show_alloc_count`` ``show_alloc_count = 1``
``-X show_ref_count`` ``show_ref_count = 1``
``-X tracemalloc=N`` ``tracemalloc = N``
================================ ================================
``-h``, ``-?`` and ``-V`` options are handled outside ``PyConfig``.