iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 432: Reframe as core init vs main interpreter init

This commit is contained in:
Nick Coghlan 2015-06-30 21:43:19 +10:00
parent f6830b511b
commit 6847250c7a
1 changed files with 141 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

280
pep-0432.txt
View File

@ -28,50 +28,42 @@ implementation is developed.
Proposal Proposal
======== ========
This PEP proposes that CPython move to an explicit multi-phase initialization This PEP proposes that initialization of the CPython runtime be split into
process, where a preliminary interpreter is put in place with limited OS two clearly distinct phases:
interaction capabilities early in the startup sequence. This essential core
remains in place while all of the configuration settings are determined, * core runtime initialization
until a final configuration call takes those settings and finishes * main interpreter initialization
bootstrapping the interpreter immediately before locating and executing
the main module. The proposed design also has significant implications for:
* main module execution
* subinterpreter initialization
In the new design, the interpreter will move through the following In the new design, the interpreter will move through the following
well-defined phases during the initialization sequence: well-defined phases during the initialization sequence:
* Pre-Initialization - no interpreter available * Pre-Initialization - no interpreter available
* Initializing - interpreter partially available * Core Initialized - main interpreter partially available,
* Initialized - interpreter available, __main__ related metadata subinterpreter creation not yet available
incomplete * Initialized - main interpreter fully available, subinterpreter creation
available
With the interpreter itself fully initialised, main module execution will
then proceed through two phases:
* Main Preparation - __main__ related metadata populated
* Main Execution - bytecode executing in the __main__ module namespace
(Embedding applications may choose not to use the Main Preparation and
Execution phases)
As a concrete use case to help guide any design changes, and to solve a known As a concrete use case to help guide any design changes, and to solve a known
problem where the appropriate defaults for system utilities differ from those problem where the appropriate defaults for system utilities differ from those
for running user scripts, this PEP also proposes the creation and for running user scripts, this PEP also proposes the creation and
distribution of a separate system Python (``pysystem``) executable distribution of a separate system Python (``pysystem``) executable
which, by default, ignores user site directories and environment variables, which, by default, operates in "isolated mode" (as selected by the CPython
and does not implicitly set ``sys.path[0]`` based on the current directory ``-I`` switch).
or the script being executed (it will, however, still support virtual
environments).
To keep the implementation complexity under control, this PEP does *not* To keep the implementation complexity under control, this PEP does *not*
propose wholesale changes to the way the interpreter state is accessed at propose wholesale changes to the way the interpreter state is accessed at
runtime. Changing the order in which the existing initialization steps runtime. Changing the order in which the existing initialization steps
occur in order to make occur in order to make the startup sequence easier to maintain is already a
the startup sequence easier to maintain is already a substantial change, and substantial change, and attempting to make those other changes at the same time
attempting to make those other changes at the same time will make the will make the change significantly more invasive and much harder to review.
change significantly more invasive and much harder to review. However, such However, such proposals may be suitable topics for follow-on PEPs or patches
proposals may be suitable topics for follow-on PEPs or patches - one key - one key benefit of this PEP is decreasing the coupling between the internal
benefit of this PEP is decreasing the coupling between the internal storage storage model and the configuration interface, so such changes should be easier
model and the configuration interface, so such changes should be easier
once this PEP has been implemented. once this PEP has been implemented.
@ -94,10 +86,10 @@ API cannot be used safely.
A number of proposals are on the table for even *more* sophisticated A number of proposals are on the table for even *more* sophisticated
startup behaviour, such as better control over ``sys.path`` startup behaviour, such as better control over ``sys.path``
initialization (easily adding additional directories on the command line initialization (e.g. easily adding additional directories on the command line
in a cross-platform fashion [7_], as well as controlling the configuration of in a cross-platform fashion [7_], controlling the configuration of
``sys.path[0]`` [8_]), easier configuration of utilities like coverage ``sys.path[0]`` [8_]), easier configuration of utilities like coverage
tracing when launching Python subprocesses [9_]. tracing when launching Python subprocesses [9_]).
Rather than continuing to bolt such behaviour onto an already complicated Rather than continuing to bolt such behaviour onto an already complicated
system, this PEP proposes to start simplifying the status quo by introducing system, this PEP proposes to start simplifying the status quo by introducing
@ -259,54 +251,33 @@ Three distinct interpreter initialisation phases are proposed:
* Pre-Initialization: * Pre-Initialization:
* no interpreter is available. * no interpreter is available.
* ``Py_IsInitializing()`` returns ``0`` * ``Py_IsCoreInitialized()`` returns ``0``
* ``Py_IsInitialized()`` returns ``0`` * ``Py_IsInitialized()`` returns ``0``
* The embedding application determines the settings required to create the * The embedding application determines the settings required to create the
main interpreter and moves to the next phase by calling main interpreter and moves to the next phase by calling
``Py_BeginInitialization``. ``Py_InitializationCore``.
* Initializing: * Core Initialized:
* the main interpreter is available, but only partially configured. * the main interpreter is available, but only partially configured.
* ``Py_IsInitializing()`` returns ``1`` * ``Py_IsCoreInitialized()`` returns ``1``
* ``Py_IsInitialized()`` returns ``0`` * ``Py_IsInitialized()`` returns ``0``
* The embedding application determines and applies the settings * The embedding application determines and applies the settings
required to complete the initialization process by calling required to complete the initialization process by calling
``Py_ReadConfig`` and ``Py_EndInitialization``. ``Py_ReadMainInterpreterConfig`` and ``Py_InitializeMainInterpreter``.
* Initialized: * Initialized:
* the main interpreter is available and fully operational, but * the main interpreter is available and fully operational, but
``__main__`` related metadata is incomplete ``__main__`` related metadata is incomplete
* ``Py_IsInitializing()`` returns ``0`` * ``Py_IsCoreInitialized()`` returns ``1``
* ``Py_IsInitialized()`` returns ``1`` * ``Py_IsInitialized()`` returns ``1``
Main Execution Phases
---------------------
After initializing the interpreter, the embedding application may continue
on to execute code in the ``__main__`` module namespace.
* Main Preparation:
* subphase of Initialized (not separately identified at runtime)
* fully populates ``__main__`` related metadata
* may execute code in ``__main__`` namespace (e.g. ``PYTHONSTARTUP``)
* invoked as ``PyRun_PrepareMain``
* Main Execution:
* subphase of Initialized (not separately identified at runtime)
* user supplied bytecode is being executed in the ``__main__`` namespace
* invoked as ``PyRun_ExecMain``
Invocation of Phases Invocation of Phases
-------------------- --------------------
All listed phases will be used by the standard CPython interpreter and the All listed phases will be used by the standard CPython interpreter and the
proposed System Python interpreter. Other embedding applications may proposed System Python interpreter.
choose to skip the step of executing code in the ``__main__`` namespace.
An embedding application may still continue to leave initialization almost An embedding application may still continue to leave initialization almost
entirely under CPython's control by using the existing ``Py_Initialize`` entirely under CPython's control by using the existing ``Py_Initialize``
@ -317,21 +288,21 @@ over the initialization process::
/* Phase 1: Pre-Initialization */ /* Phase 1: Pre-Initialization */
PyCoreConfig core_config = PyCoreConfig_INIT; PyCoreConfig core_config = PyCoreConfig_INIT;
PyConfig config = PyConfig_INIT; PyMainInterpreterConfig config = PyMainInterpreterConfig_INIT;
/* Easily control the core configuration */ /* Easily control the core configuration */
core_config.ignore_environment = 1; /* Ignore environment variables */ core_config.ignore_environment = 1; /* Ignore environment variables */
core_config.use_hash_seed = 0; /* Full hash randomisation */ core_config.use_hash_seed = 0; /* Full hash randomisation */
Py_BeginInitialization(&core_config); Py_InitializeCore(&core_config);
/* Phase 2: Initialization */ /* Phase 2: Initialization */
/* Optionally preconfigure some settings here - they will then be /* Optionally preconfigure some settings here - they will then be
* used to derive other settings */ * used to derive other settings */
Py_ReadConfig(&config); Py_ReadMainInterpreterConfig(&config);
/* Can completely override derived settings here */ /* Can completely override derived settings here */
Py_EndInitialization(&config); Py_InitializeMainInterpreter(&config);
/* Phase 3: Initialized */ /* Phase 3: Initialized */
/* If an embedding application has no real concept of a main module /* If an embedding application has no real concept of a main module
* it can just stop the initialization process here. * it can just stop the initialization process here.
* Alternatively, it can launch __main__ via the PyRun_*Main functions. * Alternatively, it can launch __main__ via the relevant API functions.
*/ */
@ -356,7 +327,7 @@ system.
The proposed API for this step in the startup sequence is:: The proposed API for this step in the startup sequence is::
void Py_BeginInitialization(const PyCoreConfig *config); void Py_InitializeCore(const PyCoreConfig *config);
Like ``Py_Initialize``, this part of the new API treats initialization Like ``Py_Initialize``, this part of the new API treats initialization
failures failures
@ -366,7 +337,7 @@ to return error codes instead of aborting would be an even larger task than
the one already being proposed. the one already being proposed.
The new ``PyCoreConfig`` struct holds the settings required for preliminary The new ``PyCoreConfig`` struct holds the settings required for preliminary
configuration:: configuration of the core runtime and creation of the main interpreter::
/* Note: if changing anything in PyCoreConfig, also update /* Note: if changing anything in PyCoreConfig, also update
* PyCoreConfig_INIT */ * PyCoreConfig_INIT */
@ -435,18 +406,21 @@ The aim is to keep this initial level of configuration as small as possible
in order to keep the bootstrapping environment consistent across in order to keep the bootstrapping environment consistent across
different embedding applications. If we can create a valid interpreter state different embedding applications. If we can create a valid interpreter state
without the setting, then the setting should go in the configuration passed without the setting, then the setting should go in the configuration passed
to ``Py_EndInitialization()`` rather than in the core configuration. to ``Py_InitializeMainInterpreter()`` rather than in the core configuration.
A new query API will allow code to determine if the interpreter is in the A new query API will allow code to determine if the interpreter is in the
bootstrapping state between the creation of the interpreter state and the bootstrapping state between the creation of the interpreter state and the
completion of the bulk of the initialization process:: completion of the bulk of the initialization process::
int Py_IsInitializing(); int Py_IsCoreInitialized();
Attempting to call ``Py_BeginInitialization()`` again when Attempting to call ``Py_InitializeCore()`` again when
``Py_IsInitializing()`` or ``Py_IsInitialized()`` is true is a fatal error. ``Py_IsCoreInitialized()`` is true is a fatal error.
While in the initializing state, the interpreter should be fully functional As frozen bytecode may now be legitimately run in an interpreter which is not
yet fully initialized, ``sys.flags`` will gain a new ``initialized`` flag.
With the core runtime initialised, the interpreter should be fully functional
except that: except that:
* compilation is not allowed (as the parser and compiler are not yet * compilation is not allowed (as the parser and compiler are not yet
@ -463,23 +437,25 @@ except that:
* ``sys.exec_prefix`` * ``sys.exec_prefix``
* ``sys.prefix`` * ``sys.prefix``
* ``sys.warnoptions`` * ``sys.warnoptions``
* ``sys.flags``
* ``sys.dont_write_bytecode`` * ``sys.dont_write_bytecode``
* ``sys.stdin`` * ``sys.stdin``
* ``sys.stdout`` * ``sys.stdout``
* The filesystem encoding is not yet defined * The filesystem encoding is not yet defined
* The IO encoding is not yet defined * The IO encoding is not yet defined
* CPython signal handlers are not yet installed * CPython signal handlers are not yet installed
* only builtin and frozen modules may be imported (due to above limitations) * Only builtin and frozen modules may be imported (due to above limitations)
* ``sys.stderr`` is set to a temporary IO object using unbuffered binary * ``sys.stderr`` is set to a temporary IO object using unbuffered binary
mode mode
* The ``sys.flags`` attribute exists, but may contain flags may not yet
have their final values.
* The ``sys.flags.initialized`` attribute is set to ``0``
* The ``warnings`` module is not yet initialized * The ``warnings`` module is not yet initialized
* The ``__main__`` module does not yet exist * The ``__main__`` module does not yet exist
<TBD: identify any other notable missing functionality> <TBD: identify any other notable missing functionality>
The main things made available by this step will be the core Python The main things made available by this step will be the core Python
datatypes, in particular dictionaries, lists and strings. This allows them data types, in particular dictionaries, lists and strings. This allows them
to be used safely for all of the remaining configuration steps (unlike the to be used safely for all of the remaining configuration steps (unlike the
status quo). status quo).
@ -487,9 +463,10 @@ In addition, the current thread will possess a valid Python thread state,
allowing any further configuration data to be stored on the interpreter allowing any further configuration data to be stored on the interpreter
object rather than in C process globals. object rather than in C process globals.
Any call to ``Py_BeginInitialization()`` must have a matching call to Any call to ``Py_InitializeCore()`` must have a matching call to
``Py_Finalize()``. It is acceptable to skip calling Py_EndInitialization() in ``Py_Finalize()``. It is acceptable to skip calling
between (e.g. if attempting to read the configuration settings fails) ``Py_InitializeMainInterpreter()`` in between (e.g. if attempting to read the
main interpreter configuration settings fails)
Determining the remaining configuration settings Determining the remaining configuration settings
@ -499,7 +476,7 @@ The next step in the initialization sequence is to determine the full
settings needed to complete the process. No changes are made to the settings needed to complete the process. No changes are made to the
interpreter state at this point. The core API for this step is:: interpreter state at this point. The core API for this step is::
int Py_ReadConfig(PyConfig *config); int Py_ReadMainInterpreterConfig(PyMainInterpreterConfig *config);
The config argument should be a pointer to a config struct (which may be The config argument should be a pointer to a config struct (which may be
a temporary one stored on the C stack). For any already configured value a temporary one stored on the C stack). For any already configured value
@ -512,35 +489,47 @@ CPython version and only a read-only view needs to be exposed to Python
code (which is relatively straightforward, thanks to the infrastructure code (which is relatively straightforward, thanks to the infrastructure
already put in place to expose ``sys.implementation``). already put in place to expose ``sys.implementation``).
Unlike ``Py_Initialize`` and ``Py_BeginInitialization``, this call will raise Unlike ``Py_Initialize`` and ``Py_InitializeCore``, this call will raise
an exception and report an error return rather than exhibiting fatal errors an exception and report an error return rather than exhibiting fatal errors
if a problem is found with the config data. if a problem is found with the config data.
Any supported configuration setting which is not already set will be Any supported configuration setting which is not already set will be
populated appropriately in the supplied configuration struct. The default populated appropriately in the supplied configuration struct. The default
configuration can be overridden entirely by setting the value *before* configuration can be overridden entirely by setting the value *before*
calling ``Py_ReadConfiguration``. The provided value will then also be used calling ``Py_ReadMainInterpreterConfig``. The provided value will then also be
in calculating any other settings derived from that value. used in calculating any other settings derived from that value.
Alternatively, settings may be overridden *after* the Alternatively, settings may be overridden *after* the
``Py_ReadConfiguration`` call (this can be useful if an embedding ``Py_ReadMainInterpreterConfig`` call (this can be useful if an embedding
application wants to adjust a setting rather than replace it completely, application wants to adjust a setting rather than replace it completely,
such as removing ``sys.path[0]``). such as removing ``sys.path[0]``).
Merely reading the configuration has no effect on the interpreter state: it Merely reading the configuration has no effect on the interpreter state: it
only modifies the passed in configuration struct. The settings are not only modifies the passed in configuration struct. The settings are not
applied to the running interpreter until the ``Py_EndInitialization`` call applied to the running interpreter until the ``Py_InitializeMainInterpreter``
(see below). call (see below).
Supported configuration settings Supported configuration settings
-------------------------------- --------------------------------
The new ``PyConfig`` struct holds the settings required to complete the The interpreter configuration is split into two parts: settings which are
interpreter configuration. All fields are either pointers to Python either relevant only to the main interpreter or must be identical across the
data types (not set == ``NULL``) or numeric flags (not set == ``-1``):: main interpreter and all subinterpreters, and settings which may vary across
subinterpreters.
/* Note: if changing anything in PyConfig, also update PyConfig_INIT */ NOTE: For initial implementation purposes, only the flag indicating whether
or not the interpreter is the main interpreter will be configured on a per
interpreter basis. Other fields will be reviewed for whether or not they can
feasibly be made interpreter specific over the course of the implementation.
The ``PyMainInterpreterConfig`` struct holds the settings required to
complete the main interpreter configuration. These settings are also all
passed through unmodified to subinterpreters. Fields are either pointers to
Python data types (not set == ``NULL``) or numeric flags (not set == ``-1``)::
/* Note: if changing anything in PyMainInterpreterConfig, also update
* PyMainInterpreterConfig_INIT */
typedef struct { typedef struct {
/* Argument processing */ /* Argument processing */
PyListObject *raw_argv; PyListObject *raw_argv;
@ -613,10 +602,10 @@ data types (not set == ``NULL``) or numeric flags (not set == ``-1``)::
int show_banner; /* -q switch (inverted) */ int show_banner; /* -q switch (inverted) */
int inspect_main; /* -i switch, PYTHONINSPECT */ int inspect_main; /* -i switch, PYTHONINSPECT */
} PyConfig; } PyMainInterpreterConfig;
/* Struct initialization is pretty ugly in C89. Avoiding this mess would /* Struct initialization is pretty horrible in C89. Avoiding this mess would
* be the most attractive aspect of using a PyDictObject* instead... */ * be the most attractive aspect of using a PyDictObject* instead... */
#define _PyArgConfig_INIT NULL, NULL, NULL, NULL #define _PyArgConfig_INIT NULL, NULL, NULL, NULL
#define _PyLocationConfig_INIT NULL, NULL, NULL, NULL, NULL, NULL #define _PyLocationConfig_INIT NULL, NULL, NULL, NULL, NULL, NULL
@ -631,13 +620,28 @@ data types (not set == ``NULL``) or numeric flags (not set == ``-1``)::
#define _PyMainConfig_INIT -1, NULL, NULL, NULL, NULL, NULL, -1 #define _PyMainConfig_INIT -1, NULL, NULL, NULL, NULL, NULL, -1
#define _PyInteractiveConfig_INIT NULL, -1, -1 #define _PyInteractiveConfig_INIT NULL, -1, -1
#define PyConfig_INIT {_PyArgConfig_INIT, _PyLocationConfig_INIT, #define PyMainInterpreterConfig_INIT {
_PyArgConfig_INIT, _PyLocationConfig_INIT,
_PySiteConfig_INIT, _PyImportConfig_INIT, _PySiteConfig_INIT, _PyImportConfig_INIT,
_PyStreamConfig_INIT, _PyFilesystemConfig_INIT, _PyStreamConfig_INIT, _PyFilesystemConfig_INIT,
_PyDebuggingConfig_INIT, _PyCodeGenConfig_INIT, _PyDebuggingConfig_INIT, _PyCodeGenConfig_INIT,
_PySignalConfig_INIT, _PyImplicitConfig_INIT, _PySignalConfig_INIT, _PyImplicitConfig_INIT,
_PyMainConfig_INIT, _PyInteractiveConfig_INIT} _PyMainConfig_INIT, _PyInteractiveConfig_INIT}
The ``PyInterpreterConfig`` struct holds the settings that may vary between
the main interpreter and subinterpreters. For the main interpreter, these
settings are automatically populated by ``Py_InitializeMainInterpreter()``.
::
/* Note: if changing anything in PyInterpreterConfig, also update
* PyInterpreterConfig_INIT */
typedef struct {
int is_main_interpreter; /* Easily check for subinterpreters */
} PyInterpreterConfig;
#define PyInterpreterConfig_INIT {0}
<TBD: did I miss anything?> <TBD: did I miss anything?>
@ -645,26 +649,25 @@ Completing the interpreter initialization
----------------------------------------- -----------------------------------------
The final step in the initialization process is to actually put the The final step in the initialization process is to actually put the
configuration settings into effect and finish bootstrapping the interpreter configuration settings into effect and finish bootstrapping the main
up to full operation:: interpreter up to full operation::
int Py_EndInitialization(const PyConfig *config); int Py_InitializeMainInterpreter(const PyMainInterpreterConfig *config);
Like Py_ReadConfiguration, this call will raise an exception and report an Like ``Py_ReadMainInterpreterConfig``, this call will raise an exception and
error return rather than exhibiting fatal errors if a problem is found with report an error return rather than exhibiting fatal errors if a problem is
the config data. found with the config data.
All configuration settings are required - the configuration struct All configuration settings are required - the configuration struct
should always be passed through ``Py_ReadConfig()`` to ensure it should always be passed through ``Py_ReadMainInterpreterConfig`` to ensure it
is fully populated. is fully populated.
After a successful call, ``Py_IsInitializing()`` will be false, while After a successful call ``Py_IsInitialized()`` will become true. The caveats
``Py_IsInitialized()`` will become true. The caveats described above for the described above for the interpreter during the phase where only the core
interpreter during the initialization phase will no longer hold. runtime is initialized will no longer hold.
Attempting to call ``Py_EndInitialization()`` again when Attempting to call ``Py_InitializeMainInterpreter()`` again when
``Py_IsInitializing()`` is false or ``Py_IsInitialized()`` is true is an ``Py_IsInitialized()`` is true is an error.
error.
However, some metadata related to the ``__main__`` module may still be However, some metadata related to the ``__main__`` module may still be
incomplete: incomplete:
@ -702,6 +705,10 @@ It is handled by calling the following API::
int PyRun_PrepareMain(); int PyRun_PrepareMain();
This operation is only permitted for the main interpreter, and will raise
``RuntimeError`` when invoked from a thread where the current thread state
belongs to a subinterpreter.
The actual processing is driven by the main related settings stored in The actual processing is driven by the main related settings stored in
the interpreter state as part of the configuration struct. the interpreter state as part of the configuration struct.
@ -760,6 +767,10 @@ It is handled by calling the following API::
int PyRun_ExecMain(); int PyRun_ExecMain();
This operation is only permitted for the main interpreter, and will raise
``RuntimeError`` when invoked from a thread where the current thread state
belongs to a subinterpreter.
The actual processing is driven by the main related settings stored in The actual processing is driven by the main related settings stored in
the interpreter state as part of the configuration struct. the interpreter state as part of the configuration struct.
@ -771,22 +782,22 @@ be reported.
If ``main_stream`` and ``prompt_stream`` are both set, main execution will If ``main_stream`` and ``prompt_stream`` are both set, main execution will
be delegated to a new API:: be delegated to a new API::
int PyRun_InteractiveMain(PyObject *input, PyObject* output); int _PyRun_InteractiveMain(PyObject *input, PyObject* output);
If ``main_stream`` is set and ``prompt_stream`` is NULL, main execution will If ``main_stream`` is set and ``prompt_stream`` is NULL, main execution will
be delegated to a new API:: be delegated to a new API::
int PyRun_StreamInMain(PyObject *input); int _PyRun_StreamInMain(PyObject *input);
If ``main_code`` is set, main execution will be delegated to a new If ``main_code`` is set, main execution will be delegated to a new
API:: API::
int PyRun_CodeInMain(PyCodeObject *code); int _PyRun_CodeInMain(PyCodeObject *code);
After execution of main completes, if ``inspect_main`` is set, or After execution of main completes, if ``inspect_main`` is set, or
the ``PYTHONINSPECT`` environment variable has been set, then the ``PYTHONINSPECT`` environment variable has been set, then
``PyRun_ExecMain`` will invoke ``PyRun_ExecMain`` will invoke
``PyRun_InteractiveMain(sys.__stdin__, sys.__stdout__)``. ``_PyRun_InteractiveMain(sys.__stdin__, sys.__stdout__)``.
Internal Storage of Configuration Data Internal Storage of Configuration Data
@ -794,8 +805,8 @@ Internal Storage of Configuration Data
The interpreter state will be updated to include details of the configuration The interpreter state will be updated to include details of the configuration
settings supplied during initialization by extending the interpreter state settings supplied during initialization by extending the interpreter state
object with an embedded copy of the ``PyCoreConfig`` and ``PyConfig`` object with an embedded copy of the ``PyCoreConfig``,
structs. ``PyMainInterpreterConfig`` and ``PyInterpreterConfig`` structs.
For debugging purposes, the configuration settings will be exposed as For debugging purposes, the configuration settings will be exposed as
a ``sys._configuration`` simple namespace (similar to ``sys.flags`` and a ``sys._configuration`` simple namespace (similar to ``sys.flags`` and
@ -838,7 +849,7 @@ will be used.
While the existing ``Py_InterpreterState_Head()`` API could be used instead, While the existing ``Py_InterpreterState_Head()`` API could be used instead,
that reference changes as subinterpreters are created and destroyed, while that reference changes as subinterpreters are created and destroyed, while
``PyInterpreterState_Main()`` will always refer to the initial interpreter ``PyInterpreterState_Main()`` will always refer to the initial interpreter
state created in ``Py_BeginInitialization()``. state created in ``Py_InitializeCore()``.
A new constraint is also added to the embedding API: attempting to delete A new constraint is also added to the embedding API: attempting to delete
the main interpreter while subinterpreters still exist will now be a fatal the main interpreter while subinterpreters still exist will now be a fatal
@ -853,7 +864,7 @@ embedding a Python interpreter involves a much higher degree of coupling
than merely writing an extension. than merely writing an extension.
The only newly exposed API that will be part of the stable ABI is the The only newly exposed API that will be part of the stable ABI is the
``Py_IsInitializing()`` query. ``Py_IsCoreInitialized()`` query.
Build time configuration Build time configuration
@ -868,10 +879,10 @@ Backwards Compatibility
----------------------- -----------------------
Backwards compatibility will be preserved primarily by ensuring that Backwards compatibility will be preserved primarily by ensuring that
``Py_ReadConfig()`` interrogates all the previously defined ``Py_ReadMainInterpreterConfig()`` interrogates all the previously defined
configuration settings stored in global variables and environment variables, configuration settings stored in global variables and environment variables,
and that ``Py_EndInitialization()`` writes affected settings back to the and that ``Py_InitializeMainInterpreter()`` writes affected settings back to
relevant locations. the relevant locations.
One acknowledged incompatiblity is that some environment variables which One acknowledged incompatiblity is that some environment variables which
are currently read lazily may instead be read once during interpreter are currently read lazily may instead be read once during interpreter
@ -892,7 +903,7 @@ does today, ensuring that ``sys.argv`` is not populated until a subsequent
``PySys_SetArgv`` call. All APIs that currently support being called ``PySys_SetArgv`` call. All APIs that currently support being called
prior to ``Py_Initialize()`` will prior to ``Py_Initialize()`` will
continue to do so, and will also support being called prior to continue to do so, and will also support being called prior to
``Py_BeginInitialization()``. ``Py_InitializeCore()``.
To minimise unnecessary code churn, and to ensure the backwards compatibility To minimise unnecessary code churn, and to ensure the backwards compatibility
is well tested, the main CPython executable may continue to use some elements is well tested, the main CPython executable may continue to use some elements
@ -909,7 +920,7 @@ aspects are the fact that user site directories are enabled,
environment variables are trusted and that the directory containing the environment variables are trusted and that the directory containing the
executed file is placed at the beginning of the import path. executed file is placed at the beginning of the import path.
Issue 16499 [6_] proposes adding a ``-I`` option to change the behaviour of Issue 16499 [6_] added a ``-I`` option to change the behaviour of
the normal CPython executable, but this is a hard to discover solution (and the normal CPython executable, but this is a hard to discover solution (and
adds yet another option to an already complex CLI). This PEP proposes to adds yet another option to an already complex CLI). This PEP proposes to
instead add a separate ``pysystem`` executable instead add a separate ``pysystem`` executable
@ -940,19 +951,19 @@ argument parsing infrastructure for use during the initializing phase.
Open Questions Open Questions
============== ==============
* Error details for Py_ReadConfiguration and Py_EndInitialization (these * Error details for ``Py_ReadMainInterpreterConfig`` and
should become clear as the implementation progresses) ``Py_InitializeMainInterpreter`` (these should become clearer as the
* Should there be ``Py_PreparingMain()`` and ``Py_RunningMain()`` query APIs? implementation progresses)
* Should the answer to ``Py_IsInitialized()`` be exposed via the ``sys`` * Is initialisation of the ``PyMainInterpreterConfig`` struct too unwieldy to
module? be maintainable? Would a Python dictionary be a better choice, despite
* Is initialisation of the ``PyConfig`` struct too unwieldy to be being harder to work with from C code? Can we upgrade to requiring a C99
maintainable? Would a Python dictionary be a better choice, despite compatible compiler?
being harder to work with from C code? * Would it be better to manage the flag variables in ``PyMainInterpreterConfig``
* Would it be better to manage the flag variables in ``PyConfig`` as as Python integers or as "negative means false, positive means true, zero
Python integers or as "negative means false, positive means true, zero
means not set" so the struct can be initialized with a simple means not set" so the struct can be initialized with a simple
``memset(&config, 0, sizeof(*config))``, eliminating the need to update ``memset(&config, 0, sizeof(*config))``, eliminating the need to update
both PyConfig and PyConfig_INIT when adding new fields? both PyMainInterpreterConfig and PyMainInterpreterConfig_INIT when adding
new fields?
* The name of the new system Python executable is a bikeshed waiting to be * The name of the new system Python executable is a bikeshed waiting to be
painted. The 3 options considered so far are ``spython``, ``pysystem`` painted. The 3 options considered so far are ``spython``, ``pysystem``
and ``python-minimal``. The PEP text reflects my current preferred choice and ``python-minimal``. The PEP text reflects my current preferred choice
@ -969,15 +980,6 @@ for other pull requests to be feasible just yet. Once the overall design
settles down and it's a matter of migrating individual settings over to settles down and it's a matter of migrating individual settings over to
the new design, that level of collaboration should become more practical. the new design, that level of collaboration should become more practical.
As the number of application binaries created by the build process is now
four, the reference implementation also creates a new top level "Apps"
directory in the CPython source tree. The source files for the main
``python`` binary and the new ``pysystem`` binary will be located in that
directory. The source files for the ``_freeze_importlib`` binary and the
``_testembed`` binary have been moved out of the Modules directory (which
is intended for CPython builtin and extension modules) and into the Tools
directory.
The Status Quo The Status Quo
============== ==============