PEP 432: update to target 3.7/3.8
- timeline updates based on completely missing the 3.6/3.7 targets - API design & implementation strategy updates based on recent discussions with Eric Snow, Steve Dower & Brett Cannon
This commit is contained in:
Nick Coghlan
parent
361599ec4b
commit
9f3a93dc85
293
pep-0432.txt
293
pep-0432.txt
|
|
@ -1,5 +1,5 @@
|
|||
PEP: 432
|
||||
Title: Simplifying the CPython startup sequence
|
||||
Title: Restructuring the CPython startup sequence
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||||
|
|
@ -7,19 +7,28 @@ Status: Draft
|
|||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 28-Dec-2012
|
||||
Python-Version: 3.6
|
||||
Python-Version: 3.8
|
||||
Post-History: 28-Dec-2012, 2-Jan-2013
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes a mechanism for simplifying the startup sequence for
|
||||
This PEP proposes a mechanism for restructuring the startup sequence for
|
||||
CPython, making it easier to modify the initialization behaviour of the
|
||||
reference interpreter executable, as well as making it easier to control
|
||||
CPython's startup behaviour when creating an alternate executable or
|
||||
embedding it as a Python execution engine inside a larger application.
|
||||
|
||||
When implementation of this proposal is completed, interpreter startup will
|
||||
consist of two clearly distinct and independently configurable phases:
|
||||
|
||||
* Python runtime initialization
|
||||
* Main interpreter configuration
|
||||
|
||||
Changes are also proposed that impact main module execution and subinterpreter
|
||||
initialization.
|
||||
|
||||
Note: TBC = To Be Confirmed, TBD = To Be Determined. The appropriate
|
||||
resolution for most of these should become clearer as the reference
|
||||
implementation is developed.
|
||||
|
|
@ -32,7 +41,7 @@ This PEP proposes that initialization of the CPython runtime be split into
|
|||
two clearly distinct phases:
|
||||
|
||||
* core runtime initialization
|
||||
* main interpreter initialization
|
||||
* main interpreter configuration
|
||||
|
||||
The proposed design also has significant implications for:
|
||||
|
||||
|
|
@ -43,7 +52,7 @@ In the new design, the interpreter will move through the following
|
|||
well-defined phases during the initialization sequence:
|
||||
|
||||
* Pre-Initialization - no interpreter available
|
||||
* Core Initialized - main interpreter partially available,
|
||||
* Runtime Initialized - main interpreter partially available,
|
||||
subinterpreter creation not yet available
|
||||
* Initialized - main interpreter fully available, subinterpreter creation
|
||||
available
|
||||
|
|
@ -94,9 +103,9 @@ in a cross-platform fashion [7_], controlling the configuration of
|
|||
tracing when launching Python subprocesses [9_]).
|
||||
|
||||
Rather than continuing to bolt such behaviour onto an already complicated
|
||||
system, this PEP proposes to start simplifying the status quo by introducing
|
||||
a more structured startup sequence, with the aim of making these further
|
||||
feature requests easier to implement.
|
||||
system indefinitely, this PEP proposes to start simplifying the status quo by
|
||||
introducing a more structured startup sequence, with the aim of making these
|
||||
further feature requests easier to implement.
|
||||
|
||||
|
||||
Key Concerns
|
||||
|
|
@ -116,9 +125,9 @@ as lists, dictionaries and Unicode values being created prior to the call
|
|||
to ``Py_Initialize`` when the ``-X`` or ``-W`` options are used [1_].
|
||||
|
||||
By moving to an explicitly multi-phase startup sequence, developers should
|
||||
only need to understand which features are not available in the core
|
||||
bootstrapping phase, as the vast majority of the configuration process
|
||||
will now take place during that phase.
|
||||
only need to understand which features are not available in the main
|
||||
interpreter configuration phase, as the vast majority of the configuration
|
||||
process will now take place during that phase.
|
||||
|
||||
By basing the new design on a combination of C structures and Python
|
||||
data types, it should also be easier to modify the system in the
|
||||
|
|
@ -150,12 +159,12 @@ tear down the interpreter::
|
|||
|
||||
python3 -m timeit -s "from subprocess import call" "call(['./python', '-c', 'pass'])"
|
||||
|
||||
Current numbers on my system for Python 3.6 (using the 3.5
|
||||
Current numbers on my system for Python 3.7 (using the 3.5
|
||||
subprocess and timeit modules to execute the check, all with non-debug
|
||||
builds)::
|
||||
|
||||
$ python3 -m timeit -s "from subprocess import call" "call(['./python', '-c', 'pass'])"
|
||||
100 loops, best of 3: 15.1 msec per loop
|
||||
100 loops, best of 3: 13.9 msec per loop
|
||||
|
||||
This PEP is not expected to have any significant effect on the startup time,
|
||||
as it is aimed primarily at *reordering* the existing initialization
|
||||
|
|
@ -247,7 +256,7 @@ be able to control the following aspects of the final interpreter state:
|
|||
* Standard input as a script (i.e. a non-interactive stream)
|
||||
* Standard input as an interactive interpreter session
|
||||
|
||||
<TBD: Did I miss anything?>
|
||||
<TBD: What, if anything, is still missing from this list?>
|
||||
|
||||
Note that this just covers settings that are currently configurable in some
|
||||
manner when using the main CPython executable. While this PEP aims to make
|
||||
|
|
@ -270,18 +279,18 @@ structure a draft implementation that won't be prone to the kinds of merge
|
|||
conflicts that afflicted the original attempt.
|
||||
|
||||
Accordingly, the implementation strategy now being proposed is to implement this
|
||||
refactoring as a private API for CPython 3.6, before exposing the new functions
|
||||
and structures as public API elements in CPython 3.7.
|
||||
refactoring as a private API for CPython 3.7, before exposing the new functions
|
||||
and structures as public API elements in CPython 3.8.
|
||||
|
||||
The affected APIs with a leading underscore, as they would be named in CPython
|
||||
3.6:
|
||||
3.7:
|
||||
|
||||
* ``_Py_IsCoreInitialized``
|
||||
* ``_Py_InitializeCore``
|
||||
* ``_PyCoreConfig``
|
||||
* ``_PyCoreConfig_INIT``
|
||||
* ``_Py_IsRuntimeInitialized``
|
||||
* ``_Py_InitializeRuntime``
|
||||
* ``_PyRuntimeConfig``
|
||||
* ``_PyRuntimeConfig_INIT``
|
||||
* ``_Py_ReadHashSeed``
|
||||
* ``_Py_InitializeMainInterpreter``
|
||||
* ``_Py_ConfigureMainInterpreter``
|
||||
* ``_PyMainInterpreterConfig``
|
||||
* ``_PyInterpreterConfig``
|
||||
* ``_Py_ReadMainInterpreterConfig``
|
||||
|
|
@ -294,15 +303,15 @@ intended to be retained permanently as private CPython implementation details.
|
|||
|
||||
The principle benefit of this approach is allowing the refactoring to adopt the
|
||||
new configuration structures to be handled on a setting by setting basis
|
||||
over the course of the 3.6 and 3.7 development cycles, rather than having to
|
||||
migrate them in a single monolithic change. It also means any new settings can
|
||||
be handled using the new approach, even if some existing settings have yet to
|
||||
be migrated.
|
||||
over the course of the 3.7 and 3.8 development cycles, rather than having to
|
||||
migrate them in a single monolithic change. It also means any new settings
|
||||
introduced for these releases can be handled using the new approach from the
|
||||
start, even if some existing settings have yet to be migrated.
|
||||
|
||||
If all existing settings are successfully migrated to the new initialization
|
||||
model in time for the 3.6.0a4 release in August 2016, then the proposal would
|
||||
be to make the APIs public for the 3.6.0b1 release in September 2016, rather
|
||||
than waiting for 3.7.
|
||||
model in time for the 3.7.0a4 release in December 2017, then the proposal would
|
||||
be to make the APIs public for the 3.7.0b1 release in January 2018, rather
|
||||
than waiting for 3.8.
|
||||
|
||||
|
||||
Design Details
|
||||
|
|
@ -311,12 +320,13 @@ Design Details
|
|||
(Note: details here are still very much in flux, but preliminary feedback
|
||||
is appreciated anyway)
|
||||
|
||||
The main theme of this proposal is to create the interpreter state for
|
||||
the main interpreter *much* earlier in the startup process. This will allow
|
||||
most of the CPython API to be used during the remainder of the initialization
|
||||
process, potentially simplifying a number of operations that currently need
|
||||
to rely on basic C functionality rather than being able to use the richer
|
||||
data structures provided by the CPython C API.
|
||||
The main theme of this proposal is to initialize the core language runtime
|
||||
and create a partially initialized interpreter state for the main interpreter
|
||||
*much* earlier in the startup process. This will allow most of the CPython API
|
||||
to be used during the remainder of the initialization process, potentially
|
||||
simplifying a number of operations that currently need to rely on basic C
|
||||
functionality rather than being able to use the richer data structures provided
|
||||
by the CPython C API.
|
||||
|
||||
In the following, the term "embedding application" also covers the standard
|
||||
CPython command line application.
|
||||
|
|
@ -329,29 +339,31 @@ Three distinct interpreter initialisation phases are proposed:
|
|||
|
||||
* Pre-Initialization:
|
||||
|
||||
* no interpreter is available.
|
||||
* ``Py_IsCoreInitialized()`` returns ``0``
|
||||
* no interpreter is available
|
||||
* ``Py_IsRuntimeInitialized()`` returns ``0``
|
||||
* ``Py_IsInitialized()`` returns ``0``
|
||||
* The embedding application determines the settings required to create the
|
||||
main interpreter and moves to the next phase by calling
|
||||
``Py_InitializeCore``.
|
||||
* The embedding application determines the settings required to initialize
|
||||
the core CPython runtime and create the main interpreter and moves to the
|
||||
next phase by calling ``Py_InitializeRuntime``
|
||||
|
||||
* Core Initialized:
|
||||
* Initializing:
|
||||
|
||||
* the main interpreter is available, but only partially configured.
|
||||
* ``Py_IsCoreInitialized()`` returns ``1``
|
||||
* the builtin data types and other core runtime services are available
|
||||
* the main interpreter is available, but only partially configured
|
||||
* ``Py_IsRuntimeInitialized()`` returns ``1``
|
||||
* ``Py_IsInitialized()`` returns ``0``
|
||||
* The embedding application determines and applies the settings
|
||||
required to complete the initialization process by calling
|
||||
``Py_ReadMainInterpreterConfig`` and ``Py_InitializeMainInterpreter``.
|
||||
``Py_ReadMainInterpreterConfig`` and ``Py_ConfigureMainInterpreter``.
|
||||
|
||||
* Initialized:
|
||||
|
||||
* the main interpreter is available and fully operational, but
|
||||
``__main__`` related metadata is incomplete
|
||||
* ``Py_IsCoreInitialized()`` returns ``1``
|
||||
* ``Py_IsRuntimeInitialized()`` returns ``1``
|
||||
* ``Py_IsInitialized()`` returns ``1``
|
||||
|
||||
|
||||
Invocation of Phases
|
||||
--------------------
|
||||
|
||||
|
|
@ -366,18 +378,18 @@ grained API, which allows the embedding application greater control
|
|||
over the initialization process::
|
||||
|
||||
/* Phase 1: Pre-Initialization */
|
||||
PyCoreConfig core_config = PyCoreConfig_INIT;
|
||||
PyMainInterpreterConfig config;
|
||||
PyRuntimeConfig runtime_config = PyRuntimeConfig_INIT;
|
||||
PyMainInterpreterConfig interpreter_config;
|
||||
/* Easily control the core configuration */
|
||||
core_config.ignore_environment = 1; /* Ignore environment variables */
|
||||
core_config.use_hash_seed = 0; /* Full hash randomisation */
|
||||
Py_InitializeCore(&core_config);
|
||||
/* Phase 2: Initialization */
|
||||
runtime_config.ignore_environment = 1; /* Ignore environment variables */
|
||||
runtime_config.use_hash_seed = 0; /* Full hash randomisation */
|
||||
Py_InitializeRuntime(&runtime_config);
|
||||
/* Phase 2: Initializing */
|
||||
/* Optionally preconfigure some settings here - they will then be
|
||||
* used to derive other settings */
|
||||
Py_ReadMainInterpreterConfig(&config);
|
||||
Py_ReadMainInterpreterConfig(&interpreter_config);
|
||||
/* Can completely override derived settings here */
|
||||
Py_InitializeMainInterpreter(&config);
|
||||
Py_ConfigureMainInterpreter(&interpreter_config);
|
||||
/* Phase 3: Initialized */
|
||||
/* If an embedding application has no real concept of a main module
|
||||
* it can just stop the initialization process here.
|
||||
|
|
@ -389,7 +401,7 @@ Pre-Initialization Phase
|
|||
------------------------
|
||||
|
||||
The pre-initialization phase is where an embedding application determines
|
||||
the settings which are absolutely required before the interpreter can be
|
||||
the settings which are absolutely required before the CPython runtime can be
|
||||
initialized at all. Currently, the primary configuration settings in this
|
||||
category are those related to the randomised hash algorithm - the hash
|
||||
algorithms must be consistent for the lifetime of the process, and so they
|
||||
|
|
@ -406,7 +418,7 @@ system.
|
|||
|
||||
The proposed API for this step in the startup sequence is::
|
||||
|
||||
void Py_InitializeCore(const PyCoreConfig *config);
|
||||
void Py_InitializeRuntime(const PyRuntimeConfig *config);
|
||||
|
||||
Like ``Py_Initialize``, this part of the new API treats initialization failures
|
||||
as fatal errors. While that's still not particularly embedding friendly,
|
||||
|
|
@ -414,34 +426,31 @@ the operations in this step *really* shouldn't be failing, and changing them
|
|||
to return error codes instead of aborting would be an even larger task than
|
||||
the one already being proposed.
|
||||
|
||||
The new ``PyCoreConfig`` struct holds the settings required for preliminary
|
||||
The new ``PyRuntimeConfig`` struct holds the settings required for preliminary
|
||||
configuration of the core runtime and creation of the main interpreter::
|
||||
|
||||
/* Note: if changing anything in PyCoreConfig, also update
|
||||
* PyCoreConfig_INIT */
|
||||
/* Note: if changing anything in PyRuntimeConfig, also update
|
||||
* PyRuntimeConfig_INIT */
|
||||
typedef struct {
|
||||
int ignore_environment; /* -E switch, -I switch */
|
||||
bool ignore_environment; /* -E switch, -I switch */
|
||||
int use_hash_seed; /* PYTHONHASHSEED */
|
||||
unsigned long hash_seed; /* PYTHONHASHSEED */
|
||||
int _disable_importlib; /* Needed by freeze_importlib */
|
||||
} PyCoreConfig;
|
||||
bool _disable_importlib; /* Needed by freeze_importlib */
|
||||
} PyRuntimeConfig;
|
||||
|
||||
#define PyCoreConfig_INIT {.use_hash_seed=-1}
|
||||
/* The above assumes PEP 7 is updated to permit use of C99 features
|
||||
* Without designated initialisers, the macro would be:
|
||||
* #define PyCoreConfig_INIT {0, -1, 0, 0}
|
||||
*/
|
||||
/* Rely on the "designated initializer" feature of C99 */
|
||||
#define PyRuntimeConfig_INIT {.use_hash_seed=-1}
|
||||
|
||||
The core configuration settings pointer may be ``NULL``, in which case the
|
||||
default values are ``ignore_environment = -1`` and ``use_hash_seed = -1``.
|
||||
default values are as specified in ``PyRuntimeConfig_INIT``.
|
||||
|
||||
The ``PyCoreConfig_INIT`` macro is designed to allow easy initialization
|
||||
The ``PyRuntimeConfig_INIT`` macro is designed to allow easy initialization
|
||||
of a struct instance with sensible defaults::
|
||||
|
||||
PyCoreConfig core_config = PyCoreConfig_INIT;
|
||||
PyRuntimeConfig runtime_config = PyRuntimeConfig_INIT;
|
||||
|
||||
``ignore_environment`` controls the processing of all Python related
|
||||
environment variables. If the flag is zero, then environment variables are
|
||||
environment variables. If the flag is false, then environment variables are
|
||||
processed normally. Otherwise, all Python-specific environment variables
|
||||
are considered undefined (exceptions may be made for some OS specific
|
||||
environment variables, such as those used on Mac OS X to communicate
|
||||
|
|
@ -480,30 +489,32 @@ value indicates an error (most likely in the conversion to an integer).
|
|||
The ``_disable_importlib`` setting is used as part of the CPython build
|
||||
process to create an interpreter with no import capability at all. It is
|
||||
considered private to the CPython development team (hence the leading
|
||||
underscore), as the only known use case is to permit compiler changes
|
||||
that invalidate the previously frozen bytecode for ``importlib._bootstrap``
|
||||
without breaking the build process.
|
||||
underscore), as the only currently supported use case is to permit compiler
|
||||
changes that invalidate the previously frozen bytecode for
|
||||
``importlib._bootstrap`` without breaking the build process.
|
||||
|
||||
The aim is to keep this initial level of configuration as small as possible
|
||||
in order to keep the bootstrapping environment consistent across
|
||||
different embedding applications. If we can create a valid interpreter state
|
||||
without the setting, then the setting should go in the configuration passed
|
||||
to ``Py_InitializeMainInterpreter()`` rather than in the core configuration.
|
||||
to ``Py_ConfigureMainInterpreter()`` rather than in the core runtime
|
||||
configuration.
|
||||
|
||||
A new query API will allow code to determine if the interpreter is in the
|
||||
bootstrapping state between the creation of the interpreter state and the
|
||||
completion of the bulk of the initialization process::
|
||||
bootstrapping state between the core runtime initialization and the creation of
|
||||
the main interpreter state and the completion of the bulk of the main
|
||||
interpreter initialization process::
|
||||
|
||||
int Py_IsCoreInitialized();
|
||||
int Py_IsRuntimeInitialized();
|
||||
|
||||
Attempting to call ``Py_InitializeCore()`` again when
|
||||
``Py_IsCoreInitialized()`` is already true is a fatal error.
|
||||
Attempting to call ``Py_InitializeRuntime()`` again when
|
||||
``Py_IsRuntimeInitialized()`` is already true is a fatal error.
|
||||
|
||||
As frozen bytecode may now be legitimately run in an interpreter which is not
|
||||
yet fully initialized, ``sys.flags`` will gain a new ``initialized`` flag.
|
||||
|
||||
With the core runtime initialised, the interpreter should be fully functional
|
||||
except that:
|
||||
With the core runtime initialised, the main interpreter and most of the CPython
|
||||
C API should be fully functional except that:
|
||||
|
||||
* compilation is not allowed (as the parser and compiler are not yet
|
||||
configured properly)
|
||||
|
|
@ -542,12 +553,12 @@ to be used safely for all of the remaining configuration steps (unlike the
|
|||
status quo).
|
||||
|
||||
In addition, the current thread will possess a valid Python thread state,
|
||||
allowing any further configuration data to be stored on the interpreter
|
||||
allowing any further configuration data to be stored on the main interpreter
|
||||
object rather than in C process globals.
|
||||
|
||||
Any call to ``Py_InitializeCore()`` must have a matching call to
|
||||
Any call to ``Py_InitializeRuntime()`` must have a matching call to
|
||||
``Py_Finalize()``. It is acceptable to skip calling
|
||||
``Py_InitializeMainInterpreter()`` in between (e.g. if attempting to read the
|
||||
``Py_ConfigureMainInterpreter()`` in between (e.g. if attempting to read the
|
||||
main interpreter configuration settings fails).
|
||||
|
||||
|
||||
|
|
@ -562,8 +573,8 @@ interpreter state at this point. The core API for this step is::
|
|||
|
||||
The config argument should be a pointer to a config struct (which may be
|
||||
a temporary one stored on the C stack). For any already configured value
|
||||
(i.e. non-NULL pointer or non-negative numeric value), CPython will sanity
|
||||
check the supplied value, but otherwise accept it as correct.
|
||||
(i.e. any non-NULL pointer), CPython will sanity check the supplied value,
|
||||
but otherwise accept it as correct.
|
||||
|
||||
A struct is used rather than a Python dictionary as the struct is easier
|
||||
to work with from C, the list of supported fields is fixed for a given
|
||||
|
|
@ -571,7 +582,7 @@ CPython version and only a read-only view needs to be exposed to Python
|
|||
code (which is relatively straightforward, thanks to the infrastructure
|
||||
already put in place to expose ``sys.implementation``).
|
||||
|
||||
Unlike ``Py_Initialize`` and ``Py_InitializeCore``, this call will raise
|
||||
Unlike ``Py_Initialize`` and ``Py_InitializeRuntime``, this call will raise
|
||||
an exception and report an error return rather than exhibiting fatal errors
|
||||
if a problem is found with the config data.
|
||||
|
||||
|
|
@ -588,7 +599,7 @@ such as removing ``sys.path[0]``).
|
|||
|
||||
Merely reading the configuration has no effect on the interpreter state: it
|
||||
only modifies the passed in configuration struct. The settings are not
|
||||
applied to the running interpreter until the ``Py_InitializeMainInterpreter``
|
||||
applied to the running interpreter until the ``Py_ConfigureMainInterpreter``
|
||||
call (see below).
|
||||
|
||||
|
||||
|
|
@ -610,8 +621,6 @@ complete the main interpreter configuration. These settings are also all
|
|||
passed through unmodified to subinterpreters. Fields are always pointers to
|
||||
Python data types, with unset values indicated by ``NULL``::
|
||||
|
||||
/* Note: if changing anything in PyMainInterpreterConfig, also update
|
||||
* PyMainInterpreterConfig_INIT */
|
||||
typedef struct {
|
||||
/* Argument processing */
|
||||
PyListObject *raw_argv;
|
||||
|
|
@ -686,20 +695,20 @@ Python data types, with unset values indicated by ``NULL``::
|
|||
|
||||
} PyMainInterpreterConfig;
|
||||
|
||||
/* Storing all state as Python object pointers */
|
||||
|
||||
The ``PyInterpreterConfig`` struct holds the settings that may vary between
|
||||
the main interpreter and subinterpreters. For the main interpreter, these
|
||||
settings are automatically populated by ``Py_InitializeMainInterpreter()``.
|
||||
settings are automatically populated by ``Py_ConfigureMainInterpreter()``.
|
||||
|
||||
::
|
||||
|
||||
/* Note: if changing anything in PyInterpreterConfig, also update
|
||||
* PyInterpreterConfig_INIT */
|
||||
typedef struct {
|
||||
PyBoolObject *is_main_interpreter; /* Easily check for subinterpreters */
|
||||
} PyInterpreterConfig;
|
||||
|
||||
As these structs consist solely of object pointers, no explicit initializer
|
||||
definitions are needed - C99's default initialization of struct memory to zero
|
||||
is sufficient.
|
||||
|
||||
<TBD: did I miss anything?>
|
||||
|
||||
|
||||
|
|
@ -710,7 +719,7 @@ The final step in the initialization process is to actually put the
|
|||
configuration settings into effect and finish bootstrapping the main
|
||||
interpreter up to full operation::
|
||||
|
||||
int Py_InitializeMainInterpreter(const PyMainInterpreterConfig *config);
|
||||
int Py_ConfigureMainInterpreter(const PyMainInterpreterConfig *config);
|
||||
|
||||
Like ``Py_ReadMainInterpreterConfig``, this call will raise an exception and
|
||||
report an error return rather than exhibiting fatal errors if a problem is
|
||||
|
|
@ -724,7 +733,7 @@ After a successful call ``Py_IsInitialized()`` will become true. The caveats
|
|||
described above for the interpreter during the phase where only the core
|
||||
runtime is initialized will no longer hold.
|
||||
|
||||
Attempting to call ``Py_InitializeMainInterpreter()`` again when
|
||||
Attempting to call ``Py_ConfigureMainInterpreter()`` again when
|
||||
``Py_IsInitialized()`` is true is an error.
|
||||
|
||||
However, some metadata related to the ``__main__`` module may still be
|
||||
|
|
@ -746,10 +755,10 @@ incomplete:
|
|||
builtin module
|
||||
|
||||
This function will normally implicitly import site as its final operation
|
||||
(after ``Py_IsInitialized()`` is already set). Clearing the
|
||||
"enable_site_config" flag in the configuration settings will disable this
|
||||
behaviour, as well as eliminating any side effects on global state if
|
||||
``import site`` is later explicitly executed in the process.
|
||||
(after ``Py_IsInitialized()`` is already set). Setting the
|
||||
"enable_site_config" flag to ``Py_False`` in the configuration settings will
|
||||
disable this behaviour, as well as eliminating any side effects on global
|
||||
state if ``import site`` is later explicitly executed in the process.
|
||||
|
||||
|
||||
Preparing the main module
|
||||
|
|
@ -840,16 +849,16 @@ If both ``main_stream`` and ``main_code`` are set, ``RuntimeError`` will
|
|||
be reported.
|
||||
|
||||
If ``main_stream`` and ``prompt_stream`` are both set, main execution will
|
||||
be delegated to a new API::
|
||||
be delegated to a new internal API::
|
||||
|
||||
int _PyRun_InteractiveMain(PyObject *input, PyObject* output);
|
||||
|
||||
If ``main_stream`` is set and ``prompt_stream`` is NULL, main execution will
|
||||
be delegated to a new API::
|
||||
be delegated to a new internal API::
|
||||
|
||||
int _PyRun_StreamInMain(PyObject *input);
|
||||
|
||||
If ``main_code`` is set, main execution will be delegated to a new
|
||||
If ``main_code`` is set, main execution will be delegated to a new internal
|
||||
API::
|
||||
|
||||
int _PyRun_CodeInMain(PyCodeObject *code);
|
||||
|
|
@ -865,17 +874,24 @@ Internal Storage of Configuration Data
|
|||
|
||||
The interpreter state will be updated to include details of the configuration
|
||||
settings supplied during initialization by extending the interpreter state
|
||||
object with an embedded copy of the ``PyCoreConfig``,
|
||||
object with an embedded copy of the ``PyRuntimeConfig``,
|
||||
``PyMainInterpreterConfig`` and ``PyInterpreterConfig`` structs.
|
||||
|
||||
For debugging purposes, the configuration settings will be exposed as
|
||||
a ``sys._configuration`` simple namespace (similar to ``sys.flags`` and
|
||||
``sys.implementation``. Field names will match those in the configuration
|
||||
structs, except for ``hash_seed``, which will be deliberately excluded.
|
||||
``sys.implementation``. The attributes will be themselves by simple namespaces
|
||||
corresponding to the three levels of configurations setting:
|
||||
|
||||
* ``runtime``
|
||||
* ``main_interpreter``
|
||||
* ``interpreter``
|
||||
|
||||
Field names will match those in the configuration structs, except for
|
||||
``hash_seed``, which will be deliberately excluded.
|
||||
|
||||
An underscored attribute is chosen deliberately, as these configuration
|
||||
settings are part of the CPython implementation, rather than part of the
|
||||
Python language definition. If settings are needed to support
|
||||
Python language definition. If new settings are needed to support
|
||||
cross-implementation compatibility in the standard library, then those
|
||||
should be agreed with the other implementations and exposed as new required
|
||||
attributes on ``sys.implementation``, as described in PEP 421.
|
||||
|
|
@ -889,7 +905,7 @@ Creating and Configuring Subinterpreters
|
|||
|
||||
As the new configuration settings are stored in the interpreter state, they
|
||||
need to be initialised when a new subinterpreter is created. This turns out
|
||||
to be trickier than one might think due to ``PyThreadState_Swap(NULL);``
|
||||
to be trickier than one might expect due to ``PyThreadState_Swap(NULL);``
|
||||
(which is fortunately exercised by CPython's own embedding tests, allowing
|
||||
this problem to be detected during development).
|
||||
|
||||
|
|
@ -898,7 +914,7 @@ add a new API::
|
|||
|
||||
Py_InterpreterState *Py_InterpreterState_Main();
|
||||
|
||||
This will be a counterpart to Py_InterpreterState_Head(), reporting the
|
||||
This will be a counterpart to ``Py_InterpreterState_Head()``, only reporting the
|
||||
oldest currently existing interpreter rather than the newest. If
|
||||
``Py_NewInterpreter()`` is called from a thread with an existing thread
|
||||
state, then the interpreter configuration for that thread will be
|
||||
|
|
@ -909,7 +925,7 @@ will be used.
|
|||
While the existing ``Py_InterpreterState_Head()`` API could be used instead,
|
||||
that reference changes as subinterpreters are created and destroyed, while
|
||||
``PyInterpreterState_Main()`` will always refer to the initial interpreter
|
||||
state created in ``Py_InitializeCore()``.
|
||||
state created in ``Py_InitializeRuntime()``.
|
||||
|
||||
A new constraint is also added to the embedding API: attempting to delete
|
||||
the main interpreter while subinterpreters still exist will now be a fatal
|
||||
|
|
@ -921,10 +937,10 @@ Stable ABI
|
|||
|
||||
Most of the APIs proposed in this PEP are excluded from the stable ABI, as
|
||||
embedding a Python interpreter involves a much higher degree of coupling
|
||||
than merely writing an extension.
|
||||
than merely writing an extension module.
|
||||
|
||||
The only newly exposed API that will be part of the stable ABI is the
|
||||
``Py_IsCoreInitialized()`` query.
|
||||
``Py_IsRuntimeInitialized()`` query.
|
||||
|
||||
|
||||
Build time configuration
|
||||
|
|
@ -941,14 +957,14 @@ Backwards Compatibility
|
|||
Backwards compatibility will be preserved primarily by ensuring that
|
||||
``Py_ReadMainInterpreterConfig()`` interrogates all the previously defined
|
||||
configuration settings stored in global variables and environment variables,
|
||||
and that ``Py_InitializeMainInterpreter()`` writes affected settings back to
|
||||
and that ``Py_ConfigureMainInterpreter()`` writes affected settings back to
|
||||
the relevant locations.
|
||||
|
||||
One acknowledged incompatibility is that some environment variables which
|
||||
are currently read lazily may instead be read once during interpreter
|
||||
initialization. As the PEP matures, these will be discussed in more detail
|
||||
on a case by case basis. The environment variables which are currently
|
||||
known to be looked up dynamically are:
|
||||
initialization. As the reference implementation matures, these will be
|
||||
discussed in more detail on a case by case basis. The environment variables
|
||||
which are currently known to be looked up dynamically are:
|
||||
|
||||
* ``PYTHONCASEOK``: writing to ``os.environ['PYTHONCASEOK']`` will no longer
|
||||
dynamically alter the interpreter's handling of filename case differences
|
||||
|
|
@ -960,10 +976,10 @@ The ``Py_Initialize()`` style of initialization will continue to be
|
|||
supported. It will use (at least some elements of) the new API
|
||||
internally, but will continue to exhibit the same behaviour as it
|
||||
does today, ensuring that ``sys.argv`` is not populated until a subsequent
|
||||
``PySys_SetArgv`` call. All APIs that currently support being called
|
||||
``PySys_SetArgv`` call (TBC). All APIs that currently support being called
|
||||
prior to ``Py_Initialize()`` will
|
||||
continue to do so, and will also support being called prior to
|
||||
``Py_InitializeCore()``.
|
||||
``Py_InitializeRuntime()``.
|
||||
|
||||
To minimise unnecessary code churn, and to ensure the backwards compatibility
|
||||
is well tested, the main CPython executable may continue to use some elements
|
||||
|
|
@ -1012,43 +1028,30 @@ Open Questions
|
|||
==============
|
||||
|
||||
* Error details for ``Py_ReadMainInterpreterConfig`` and
|
||||
``Py_InitializeMainInterpreter`` (these should become clearer as the
|
||||
``Py_ConfigureMainInterpreter`` (these should become clearer as the
|
||||
implementation progresses)
|
||||
* Is initialisation of the ``PyMainInterpreterConfig`` struct too unwieldy to
|
||||
be maintainable? Would a Python dictionary be a better choice, despite
|
||||
being harder to work with from C code? Can we upgrade to requiring a C99
|
||||
compatible compiler?
|
||||
* Would it be better to manage the flag variables in ``PyMainInterpreterConfig``
|
||||
as Python integers or as "negative means false, positive means true, zero
|
||||
means not set" so the struct can be initialized with a simple
|
||||
``memset(&config, 0, sizeof(*config))``, eliminating the need to update
|
||||
both PyMainInterpreterConfig and PyMainInterpreterConfig_INIT when adding
|
||||
new fields?
|
||||
* The name of the new system Python executable is a bikeshed waiting to be
|
||||
painted. The 4 options considered so far are ``spython``, ``pysystem``,
|
||||
``python-minimal`` and `system-python``. The PEP text reflects my current
|
||||
preferred choice (``system-python``).
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
A reference implementation is available as a feature branch in my BitBucket
|
||||
sandbox [2_].
|
||||
A reference implementation is available as a feature branch in Nick Coghlan's
|
||||
CPython sandbox on BitBucket [2_].
|
||||
|
||||
It implements the ``_Py_InitializeCore`` and ``_Py_InitializeMainInterpreter``
|
||||
aspects of the refactoring as a private API. All ``_PyCoreConfig`` settings
|
||||
It implements the ``_Py_InitializeRuntime`` and ``_Py_ConfigureMainInterpreter``
|
||||
aspects of the refactoring as a private API. All ``_PyRuntimeConfig`` settings
|
||||
are included, but currently only the "install signal handlers" setting is
|
||||
implemented for the main interpreter configuration.
|
||||
|
||||
The currently available interfaces in the reference implementation:
|
||||
The feature branch is a couple of iterations behind the API design in the PEP
|
||||
so the exact interfaces currently available are:
|
||||
|
||||
* ``_Py_IsCoreInitialized``
|
||||
* ``_Py_InitializeCore``
|
||||
* ``_PyCoreConfig``
|
||||
* ``_PyCoreConfig_INIT``
|
||||
* ``_Py_IsCoreInitialized`` (rename pending: ``_Py_IsRuntimeInitialized``)
|
||||
* ``_Py_InitializeCore`` (rename pending: ``_Py_InitializeRuntime``)
|
||||
* ``_PyCoreConfig`` (rename pending: ``_Py_RuntimeConfig``)
|
||||
* ``_PyCoreConfig_INIT`` (rename pending: ``_Py_RuntimeConfig_INIT``)
|
||||
* ``_Py_ReadHashSeed``
|
||||
* ``_Py_InitializeMainInterpreter``
|
||||
* ``_Py_InitializeMainInterpreter`` (rename pending: ``_Py_ConfigureMainInterpreter``)
|
||||
* ``_PyMainInterpreterConfig``
|
||||
* ``_PyMainInterpreterConfig_INIT``
|
||||
* ``_Py_ReadMainInterpreterConfig``
|
||||
|
|
@ -1062,7 +1065,7 @@ a fairly ad hoc fashion over the past 20+ years, leading to a rather
|
|||
inconsistent interface with varying levels of documentation.
|
||||
|
||||
(Note: some of the info below could probably be cleaned up and added to the
|
||||
C API documentation for at least 3.3. - it's all CPython specific, so it
|
||||
C API documentation for 3.6 - it's all CPython specific, so it
|
||||
doesn't belong in the language reference)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue