Revise implementation strategy for PEP 432

pep-0432.txt

@@ -7,7 +7,7 @@ Status: Draft
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 28-Dec-2012
-Python-Version: 3.5
+Python-Version: 3.6
 Post-History: 28-Dec-2012, 2-Jan-2013
 
 
@@ -100,7 +100,7 @@ feature requests easier to implement.
 Key Concerns
 ============
 
-There are a couple of key concerns that any change to the startup sequence
+There are a few key concerns that any change to the startup sequence
 needs to take into account.
 
 
@@ -123,6 +123,17 @@ data types, it should also be easier to modify the system in the
 future to add new configuration options.
 
 
+Testability
+-----------
+
+One of the problems with the complexity of the CPython startup sequence is the
+combinatorial explosion of possible interactions between different configuration
+settings.
+
+This concern impacts both the design of the new initialisation system, and
+the proposed approach for getting there.
+
+
 Performance
 -----------
 
@@ -226,6 +237,54 @@ additional settings arise naturally in the course of migrating existing
 settings to the new structure).
 
 
+Implementation Strategy
+=======================
+
+An initial attempt was made at implementing an earlier version of this PEP for
+Python 3.4 [2_], with one of the significant problems encountered being merge
+conflicts after the initial structural changes were put in place to start the
+refactoring process. Unlike some other previous major changes, such as the
+switch to an AST-based compiler in Python 2.5, or the switch to the importlib
+implementation of the import system in Python 3.3, there is no clear way to
+structure a draft implementation that won't be prone to the kinds of merge
+conflicts that afflicted the original attempt.
+
+Accordingly, the implementation strategy now being proposed is to implement this
+refactoring as a private API for CPython 3.6, before exposing the new functions
+and structures as public API elements in CPython 3.7.
+
+The affected APIs with a leading underscore, as they would be named in CPython
+3.6:
+
+* ``_Py_IsCoreInitialized``
+* ``_Py_InitializeCore``
+* ``_PyCoreConfig``
+* ``_PyCoreConfig_INIT``
+* ``_Py_ReadHashSeed``
+* ``_Py_InitializeMainInterpreter``
+* ``_PyMainInterpreterConfig``
+* ``_PyMainInterpreterConfig_INIT``
+* ``_Py_ReadMainInterpreterConfig``
+* ``_PyRun_PrepareMain``
+* ``_PyRun_ExecMain``
+* ``_Py_InterpreterState_Main``
+
+New APIs described in the rest of the PEP with a leading underscore are
+intended to be retained permanently as private CPython implementation details.
+
+The principle benefit of this approach is allowing the refactoring to adopt the
+new configuration structures to be handled on a setting by setting basis
+over the course of the 3.6 and 3.7 development cycles, rather than having to
+migrate them in a single monolithic change. It also means any new settings can
+be handled using the new approach, even if some existing settings have yet to
+be migrated.
+
+If all existing settings are successfully migrated to the new initialization
+model in time for the 3.6.0a4 release in August 2016, then the proposal would
+be to make the APIs public for the 3.6.0b1 release in September 2016, rather
+than waiting for 3.7.
+
+
 Design Details
 ==============
 
@@ -255,7 +314,7 @@ Three distinct interpreter initialisation phases are proposed:
   * ``Py_IsInitialized()`` returns ``0``
   * The embedding application determines the settings required to create the
     main interpreter and moves to the next phase by calling
-    ``Py_InitializationCore``.
+    ``Py_InitializeCore``.
 
 * Core Initialized:
 
@@ -329,8 +388,7 @@ The proposed API for this step in the startup sequence is::
 
     void Py_InitializeCore(const PyCoreConfig *config);
 
-Like ``Py_Initialize``, this part of the new API treats initialization
+Like ``Py_Initialize``, this part of the new API treats initialization failures
-failures
 as fatal errors. While that's still not particularly embedding friendly,
 the operations in this step *really* shouldn't be failing, and changing them
 to return error codes instead of aborting would be an even larger task than
@@ -446,7 +504,7 @@ except that:
 * Only builtin and frozen modules may be imported (due to above limitations)
 * ``sys.stderr`` is set to a temporary IO object using unbuffered binary
   mode
-* The ``sys.flags`` attribute exists, but may contain flags may not yet
+* The ``sys.flags`` attribute exists, but the individual flags may not yet
   have their final values.
 * The ``sys.flags.initialized`` attribute is set to ``0``
 * The ``warnings`` module is not yet initialized
@@ -466,7 +524,7 @@ object rather than in C process globals.
 Any call to ``Py_InitializeCore()`` must have a matching call to
 ``Py_Finalize()``. It is acceptable to skip calling
 ``Py_InitializeMainInterpreter()`` in between (e.g. if attempting to read the
-main interpreter configuration settings fails)
+main interpreter configuration settings fails).
 
 
 Determining the remaining configuration settings
@@ -583,9 +641,9 @@ Python data types (not set == ``NULL``) or numeric flags (not set == ``-1``)::
         /* Main module
          *
          * If prepare_main is set, at most one of the main_* settings should
-         * be set before calling PyRun_PrepareMain (Py_ReadConfiguration will
+         * be set before calling PyRun_PrepareMain (Py_ReadMainInterpreterConfig
-         * set one of them based on the command line arguments if prepare_main
+         * will set one of them based on the command line arguments if
-         * is non-zero when that API is called).
+         * prepare_main is non-zero when that API is called).
         int prepare_main;
         PyUnicodeObject *main_source; /* -c switch */
         PyUnicodeObject *main_path;   /* filesystem path */
@@ -923,7 +981,7 @@ executed file is placed at the beginning of the import path.
 Issue 16499 [6_] added a ``-I`` option to change the behaviour of
 the normal CPython executable, but this is a hard to discover solution (and
 adds yet another option to an already complex CLI). This PEP proposes to
-instead add a separate ``pysystem`` executable
+instead add a separate ``system-python`` executable
 
 Currently, providing a separate executable with different default behaviour
 would be prohibitively hard to maintain. One of the goals of this PEP is to
@@ -965,21 +1023,19 @@ Open Questions
   both PyMainInterpreterConfig and PyMainInterpreterConfig_INIT when adding
   new fields?
 * The name of the new system Python executable is a bikeshed waiting to be
-  painted. The 3 options considered so far are ``spython``, ``pysystem``
+  painted. The 4 options considered so far are ``spython``, ``pysystem``,
-  and ``python-minimal``. The PEP text reflects my current preferred choice
+  ``python-minimal`` and `system-python``. The PEP text reflects my current
-  (``pysystem``).
+  preferred choice (``system-python``).
 
 
 Implementation
 ==============
 
-The reference implementation is being developed as a feature branch in my
+A reference implementation for an earlier design was developed as a feature
-BitBucket sandbox [2_]. Pull requests to fix the inevitably broken
+branch in my BitBucket sandbox [2_].
-Windows builds are welcome, but the basic design is still in too much flux
-for other pull requests to be feasible just yet. Once the overall design
-settles down and it's a matter of migrating individual settings over to
-the new design, that level of collaboration should become more practical.
 
+There is not yet a reference implementation for the design currently
+described in the PEP.
 
 The Status Quo
 ==============