From 30886daeb6995ffdc89596d83bd721918a868a4a Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Tue, 26 Mar 2019 12:39:43 -0600 Subject: [PATCH] PEP 554: Update alternate implementors and add rejected/deferred ideas. (#950) --- pep-0554.rst | 85 +++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 68 insertions(+), 17 deletions(-) diff --git a/pep-0554.rst b/pep-0554.rst index 0797255af..265a112e5 100644 --- a/pep-0554.rst +++ b/pep-0554.rst @@ -67,7 +67,7 @@ For creating and using interpreters: +----------------------------------+----------------------------------------------+ | signature | description | +==================================+==============================================+ -| ``list_all() -> [Intepreter]`` | Get all existing interpreters. | +| ``list_all() -> [Interpreter]`` | Get all existing interpreters. | +----------------------------------+----------------------------------------------+ | ``get_current() -> Interpreter`` | Get the currently running interpreter. | +----------------------------------+----------------------------------------------+ @@ -642,17 +642,15 @@ remove it. Alternate Python Implementations ================================ -I'll be soliciting feedback from the different Python implementors about -subinterpreter support. +I've solicited feedback from various Python implementors about support +for subinterpreters. Each has indicated that they would be able to +support subinterpreters (if they choose to) without a lot of +trouble. Here are the projects I contacted: -Multiple-interpter support in the major Python implementations: - -TBD - -* jython: yes [jython]_ -* ironpython: yes? -* pypy: maybe not? [pypy]_ -* micropython: ??? +* jython ([jython]_) +* ironpython (personal correspondence) +* pypy (personal correspondence) +* micropython (personal correspondence) .. _interpreters-list-all: @@ -1104,8 +1102,8 @@ This could be accomplished by adding a ``poison()`` method to both ends of the channel. The ``close()`` method can be used in this way (mostly), but these semantics are relatively specialized and can wait. -Reseting __main__ ------------------ +Resetting __main__ +------------------ As proposed, every call to ``Interpreter.run()`` will execute in the namespace of the interpreter's existing ``__main__`` module. This means @@ -1128,7 +1126,7 @@ Possible solutions include: * an ``Interpreter.reset_main()`` method to opt in when desired * ``importlib.util.reset_globals()`` [reset_globals]_ -Also note that reseting ``__main__`` does nothing about state stored +Also note that resetting ``__main__`` does nothing about state stored in other modules. So any solution would have to be clear about the scope of what is being reset. Conceivably we could invent a mechanism by which any (or every) module could be reset, unlike ``reload()`` @@ -1141,6 +1139,27 @@ generic module reset mechanism may prove unnecessary. This isn't a critical feature initially. It can wait until later if desirable. +Resetting an interpreter's state +-------------------------------- + +It may be nice to re-use an existing subinterpreter instead of +spinning up a new one. Since an interpreter has substantially more +state than just the ``__main__`` module, it isn't so easy to put an +interpreter back into a pristine/fresh state. In fact, there *may* +be parts of the state that cannot be reset from Python code. + +A possible solution is to add an ``Interpreter.reset()`` method. This +would put the interpreter back into the state it was in when newly +created. If called on a running interpreter it would fail (hence the +main interpreter could never be reset). This would likely be more +efficient than creating a new subinterpreter, though that depends on +what optimizations will be made later to subinterpreter creation. + +While this would potentially provide functionality that is not +otherwise available from Python code, it isn't a fundamental +functionality. So in the spirit of minimalism here, this can wait. +Regardless, I doubt it would be controversial to add it post-PEP. + File descriptors and sockets in channels ---------------------------------------- @@ -1322,6 +1341,41 @@ Rejected possible solutions: (they can pass error info out via channels); with threads you have to do something similar +Always associate each new interpreter with its own thread +--------------------------------------------------------- + +As implemented in the C-API, a subinterpreter is not inherently tied to +any thread. Furthermore, it will run in any existing thread, whether +created by Python or not. You only have to activate one of its thread +states (``PyThreadState``) in the thread first. This means that the +same thread may run more than one interpreter (though obviously +not at the same time). + +The proposed module maintains this behavior. Subinterpreters are not +tied to threads. Only calls to ``Interpreter.run()`` are. However, +one of the key objectives of this PEP is to provide a more human- +centric concurrency model. With that in mind, from a conceptual +standpoint the module *might* be easier to understand if each +subinterpreter were associated with its own thread. + +That would mean ``interpreters.create()`` would create a new thread +and ``Interpreter.run()`` would only execute in that thread (and +nothing else would). The benefit is that users would not have to +wrap ``Interpreter.run()`` calls in a new ``threading.Thread``. Nor +would they be in a position to accidentally pause the current +interpreter (in the current thread) while their subinterpreter +executes. + +The idea is rejected because the benefit is small and the cost is high. +The difference from the capability in the C-API would be potentially +confusing. The implcit creation of threads is magical. The early +creation of threads is potentially wasteful. The inability to run +arbitrary interpreters in an existing thread would prevent some valid +use cases, frustrating users. Tying interpreters to threads would +require extra runtime modifications. It would also make the module's +implementation overly complicated. Finally, it might not even make +the module easier to understand. + Implementation ============== @@ -1422,9 +1476,6 @@ References .. [jython] https://mail.python.org/pipermail/python-ideas/2017-May/045771.html -.. [pypy] - https://mail.python.org/pipermail/python-ideas/2017-September/046973.html - .. [multi-core-project] https://github.com/ericsnowcurrently/multi-core-python