PEP 734: Minor Updates (#3741)

* Expand the InterpreterPoolExecutor section.

* Identify the rest of the exception types.

* Add a note about "external" interpreters.

* Add Interpreter.close().

* Add a note about queued objects whose interpreter is destroyed.
This commit is contained in:
Eric Snow 2024-03-27 10:39:01 -06:00 committed by GitHub
parent e9a60482a1
commit 81d9734564
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 63 additions and 9 deletions

View File

@ -319,8 +319,16 @@ An ``interpreters.Interpreter`` object that represents the interpreter
There will only be one object for any given interpreter. There will only be one object for any given interpreter.
If the interpreter was created with ``interpreters.create()`` then If the interpreter was created with ``interpreters.create()`` then
it will be destroyed as soon as all ``Interpreter`` objects have been it will be destroyed as soon as all ``Interpreter`` objects with its ID
deleted. (across all interpreters) have been deleted.
``Interpreter`` objects may represent other interpreters than those
created by ``interpreters.create()``. Examples include the main
interpreter (created by Python's runtime initialization) and those
created via the C-API, using ``Py_NewInterpreter()``. Such
``Interpreter`` objects will not be able to interact with their
corresponding interpreters, e.g. via ``Interpreter.exec()``
(though we may relax this in the future).
Attributes and methods: Attributes and methods:
@ -421,6 +429,9 @@ Attributes and methods:
t = threading.Thread(target=task) t = threading.Thread(target=task)
t.start() t.start()
* ``close()``
Destroy the underlying interpreter.
Communicating Between Interpreters Communicating Between Interpreters
---------------------------------- ----------------------------------
@ -520,6 +531,13 @@ Attributes and methods:
If "syncobj" is None (the default) then the queue's default If "syncobj" is None (the default) then the queue's default
value is used. value is used.
If an object is still in the queue, and the interpreter which put
it in the queue (i.e. to which it belongs) is destroyed, then the
object is immediately removed from the queue. (We may later add
an option to replace the removed object in the queue with a
sentinel or to raise an exception for the corresponding ``get()``
call.)
* ``put_nowait(obj, *, syncobj=None)`` * ``put_nowait(obj, *, syncobj=None)``
Like ``put()`` but effectively with an immediate timeout. Like ``put()`` but effectively with an immediate timeout.
Thus if the queue is full, it immediately raises Thus if the queue is full, it immediately raises
@ -662,6 +680,17 @@ pass an object around to indicate who can use the resource::
Exceptions Exceptions
---------- ----------
* ``InterpreterError``
Indicates that some interpreter-related failure occurred.
This exception is a subclass of ``Exception``.
* ``InterpreterNotFoundError``
Raised from ``Interpreter`` methods after the underlying
interpreter has been destroyed, e.g. via the C-API.
This exception is a subclass of ``InterpreterError``.
* ``ExecutionFailed`` * ``ExecutionFailed``
Raised from ``Interpreter.exec()`` and ``Interpreter.call()`` Raised from ``Interpreter.exec()`` and ``Interpreter.call()``
when there's an uncaught exception. when there's an uncaught exception.
@ -677,28 +706,53 @@ Exceptions
* ``snapshot`` - a ``traceback.TracebackException`` object * ``snapshot`` - a ``traceback.TracebackException`` object
for the original exception for the original exception
This exception is a subclass of ``RuntimeError``. This exception is a subclass of ``InterpreterError``.
* ``QueueError``
Indicates that some queue-related failure occurred.
This exception is a subclass of ``Exception``.
* ``QueueNotFoundError``
Raised from ``interpreters.Queue`` methods after the underlying
queue has been destroyed.
This exception is a subclass of ``QueueError``.
* ``QueueEmpty`` * ``QueueEmpty``
Raised from ``Queue.get()`` (or ``get_nowait()`` with no default) Raised from ``Queue.get()`` (or ``get_nowait()`` with no default)
when the queue is empty. when the queue is empty.
This exception is a subclass of ``queue.Empty``. This exception is a subclass of both ``QueueError``
and the stdlib ``queue.Empty``.
* ``QueueFull`` * ``QueueFull``
Raised from ``Queue.put()`` (with a timeout) or ``put_nowait()`` Raised from ``Queue.put()`` (with a timeout) or ``put_nowait()``
when the queue is already at its max size. when the queue is already at its max size.
This exception is a subclass of ``queue.Full``. This exception is a subclass of both ``QueueError``
and the stdlib ``queue.Empty``.
InterpreterPoolExecutor InterpreterPoolExecutor
----------------------- -----------------------
Along with the new ``interpreters`` module, there will be a new Along with the new ``interpreters`` module, there will be a new
``concurrent.futures.InterpreterPoolExecutor``. Each worker executes ``concurrent.futures.InterpreterPoolExecutor``. It will be a
in its own thread with its own subinterpreter. Communication may derivative of ``ThreadPoolExecutor``, where each worker executes
still be done through ``interpreters.Queue`` objects, in its own thread, but each with its own subinterpreter.
set with the initializer.
Like the other executors, ``InterpreterPoolExecutor`` will support
callables for tasks, and for the initializer. Also like the other
executors, the arguments in both cases will be mostly unrestricted.
The callables and arguments will typically be serialized when sent
to a worker's interpreter, e.g. with pickle, like how the
``ProcessPoolExecutor`` works. This contrasts with
``Interpreter.call()``, which will (at least initially)
be much more restricted.
Communication between workers, or between the executor
(or generally its interpreter) and the workers, may still be done
through ``interpreters.Queue`` objects, set with the initializer.
sys.implementation.supports_isolated_interpreters sys.implementation.supports_isolated_interpreters
------------------------------------------------- -------------------------------------------------