PEP 558: adjustments after updating implementation (#2030)

* `pdb` stores `__return__` and `__exception__`` entries on
  arbitrary frames, so that feature needs to be preserved
* Some fast locals proxy operations will implicitly update the
  underlying shared mapping on the frame
* Anyone explicitly calling `LocalsToFast` is going to want the
  read/write proxy, not any of the read-only options (plus it's
  hard to fit an essay in an error message)
* Remove lingering reference to the removed `PyLocals_RefreshViews()`
This commit is contained in:
Nick Coghlan 2021-07-11 00:05:15 +10:00 committed by GitHub
parent 7b707b51a8
commit f6c6e24976
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 35 additions and 22 deletions

View File

@ -363,7 +363,10 @@ API:
``__getitem__`` operations on the proxy will populate the ``fast_refs`` mapping
(if it is not already populated), and then either return the relevant value
(if the key is found in the ``fast_refs`` mapping), or else raise ``KeyError``.
(if the key is found in either the ``fast_refs`` mapping or the ``f_locals``
dynamic snapshot stored on the frame), or else raise ``KeyError``. Variables
that are defined, but not yet bound raise ``KeyError`` (just as they're
omitted from the result of ``locals()``).
As the frame storage is always accessed directly, the proxy will automatically
pick up name binding operations that take place as the function executes.
@ -373,9 +376,11 @@ directly affect the corresponding fast local or cell reference on the underlying
frame, ensuring that changes are immediately visible to the running Python code,
rather than needing to be written back to the runtime storage at some later time.
Unlike the existing ``f_locals`` implementation on optimised frames, the frame
locals proxy will raise ``KeyError`` for attempts to write to keys that aren't
defined as local or closure variables on the underyling frame.
Keys that are not defined as local or closure variables on the underlying frame
will instead be written to the ``f_locals`` shared dynamic snapshot on optimised
frames. This allows utilities like ``pdb`` (which writes ``__return__`` and
``__exception__`` values into the frame ``f_locals`` mapping) to continue
working as they always have.
Other ``Mapping`` and ``MutableMapping`` methods will behave as expected for a
mapping with these essential method semantics.
@ -485,6 +490,12 @@ will be updated only in the following circumstance:
* any call to ``PyFrame_GetLocals()``, ``PyFrame_GetLocalsCopy()``,
``_PyFrame_BorrowLocals()``, ``PyFrame_FastToLocals()``, or
``PyFrame_FastToLocalsWithError()`` for the frame
* any operation on a fast locals proxy object that requires the shared
mapping to be up to date on the underlying frame. In the initial reference
implementation, those operations are any that require a full set of mapping
keys and/or values, including ``len(flp)``, ``flp.keys()``, ``flp.values()``,
``flp.items()``, ``flp.copy()``, iteration, containment checks, object
comparison, and rendering as a string.
Accessing the frame "view" APIs will *not* implicitly update the shared dynamic
snapshot, and the CPython trace hook handling will no longer implicitly update
@ -522,9 +533,9 @@ to avoid having to access the internals of the frame struct from the
The ``PyFrame_LocalsToFast()`` function will be changed to always emit
``RuntimeError``, explaining that it is no longer a supported operation, and
affected code should be updated to use ``PyFrame_GetLocals(frame)``,
``PyFrame_GetLocalsCopy(frame)``, ``PyFrame_GetLocalsView(frame)``, or
``PyObject_GetAttrString(frame, "f_locals")`` instead.
affected code should be updated to use
``PyObject_GetAttrString(frame, "f_locals")`` to obtain a read/write proxy
instead.
In addition to the above documented interfaces, the draft reference
implementation also exposes the following undocumented interfaces::
@ -551,9 +562,9 @@ this PEP incorporate's Victor Stinner's proposal to no longer implicitly call
``PyFrame_FastToLocalsWithError()`` before calling trace hooks implemented in
Python.
Code using the new frame view APIs won't need the dynamic locals snapshot
refreshed, while code using the ``PyEval_GetLocals()`` API will implicitly
refresh it when making that call.
Code using the new frame view APIs will have the dynamic locals snapshot
implicitly refreshed when accessing methods that need it, while code using the
``PyEval_GetLocals()`` API will implicitly refresh it when making that call.
The PEP necessarily also drops the implicit call to ``PyFrame_LocalsToFast()``
when returning from a trace hook, as that API now always raises an exception.
@ -721,22 +732,24 @@ emulation of CPython's frame API is already an opt-in flag in some Python
implementations).
Dropping support for storing additional data on optimised frames
----------------------------------------------------------------
Continuing to support storing additional data on optimised frames
-----------------------------------------------------------------
Earlier iterations of this PEP proposed preserving the ability to store
One of the draft iterations of this PEP proposed removing the ability to store
additional data on optimised frames by writing to ``frame.f_locals`` keys that
didn't correspond to local or closure variable names on the underlying frame.
While that property has been retained for the historical ``PyEval_GetLocals()``
C API, it has been dropped from the new fast locals proxy proposal in order to
simplify the semantics and implementation.
While this idea offered some attractive simplification of the fast locals proxy
implementation, ``pdb`` stores ``__return__`` and ``__exception__`` values on
arbitrary frames, so the standard library test suite fails if that functionality
no longer works.
Note: if this change proves problematic in practice, it would be reasonably
straightforward to amend the implementation to store unknown keys in the C level
``f_locals`` mapping, the same way ``PyEval_GetLocals()`` allows. However,
starting the new API with it disallowed offers the best chance of potentially
being able to deprecate and remove the behaviour entirely in the future.
Accordingly, the ability to store arbitrary keys was retained, at the expense
of certain operations on proxy objects currently being slower than desired (as
they need to update the dynamic snapshot in order to provide a reliable answer).
Future implementation improvements should allow that lost performance to be
recovered by only refreshing the snapshot when it is known to be out of date.
Historical semantics at function scope
@ -804,7 +817,7 @@ into the following cases:
be visible to Python code. This is the ``PyLocals_GetCopy()`` API.
* always wanting a read-only view of the current locals namespace, without
incurring the runtime overhead of making a full copy each time. This is the
``PyLocals_GetView()`` and ``PyLocals_RefreshViews()`` APIs.
``PyLocals_GetView()`` API.
Historically, these kinds of checks and operations would only have been
possible if a Python implementation emulated the full CPython frame API. With