PEP 558: Remove dynamic frame semantics proposal (#1051)
Changing the frame API semantics based on whether or not a tracing function is active is tricky to implement and hard to document clearly, so this simplifies the proposal by instead having the frame API always expose a write-through proxy at function scope, and restricting the dynamic snapshot behaviour to the locals() builtin.
This commit is contained in:
Nick Coghlan
GitHub
parent
b8b55f631e
commit
44f2986f8d
118
pep-0558.rst
118
pep-0558.rst
|
|
@ -6,7 +6,7 @@ Status: Draft
|
|||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 2017-09-08
|
||||
Python-Version: 3.7
|
||||
Python-Version: 3.8
|
||||
Post-History: 2017-09-08
|
||||
|
||||
|
||||
|
|
@ -34,7 +34,7 @@ up to and including replication of local variable mutation bugs that
|
|||
can arise when a trace hook is installed [1]_.
|
||||
|
||||
While this PEP considers CPython's current behaviour when no trace hooks are
|
||||
installed to be acceptable (and even desirable), it considers the current
|
||||
installed to be acceptable (and largely desirable), it considers the current
|
||||
behaviour when trace hooks are installed to be problematic, as it causes bugs
|
||||
like [1]_ *without* even reliably enabling the desired functionality of allowing
|
||||
debuggers like ``pdb`` to mutate local variables [3]_.
|
||||
|
|
@ -71,6 +71,10 @@ scope that bring the ``locals()`` builtin semantics closer to those used in
|
|||
regular operation, while also making the related frame API semantics clearer
|
||||
and easier for interactive debuggers to rely on.
|
||||
|
||||
The proposed tracing mode changes also affect the semantics of frame object
|
||||
references obtained through other means, such as via a traceback, or via the
|
||||
``sys._getframe()`` API.
|
||||
|
||||
|
||||
New ``locals()`` documentation
|
||||
------------------------------
|
||||
|
|
@ -259,40 +263,31 @@ defined ``locals()`` builtin, trace functions necessarily use the implementation
|
|||
dependent ``frame.f_locals`` interface, as a frame reference is what gets
|
||||
passed to hook implementations.
|
||||
|
||||
In regular operation, nothing will change - ``frame.f_locals`` will be a direct
|
||||
reference to the dynamic snapshot, and ``locals()`` will return a reference to
|
||||
that snapshot. This reflects the fact that it's only CPython's tracing mode
|
||||
semantics that are currently problematic.
|
||||
|
||||
In tracing mode, however, we will change ``frame.f_locals`` to instead return
|
||||
a dedicated proxy type (implemented as a private subclass of the existing
|
||||
Instead of being a direct reference to the dynamic snapshot returned by
|
||||
``locals()``, ``frame.f_locals`` will be updated to instead return a dedicated
|
||||
proxy type (implemented as a private subclass of the existing
|
||||
``types.MappingProxyType``) that has two internal attributes not exposed as
|
||||
part of either the Python or public C API:
|
||||
|
||||
* *mapping*: the dynamic snapshot that would be returned by ``frame.f_locals``
|
||||
during regular operation
|
||||
* *mapping*: the dynamic snapshot that is returned by the ``locals()`` builtin
|
||||
* *frame*: the underlying frame that the snapshot is for
|
||||
|
||||
The ``locals()`` builtin would be aware of this proxy type, and continue to
|
||||
return a reference to the dynamic snapshot even when in tracing mode.
|
||||
``__setitem__`` and ``__delitem__`` operations on the proxy will affect not only
|
||||
the dynamic snapshot, but *also* the corresponding fast local or cell reference
|
||||
on the underlying frame.
|
||||
|
||||
As long as the process remains in tracing mode, then ``__setitem__`` and
|
||||
``__delitem__`` operations on the proxy will affect not only the dynamic
|
||||
snapshot, but *also* the corresponding fast local or cell reference on the
|
||||
underlying frame.
|
||||
|
||||
If the process leaves tracing mode (i.e. all previously installed trace hooks
|
||||
are uninstalled), then any already created proxy objects will remain in place,
|
||||
but their ``__setitem__`` and ``__delitem__`` methods will skip mutating
|
||||
the underlying frame.
|
||||
The ``locals()`` builtin will be made aware of this proxy type, and continue to
|
||||
return a reference to the dynamic snapshot rather than to the write-through
|
||||
proxy.
|
||||
|
||||
At the C API layer, ``PyEval_GetLocals()`` will implement the same semantics
|
||||
as the Python level ``locals()`` builtin, and a new ``PyFrame_GetLocals(frame)``
|
||||
accessor API will be provided to allow the proxy bypass logic to be encapsulated
|
||||
entirely inside the frame implementation. The C level equivalent of accessing
|
||||
``pyframe.f_locals`` in Python will be to access ``cframe->f_locals`` directly
|
||||
(the one difference is that the Python descriptor will continue to include an
|
||||
implicit snapshot refresh).
|
||||
(the one difference is that accessing ``pyframe.f_locals`` will continue to
|
||||
implicitly refresh the dynamic snapshot, whereas C code will need to explicitly
|
||||
call ``PyFrame_GetLocals(frame)`` to refresh the snapshot).
|
||||
|
||||
The ``PyFrame_LocalsToFast()`` function will be changed to always emit
|
||||
``RuntimeError``, explaining that it is no longer a supported operation, and
|
||||
|
|
@ -300,42 +295,6 @@ affected code should be updated to rely on the write-through tracing mode
|
|||
proxy instead.
|
||||
|
||||
|
||||
Open Questions
|
||||
==============
|
||||
|
||||
Is it necessary to restrict frame semantic changes to tracing mode?
|
||||
-------------------------------------------------------------------
|
||||
|
||||
It would be possible to say that ``frame.f_locals`` should *always* return a
|
||||
write-through proxy, even in regular operation.
|
||||
|
||||
This PEP currently avoids that option for a couple of key reasons, one pragmatic
|
||||
and one more philosophical:
|
||||
|
||||
* Object allocations and method wrappers aren't free, and tracing functions
|
||||
aren't the only operations that access frame locals from outside the function.
|
||||
Restricting the changes to tracing mode means that the additional memory and
|
||||
execution time overhead of these changes are going to be as close to zero in
|
||||
regular operation as we can possibly make them.
|
||||
* "Don't change what isn't broken": the current tracing mode problems are caused
|
||||
by a requirement that's specific to tracing mode (support for external
|
||||
rebinding of function local variable references), so it makes sense to also
|
||||
restrict any related fixes to tracing mode
|
||||
|
||||
The counter-argument to this is that it makes for a really subtle behavioural
|
||||
runtime state dependent distinction in how ``frame.f_locals`` works, and creates
|
||||
some new edge cases around how ``f_locals`` behaves as trace functions are added
|
||||
and removed. If the design adopted were instead "``frame.f_locals`` is always a
|
||||
write-through proxy, while ``locals()`` is always a dynamic snapshot", that
|
||||
would likely be both simpler to implement and easier to explain, suggesting
|
||||
that it may be a good idea.
|
||||
|
||||
Regardless of how the CPython reference implementation chooses to handle this,
|
||||
optimising compilers and interpreters can potentially impose additional
|
||||
restrictions on debuggers, by making local variable mutation through frame
|
||||
objects an opt-in behaviour that may disable some optimisations.
|
||||
|
||||
|
||||
Design Discussion
|
||||
=================
|
||||
|
||||
|
|
@ -377,6 +336,45 @@ the calling scope by default.
|
|||
There doesn't seem to be any reason for the PEP to change this.
|
||||
|
||||
|
||||
Changing the frame API semantics in regular operation
|
||||
-----------------------------------------------------
|
||||
|
||||
Earlier versions of this PEP proposed having the semantics of the frame
|
||||
``f_locals`` attribute depend on whether or not a tracing hook was currently
|
||||
installed - only providing the write-through proxy behaviour when a tracing hook
|
||||
was active, and otherwise behaving the same as the ``locals()`` builtin.
|
||||
|
||||
That was adopted as the original design proposal for a couple of key reasons,
|
||||
one pragmatic and one more philosophical:
|
||||
|
||||
* Object allocations and method wrappers aren't free, and tracing functions
|
||||
aren't the only operations that access frame locals from outside the function.
|
||||
Restricting the changes to tracing mode meant that the additional memory and
|
||||
execution time overhead of these changes would as close to zero in regular
|
||||
operation as we can possibly make them.
|
||||
* "Don't change what isn't broken": the current tracing mode problems are caused
|
||||
by a requirement that's specific to tracing mode (support for external
|
||||
rebinding of function local variable references), so it made sense to also
|
||||
restrict any related fixes to tracing mode
|
||||
|
||||
However, actually attempting to implement and document that dynamic approach
|
||||
highlighted the fact that it makes for a really subtle runtime state dependent
|
||||
behaviour distinction in how ``frame.f_locals`` works, and creates several
|
||||
new edge cases around how ``f_locals`` behaves as trace functions are added
|
||||
and removed.
|
||||
|
||||
Accordingly, the design was switched to the current one, where
|
||||
``frame.f_locals`` is always a write-through proxy, and ``locals()`` is always
|
||||
a dynamic snapshot, which is both simpler to implement and easier to explain.
|
||||
|
||||
Regardless of how the CPython reference implementation chooses to handle this,
|
||||
optimising compilers and interpreters also remain free to impose additional
|
||||
restrictions on debuggers, by making local variable mutation through frame
|
||||
objects an opt-in behaviour that may disable some optimisations (just as the
|
||||
emulation of CPython's frame API is already an opt-in flag in some Python
|
||||
implementations).
|
||||
|
||||
|
||||
Historical semantics at function scope
|
||||
--------------------------------------
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue