iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
python-peps/pep-0667/index.html

547 lines
47 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 667 – Consistent views of namespaces | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0667/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 667 – Consistent views of namespaces | peps.python.org'>
<meta property="og:description" content="In early versions of Python all namespaces, whether in functions, classes or modules, were all implemented the same way: as a dictionary.">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0667/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="In early versions of Python all namespaces, whether in functions, classes or modules, were all implemented the same way: as a dictionary.">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 667</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 667 – Consistent views of namespaces</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Mark Shannon &lt;mark&#32;&#97;t&#32;hotpy.org&gt;,
Tian Gao &lt;gaogaotiantian&#32;&#97;t&#32;hotmail.com&gt;</dd>
<dt class="field-even">Discussions-To<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/46631">Discourse thread</a></dd>
<dt class="field-odd">Status<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative proposal accepted for implementation">Accepted</abbr></dd>
<dt class="field-even">Type<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-odd">Created<span class="colon">:</span></dt>
<dd class="field-odd">30-Jul-2021</dd>
<dt class="field-even">Python-Version<span class="colon">:</span></dt>
<dd class="field-even">3.13</dd>
<dt class="field-odd">Post-History<span class="colon">:</span></dt>
<dd class="field-odd">20-Aug-2021</dd>
<dt class="field-even">Resolution<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/pep-667-consistent-views-of-namespaces/46631/25">Discourse message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#python">Python</a></li>
<li><a class="reference internal" href="#c-api">C-API</a><ul>
<li><a class="reference internal" href="#extensions-to-the-api">Extensions to the API</a></li>
<li><a class="reference internal" href="#changes-to-existing-apis">Changes to existing APIs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#behavior-of-f-locals-for-optimized-functions">Behavior of f_locals for optimized functions</a></li>
</ul>
</li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a><ul>
<li><a class="reference internal" href="#id1">Python</a></li>
<li><a class="reference internal" href="#id2">C-API</a><ul>
<li><a class="reference internal" href="#pyeval-getlocals">PyEval_GetLocals</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#implementation">Implementation</a><ul>
<li><a class="reference internal" href="#id3">C API</a></li>
</ul>
</li>
<li><a class="reference internal" href="#impact-on-pep-709-inlined-comprehensions">Impact on PEP 709 inlined comprehensions</a></li>
<li><a class="reference internal" href="#comparison-with-pep-558">Comparison with PEP 558</a></li>
<li><a class="reference internal" href="#id4">Implementation</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>In early versions of Python all namespaces, whether in functions,
classes or modules, were all implemented the same way: as a dictionary.</p>
<p>For performance reasons, the implementation of function namespaces was
changed. Unfortunately this meant that accessing these namespaces through
<code class="docutils literal notranslate"><span class="pre">locals()</span></code> and <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> ceased to be consistent and some
odd bugs crept in over the years as threads, generators and coroutines
were added.</p>
<p>This PEP proposes making these namespaces consistent once more.
Modifications to <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> will always be visible in
the underlying variables. Modifications to local variables will
immediately be visible in <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code>, and they will be
consistent regardless of threading or coroutines.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">locals()</span></code> function will act the same as it does now for class
and modules scopes. For function scopes it will return an instantaneous
snapshot of the underlying <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code>.</p>
</section>
<section id="motivation">
<h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2>
<p>The current implementation of <code class="docutils literal notranslate"><span class="pre">locals()</span></code> and <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> is slow,
inconsistent and buggy.
We want to make it faster, consistent, and most importantly fix the bugs.</p>
<p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">C</span><span class="p">:</span>
<span class="n">x</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">sys</span><span class="o">.</span><span class="n">_getframe</span><span class="p">()</span><span class="o">.</span><span class="n">f_locals</span><span class="p">[</span><span class="s1">&#39;x&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p>prints <code class="docutils literal notranslate"><span class="pre">2</span></code></p>
<p>but:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">f</span><span class="p">():</span>
<span class="n">x</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">sys</span><span class="o">.</span><span class="n">_getframe</span><span class="p">()</span><span class="o">.</span><span class="n">f_locals</span><span class="p">[</span><span class="s1">&#39;x&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">f</span><span class="p">()</span>
</pre></div>
</div>
<p>prints <code class="docutils literal notranslate"><span class="pre">1</span></code></p>
<p>This is inconsistent, and confusing.
With this PEP both examples would print <code class="docutils literal notranslate"><span class="pre">2</span></code>.</p>
<p>Worse than that, the current behavior can result in strange <a class="reference external" href="https://github.com/python/cpython/issues/74929">bugs</a>.</p>
<p>There are no compensating advantages for the current behavior;
it is unreliable and slow.</p>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<p>The current implementation of <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> returns a dictionary
that is created on the fly from the array of local variables.
This can result in the array and dictionary getting out of sync with
each other. Writes to the <code class="docutils literal notranslate"><span class="pre">f_locals</span></code> may not show up as
modifications to local variables. Writes to local variables can get lost.</p>
<p>By making <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> return a view on the
underlying frame, these problems go away. <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> is always in
sync with the frame because it is a view of it, not a copy of it.</p>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<section id="python">
<h3><a class="toc-backref" href="#python" role="doc-backlink">Python</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> will return a view object on the frame that
implements the <code class="docutils literal notranslate"><span class="pre">collections.abc.Mapping</span></code> interface.</p>
<p>For module and class scopes <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> will be a dictionary,
for function scopes it will be a custom class.</p>
<p><code class="docutils literal notranslate"><span class="pre">locals()</span></code> will be defined as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">locals</span><span class="p">():</span>
<span class="n">frame</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">_getframe</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">f_locals</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">f_locals</span>
<span class="k">if</span> <span class="n">frame</span><span class="o">.</span><span class="n">is_function</span><span class="p">():</span>
<span class="n">f_locals</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">f_locals</span><span class="p">)</span>
<span class="k">return</span> <span class="n">f_locals</span>
</pre></div>
</div>
<p>All writes to the <code class="docutils literal notranslate"><span class="pre">f_locals</span></code> mapping will be immediately visible
in the underlying variables. All changes to the underlying variables
will be immediately visible in the mapping. The <code class="docutils literal notranslate"><span class="pre">f_locals</span></code> object will
be a full mapping, and can have arbitrary key-value pairs added to it.</p>
<p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">l</span><span class="p">():</span>
<span class="s2">&quot;Get the locals of caller&quot;</span>
<span class="k">return</span> <span class="n">sys</span><span class="o">.</span><span class="n">_getframe</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">f_locals</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="k">if</span> <span class="mi">0</span><span class="p">:</span> <span class="n">y</span> <span class="o">=</span> <span class="mi">1</span> <span class="c1"># Make &#39;y&#39; a local variable</span>
<span class="n">x</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">l</span><span class="p">()[</span><span class="s1">&#39;x&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">l</span><span class="p">()[</span><span class="s1">&#39;y&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">l</span><span class="p">()[</span><span class="s1">&#39;z&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">y</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">locals</span><span class="p">(),</span> <span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">test()</span></code> will print <code class="docutils literal notranslate"><span class="pre">{'x':</span> <span class="pre">2,</span> <span class="pre">'y':</span> <span class="pre">4,</span> <span class="pre">'z':</span> <span class="pre">5}</span> <span class="pre">2</span></code>.</p>
<p>In Python 3.10, the above will fail with an <code class="docutils literal notranslate"><span class="pre">UnboundLocalError</span></code>,
as the definition of <code class="docutils literal notranslate"><span class="pre">y</span></code> by <code class="docutils literal notranslate"><span class="pre">l()['y']</span> <span class="pre">=</span> <span class="pre">4</span></code> is lost.</p>
<p>If the second-to-last line were changed from <code class="docutils literal notranslate"><span class="pre">y</span></code> to <code class="docutils literal notranslate"><span class="pre">z</span></code>, this would be a
<code class="docutils literal notranslate"><span class="pre">NameError</span></code>, as it is today. Keys added to <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> that are not
lexically local variables remain visible in <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code>, but do not
dynamically become local variables.</p>
</section>
<section id="c-api">
<h3><a class="toc-backref" href="#c-api" role="doc-backlink">C-API</a></h3>
<section id="extensions-to-the-api">
<h4><a class="toc-backref" href="#extensions-to-the-api" role="doc-backlink">Extensions to the API</a></h4>
<p>Three new C-API functions will be added:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span> <span class="o">*</span><span class="n">PyEval_GetFrameLocals</span><span class="p">(</span><span class="n">void</span><span class="p">)</span>
<span class="n">PyObject</span> <span class="o">*</span><span class="n">PyEval_GetFrameGlobals</span><span class="p">(</span><span class="n">void</span><span class="p">)</span>
<span class="n">PyObject</span> <span class="o">*</span><span class="n">PyEval_GetFrameBuiltins</span><span class="p">(</span><span class="n">void</span><span class="p">)</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">PyEval_GetFrameLocals()</span></code> is equivalent to: <code class="docutils literal notranslate"><span class="pre">locals()</span></code>.
<code class="docutils literal notranslate"><span class="pre">PyEval_GetFrameGlobals()</span></code> is equivalent to: <code class="docutils literal notranslate"><span class="pre">globals()</span></code>.</p>
<p>All these functions will return a new reference.</p>
</section>
<section id="changes-to-existing-apis">
<h4><a class="toc-backref" href="#changes-to-existing-apis" role="doc-backlink">Changes to existing APIs</a></h4>
<p><code class="docutils literal notranslate"><span class="pre">PyFrame_GetLocals(f)</span></code> is equivalent to <code class="docutils literal notranslate"><span class="pre">f.f_locals</span></code>, and hence its return value
will change as described above for accessing <code class="docutils literal notranslate"><span class="pre">f.f_locals</span></code>.</p>
<p>The following C-API functions will be deprecated, as they return borrowed references:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyEval_GetLocals</span><span class="p">()</span>
<span class="n">PyEval_GetGlobals</span><span class="p">()</span>
<span class="n">PyEval_GetBuiltins</span><span class="p">()</span>
</pre></div>
</div>
<p>The following functions should be used instead:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyEval_GetFrameLocals</span><span class="p">()</span>
<span class="n">PyEval_GetFrameGlobals</span><span class="p">()</span>
<span class="n">PyEval_GetFrameBuiltins</span><span class="p">()</span>
</pre></div>
</div>
<p>which return new references.</p>
<p>The semantics of <code class="docutils literal notranslate"><span class="pre">PyEval_GetLocals()</span></code> is changed as it now returns a
proxy for the frame locals in optimized frames, not a dictionary.</p>
<p>The following three functions will become no-ops, and will be deprecated:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyFrame_FastToLocalsWithError</span><span class="p">()</span>
<span class="n">PyFrame_FastToLocals</span><span class="p">()</span>
<span class="n">PyFrame_LocalsToFast</span><span class="p">()</span>
</pre></div>
</div>
</section>
</section>
<section id="behavior-of-f-locals-for-optimized-functions">
<h3><a class="toc-backref" href="#behavior-of-f-locals-for-optimized-functions" role="doc-backlink">Behavior of f_locals for optimized functions</a></h3>
<p>Although <code class="docutils literal notranslate"><span class="pre">f.f_locals</span></code> behaves as if it were the namespace of the function,
there will be some observable differences.
For example, <code class="docutils literal notranslate"><span class="pre">f.f_locals</span> <span class="pre">is</span> <span class="pre">f.f_locals</span></code> may be <code class="docutils literal notranslate"><span class="pre">False</span></code>.</p>
<p>However <code class="docutils literal notranslate"><span class="pre">f.f_locals</span> <span class="pre">==</span> <span class="pre">f.f_locals</span></code> will be <code class="docutils literal notranslate"><span class="pre">True</span></code>, and
all changes to the underlying variables, by any means, will always be visible.</p>
</section>
</section>
<section id="backwards-compatibility">
<h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2>
<section id="id1">
<h3><a class="toc-backref" href="#id1" role="doc-backlink">Python</a></h3>
<p>The current implementation has many corner cases and oddities.
Code that works around those may need to be changed.
Code that uses <code class="docutils literal notranslate"><span class="pre">locals()</span></code> for simple templating, or print debugging,
will continue to work correctly. Debuggers and other tools that use
<code class="docutils literal notranslate"><span class="pre">f_locals</span></code> to modify local variables, will now work correctly,
even in the presence of threaded code, coroutines and generators.</p>
</section>
<section id="id2">
<h3><a class="toc-backref" href="#id2" role="doc-backlink">C-API</a></h3>
<section id="pyeval-getlocals">
<h4><a class="toc-backref" href="#pyeval-getlocals" role="doc-backlink">PyEval_GetLocals</a></h4>
<p>Because <code class="docutils literal notranslate"><span class="pre">PyEval_GetLocals()</span></code> returns a borrowed reference, it requires
the proxy mapping to be cached on the frame, extending its lifetime and
creating a cycle. <code class="docutils literal notranslate"><span class="pre">PyEval_GetFrameLocals()</span></code> should be used instead.</p>
<p>This code:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">locals</span> <span class="o">=</span> <span class="n">PyEval_GetLocals</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">locals</span> <span class="o">==</span> <span class="n">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">goto</span> <span class="n">error_handler</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">Py_INCREF</span><span class="p">(</span><span class="nb">locals</span><span class="p">);</span>
</pre></div>
</div>
<p>should be replaced with:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">locals</span> <span class="o">=</span> <span class="n">PyEval_GetFrameLocals</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">locals</span> <span class="o">==</span> <span class="n">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">goto</span> <span class="n">error_handler</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
</section>
</section>
<section id="implementation">
<h2><a class="toc-backref" href="#implementation" role="doc-backlink">Implementation</a></h2>
<p>Each read of <code class="docutils literal notranslate"><span class="pre">frame.f_locals</span></code> will create a new proxy object that gives
the appearance of being the mapping of local (including cell and free)
variable names to the values of those local variables.</p>
<p>A possible implementation is sketched out below.
All attributes that start with an underscore are invisible and
cannot be accessed directly.
They serve only to illustrate the proposed design.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">NULL</span><span class="p">:</span> <span class="n">Object</span> <span class="c1"># NULL is a singleton representing the absence of a value.</span>
<span class="k">class</span> <span class="nc">CodeType</span><span class="p">:</span>
<span class="n">_name_to_offset_mapping_impl</span><span class="p">:</span> <span class="nb">dict</span> <span class="o">|</span> <span class="n">NULL</span>
<span class="n">_cells</span><span class="p">:</span> <span class="nb">frozenset</span> <span class="c1"># Set of indexes of cell and free variables</span>
<span class="o">...</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_name_to_offset_mapping_impl</span> <span class="o">=</span> <span class="n">NULL</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_variable_names</span> <span class="o">=</span> <span class="n">deduplicate</span><span class="p">(</span>
<span class="bp">self</span><span class="o">.</span><span class="n">co_varnames</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">co_cellvars</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">co_freevars</span>
<span class="p">)</span>
<span class="o">...</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">_name_to_offset_mapping</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Mapping of names to offsets in local variable array.&quot;</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_name_to_offset_mapping_impl</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_name_to_offset_mapping_impl</span> <span class="o">=</span> <span class="p">{</span>
<span class="n">name</span><span class="p">:</span> <span class="n">index</span> <span class="k">for</span> <span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_variable_names</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_name_to_offset_mapping_impl</span>
<span class="k">class</span> <span class="nc">FrameType</span><span class="p">:</span>
<span class="n">_locals</span> <span class="p">:</span> <span class="n">array</span><span class="p">[</span><span class="n">Object</span><span class="p">]</span> <span class="c1"># The values of the local variables, items may be NULL.</span>
<span class="n">_extra_locals</span><span class="p">:</span> <span class="nb">dict</span> <span class="o">|</span> <span class="n">NULL</span> <span class="c1"># Dictionary for storing extra locals not in _locals.</span>
<span class="n">_locals_cache</span><span class="p">:</span> <span class="n">FrameLocalsProxy</span> <span class="o">|</span> <span class="n">NULL</span> <span class="c1"># required to support PyEval_GetLocals()</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_extra_locals</span> <span class="o">=</span> <span class="n">NULL</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_locals_cache</span> <span class="o">=</span> <span class="n">NULL</span>
<span class="o">...</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">f_locals</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="n">FrameLocalsProxy</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">FrameLocalsProxy</span><span class="p">:</span>
<span class="s2">&quot;Implements collections.MutableMapping.&quot;</span>
<span class="vm">__slots__</span> <span class="o">=</span> <span class="p">(</span><span class="s2">&quot;_frame&quot;</span><span class="p">,</span> <span class="p">)</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">frame</span><span class="p">:</span><span class="n">FrameType</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_frame</span> <span class="o">=</span> <span class="n">frame</span>
<span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_frame</span>
<span class="n">co</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">f_code</span>
<span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_name_to_offset_mapping</span><span class="p">:</span>
<span class="n">index</span> <span class="o">=</span> <span class="n">co</span><span class="o">.</span><span class="n">_name_to_offset_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">_locals</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
<span class="k">if</span> <span class="n">val</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">if</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_cells</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">val</span><span class="o">.</span><span class="n">cell_contents</span>
<span class="k">if</span> <span class="n">val</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">return</span> <span class="n">val</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">return</span> <span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
<span class="k">def</span> <span class="fm">__setitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_frame</span>
<span class="n">co</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">f_code</span>
<span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_name_to_offset_mapping</span><span class="p">:</span>
<span class="n">index</span> <span class="o">=</span> <span class="n">co</span><span class="o">.</span><span class="n">_name_to_offset_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
<span class="n">kind</span> <span class="o">=</span> <span class="n">co</span><span class="o">.</span><span class="n">_local_kinds</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
<span class="k">if</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_cells</span>
<span class="n">cell</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">_locals</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
<span class="n">cell</span><span class="o">.</span><span class="n">cell_contents</span> <span class="o">=</span> <span class="n">val</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">_locals</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="n">val</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">val</span>
<span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_frame</span>
<span class="n">co</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">f_code</span>
<span class="k">yield from</span> <span class="nb">iter</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span><span class="p">)</span>
<span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">co</span><span class="o">.</span><span class="n">_variable_names</span><span class="p">):</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">_locals</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
<span class="k">if</span> <span class="n">val</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">continue</span>
<span class="k">if</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_cells</span><span class="p">:</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">val</span><span class="o">.</span><span class="n">cell_contents</span>
<span class="k">if</span> <span class="n">val</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">continue</span>
<span class="k">yield</span> <span class="n">name</span>
<span class="k">def</span> <span class="fm">__contains__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_frame</span>
<span class="k">if</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">f</span><span class="o">.</span><span class="n">_extra_locals</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">True</span>
<span class="k">return</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_variable_names</span>
<span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_frame</span>
<span class="n">co</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">f_code</span>
<span class="n">res</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">co</span><span class="o">.</span><span class="n">_variable_names</span><span class="p">):</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">_locals</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
<span class="k">if</span> <span class="n">val</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">continue</span>
<span class="k">if</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">co</span><span class="o">.</span><span class="n">_cells</span><span class="p">:</span>
<span class="k">if</span> <span class="n">val</span><span class="o">.</span><span class="n">cell_contents</span> <span class="ow">is</span> <span class="n">NULL</span><span class="p">:</span>
<span class="k">continue</span>
<span class="n">res</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_extra_locals</span><span class="p">)</span> <span class="o">+</span> <span class="n">res</span>
</pre></div>
</div>
<section id="id3">
<h3><a class="toc-backref" href="#id3" role="doc-backlink">C API</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">PyEval_GetLocals()</span></code> will be implemented roughly as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">PyObject</span> <span class="o">*</span><span class="n">PyEval_GetLocals</span><span class="p">(</span><span class="n">void</span><span class="p">)</span> <span class="p">{</span>
<span class="n">PyFrameObject</span> <span class="o">*</span> <span class="o">=</span> <span class="o">...</span><span class="p">;</span> <span class="o">//</span> <span class="n">Get</span> <span class="n">the</span> <span class="n">current</span> <span class="n">frame</span><span class="o">.</span>
<span class="k">if</span> <span class="p">(</span><span class="n">frame</span><span class="o">-&gt;</span><span class="n">_locals_cache</span> <span class="o">==</span> <span class="n">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">frame</span><span class="o">-&gt;</span><span class="n">_locals_cache</span> <span class="o">=</span> <span class="n">PyEval_GetFrameLocals</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">frame</span><span class="o">-&gt;</span><span class="n">_locals_cache</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>As with all functions that return a borrowed reference, care must be taken to
ensure that the reference is not used beyond the lifetime of the object.</p>
</section>
</section>
<section id="impact-on-pep-709-inlined-comprehensions">
<h2><a class="toc-backref" href="#impact-on-pep-709-inlined-comprehensions" role="doc-backlink">Impact on PEP 709 inlined comprehensions</a></h2>
<p>For inlined comprehensions within a function, <code class="docutils literal notranslate"><span class="pre">locals()</span></code> currently behaves the
same inside or outside of the comprehension, and this will not change. The
behavior of <code class="docutils literal notranslate"><span class="pre">locals()</span></code> inside functions will generally change as specified in
the rest of this PEP.</p>
<p>For inlined comprehensions at module or class scope, currently calling
<code class="docutils literal notranslate"><span class="pre">locals()</span></code> within the inlined comprehension returns a new dictionary for each
call. This PEP will make <code class="docutils literal notranslate"><span class="pre">locals()</span></code> within a function also always return a new
dictionary for each call, improving consistency; class or module scope inlined
comprehensions will appear to behave as if the inlined comprehension is still a
distinct function.</p>
</section>
<section id="comparison-with-pep-558">
<h2><a class="toc-backref" href="#comparison-with-pep-558" role="doc-backlink">Comparison with PEP 558</a></h2>
<p>This PEP and <a class="pep reference internal" href="../pep-0558/" title="PEP 558 – Defined semantics for locals()">PEP 558</a> share a common goal:
to make the semantics of <code class="docutils literal notranslate"><span class="pre">locals()</span></code> and <code class="docutils literal notranslate"><span class="pre">frame.f_locals()</span></code>
intelligible, and their operation reliable.</p>
<p>The key difference between this PEP and <a class="pep reference internal" href="../pep-0558/" title="PEP 558 – Defined semantics for locals()">PEP 558</a> is that
<a class="pep reference internal" href="../pep-0558/" title="PEP 558 – Defined semantics for locals()">PEP 558</a> keeps an internal copy of the local variables,
whereas this PEP does not.</p>
<p><a class="pep reference internal" href="../pep-0558/" title="PEP 558 – Defined semantics for locals()">PEP 558</a> does not specify exactly when the internal copy is
updated, making the behavior of <a class="pep reference internal" href="../pep-0558/" title="PEP 558 – Defined semantics for locals()">PEP 558</a> impossible to reason about.</p>
</section>
<section id="id4">
<h2><a class="toc-backref" href="#id4" role="doc-backlink">Implementation</a></h2>
<p>The implementation is in development as a <a class="reference external" href="https://github.com/python/cpython/pull/115153">draft pull request on GitHub</a>.</p>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0667.rst">https://github.com/python/peps/blob/main/peps/pep-0667.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0667.rst">2024-07-03 17:01:00 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#python">Python</a></li>
<li><a class="reference internal" href="#c-api">C-API</a><ul>
<li><a class="reference internal" href="#extensions-to-the-api">Extensions to the API</a></li>
<li><a class="reference internal" href="#changes-to-existing-apis">Changes to existing APIs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#behavior-of-f-locals-for-optimized-functions">Behavior of f_locals for optimized functions</a></li>
</ul>
</li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a><ul>
<li><a class="reference internal" href="#id1">Python</a></li>
<li><a class="reference internal" href="#id2">C-API</a><ul>
<li><a class="reference internal" href="#pyeval-getlocals">PyEval_GetLocals</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#implementation">Implementation</a><ul>
<li><a class="reference internal" href="#id3">C API</a></li>
</ul>
</li>
<li><a class="reference internal" href="#impact-on-pep-709-inlined-comprehensions">Impact on PEP 709 inlined comprehensions</a></li>
<li><a class="reference internal" href="#comparison-with-pep-558">Comparison with PEP 558</a></li>
<li><a class="reference internal" href="#id4">Implementation</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0667.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink