python-peps/pep-0562/index.html

318 lines
25 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 562 Module __getattr__ and __dir__ | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0562/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 562 Module __getattr__ and __dir__ | peps.python.org'>
<meta property="og:description" content="It is proposed to support __getattr__ and __dir__ function defined on modules to provide basic customization of module attribute access.">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0562/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="It is proposed to support __getattr__ and __dir__ function defined on modules to provide basic customization of module attribute access.">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 562</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 562 Module __getattr__ and __dir__</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Ivan Levkivskyi &lt;levkivskyi&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Created<span class="colon">:</span></dt>
<dd class="field-even">09-Sep-2017</dd>
<dt class="field-odd">Python-Version<span class="colon">:</span></dt>
<dd class="field-odd">3.7</dd>
<dt class="field-even">Post-History<span class="colon">:</span></dt>
<dd class="field-even">09-Sep-2017</dd>
<dt class="field-odd">Resolution<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2017-December/151033.html">Python-Dev message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a></li>
<li><a class="reference internal" href="#backwards-compatibility-and-impact-on-performance">Backwards compatibility and impact on performance</a></li>
<li><a class="reference internal" href="#discussion">Discussion</a></li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<div class="pep-banner canonical-doc sticky-banner admonition important">
<p class="admonition-title">Important</p>
<p>This PEP is a historical document. The up-to-date, canonical documentation can now be found at <a class="reference external" href="https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access">Customizing Module Attribute Access</a>.</p>
<p class="close-button">×</p>
<p>See <a class="pep reference internal" href="../pep-0001/" title="PEP 1 PEP Purpose and Guidelines">PEP 1</a> for how to propose changes.</p>
</div>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>It is proposed to support <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> and <code class="docutils literal notranslate"><span class="pre">__dir__</span></code> function defined
on modules to provide basic customization of module attribute access.</p>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<p>It is sometimes convenient to customize or otherwise have control over
access to module attributes. A typical example is managing deprecation
warnings. Typical workarounds are assigning <code class="docutils literal notranslate"><span class="pre">__class__</span></code> of a module object
to a custom subclass of <code class="docutils literal notranslate"><span class="pre">types.ModuleType</span></code> or replacing the <code class="docutils literal notranslate"><span class="pre">sys.modules</span></code>
item with a custom wrapper instance. It would be convenient to simplify this
procedure by recognizing <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> defined directly in a module that
would act like a normal <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> method, except that it will be defined
on module <em>instances</em>. For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># lib.py</span>
<span class="kn">from</span> <span class="nn">warnings</span> <span class="kn">import</span> <span class="n">warn</span>
<span class="n">deprecated_names</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;old_function&quot;</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">_deprecated_old_function</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">deprecated_names</span><span class="p">:</span>
<span class="n">warn</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> is deprecated&quot;</span><span class="p">,</span> <span class="ne">DeprecationWarning</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">globals</span><span class="p">()[</span><span class="sa">f</span><span class="s2">&quot;_deprecated_</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">]</span>
<span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;module </span><span class="si">{</span><span class="vm">__name__</span><span class="si">!r}</span><span class="s2"> has no attribute </span><span class="si">{</span><span class="n">name</span><span class="si">!r}</span><span class="s2">&quot;</span><span class="p">)</span>
<span class="c1"># main.py</span>
<span class="kn">from</span> <span class="nn">lib</span> <span class="kn">import</span> <span class="n">old_function</span> <span class="c1"># Works, but emits the warning</span>
</pre></div>
</div>
<p>Another widespread use case for <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> would be lazy submodule
imports. Consider a simple example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># lib/__init__.py</span>
<span class="kn">import</span> <span class="nn">importlib</span>
<span class="n">__all__</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;submod&#39;</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
<span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">__all__</span><span class="p">:</span>
<span class="k">return</span> <span class="n">importlib</span><span class="o">.</span><span class="n">import_module</span><span class="p">(</span><span class="s2">&quot;.&quot;</span> <span class="o">+</span> <span class="n">name</span><span class="p">,</span> <span class="vm">__name__</span><span class="p">)</span>
<span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;module </span><span class="si">{</span><span class="vm">__name__</span><span class="si">!r}</span><span class="s2"> has no attribute </span><span class="si">{</span><span class="n">name</span><span class="si">!r}</span><span class="s2">&quot;</span><span class="p">)</span>
<span class="c1"># lib/submod.py</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Submodule loaded&quot;</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">HeavyClass</span><span class="p">:</span>
<span class="o">...</span>
<span class="c1"># main.py</span>
<span class="kn">import</span> <span class="nn">lib</span>
<span class="n">lib</span><span class="o">.</span><span class="n">submod</span><span class="o">.</span><span class="n">HeavyClass</span> <span class="c1"># prints &quot;Submodule loaded&quot;</span>
</pre></div>
</div>
<p>There is a related proposal <a class="pep reference internal" href="../pep-0549/" title="PEP 549 Instance Descriptors">PEP 549</a> that proposes to support instance
properties for a similar functionality. The difference is this PEP proposes
a faster and simpler mechanism, but provides more basic customization.
An additional motivation for this proposal is that <a class="pep reference internal" href="../pep-0484/" title="PEP 484 Type Hints">PEP 484</a> already defines
the use of module <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> for this purpose in Python stub files,
see <a class="pep reference internal" href="../pep-0484/#stub-files" title="PEP 484 Type Hints § Stub Files">PEP 484</a>.</p>
<p>In addition, to allow modifying result of a <code class="docutils literal notranslate"><span class="pre">dir()</span></code> call on a module
to show deprecated and other dynamically generated attributes, it is
proposed to support module level <code class="docutils literal notranslate"><span class="pre">__dir__</span></code> function. For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># lib.py</span>
<span class="n">deprecated_names</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;old_function&quot;</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
<span class="n">__all__</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;new_function_one&quot;</span><span class="p">,</span> <span class="s2">&quot;new_function_two&quot;</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">new_function_one</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">new_function_two</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">def</span> <span class="fm">__dir__</span><span class="p">():</span>
<span class="k">return</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">__all__</span> <span class="o">+</span> <span class="n">deprecated_names</span><span class="p">)</span>
<span class="c1"># main.py</span>
<span class="kn">import</span> <span class="nn">lib</span>
<span class="nb">dir</span><span class="p">(</span><span class="n">lib</span><span class="p">)</span> <span class="c1"># prints [&quot;new_function_one&quot;, &quot;new_function_two&quot;, &quot;old_function&quot;, ...]</span>
</pre></div>
</div>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> function at the module level should accept one argument
which is the name of an attribute and return the computed value or raise
an <code class="docutils literal notranslate"><span class="pre">AttributeError</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Any</span><span class="p">:</span> <span class="o">...</span>
</pre></div>
</div>
<p>If an attribute is not found on a module object through the normal lookup
(i.e. <code class="docutils literal notranslate"><span class="pre">object.__getattribute__</span></code>), then <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> is searched in
the module <code class="docutils literal notranslate"><span class="pre">__dict__</span></code> before raising an <code class="docutils literal notranslate"><span class="pre">AttributeError</span></code>. If found, it is
called with the attribute name and the result is returned. Looking up a name
as a module global will bypass module <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>. This is intentional,
otherwise calling <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> for builtins will significantly harm
performance.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">__dir__</span></code> function should accept no arguments, and return
a list of strings that represents the names accessible on module:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="fm">__dir__</span><span class="p">()</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span> <span class="o">...</span>
</pre></div>
</div>
<p>If present, this function overrides the standard <code class="docutils literal notranslate"><span class="pre">dir()</span></code> search on
a module.</p>
<p>The reference implementation for this PEP can be found in <a class="footnote-reference brackets" href="#id3" id="id1">[2]</a>.</p>
</section>
<section id="backwards-compatibility-and-impact-on-performance">
<h2><a class="toc-backref" href="#backwards-compatibility-and-impact-on-performance" role="doc-backlink">Backwards compatibility and impact on performance</a></h2>
<p>This PEP may break code that uses module level (global) names <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>
and <code class="docutils literal notranslate"><span class="pre">__dir__</span></code>. (But the language reference explicitly reserves <em>all</em>
undocumented dunder names, and allows “breakage without warning”; see <a class="footnote-reference brackets" href="#id4" id="id2">[3]</a>.)
The performance implications of this PEP are minimal, since <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>
is called only for missing attributes.</p>
<p>Some tools that perform module attributes discovery might not expect
<code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>. This problem is not new however, since it is already possible
to replace a module with a module subclass with overridden <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> and
<code class="docutils literal notranslate"><span class="pre">__dir__</span></code>, but with this PEP such problems can occur more often.</p>
</section>
<section id="discussion">
<h2><a class="toc-backref" href="#discussion" role="doc-backlink">Discussion</a></h2>
<p>Note that the use of module <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> requires care to keep the referred
objects pickleable. For example, the <code class="docutils literal notranslate"><span class="pre">__name__</span></code> attribute of a function
should correspond to the name with which it is accessible via
<code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">keep_pickleable</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="n">func</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="vm">__name__</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;_deprecated_&#39;</span><span class="p">,</span> <span class="s1">&#39;&#39;</span><span class="p">)</span>
<span class="n">func</span><span class="o">.</span><span class="vm">__qualname__</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="vm">__qualname__</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;_deprecated_&#39;</span><span class="p">,</span> <span class="s1">&#39;&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">func</span>
<span class="nd">@keep_pickleable</span>
<span class="k">def</span> <span class="nf">_deprecated_old_function</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>One should be also careful to avoid recursion as one would do with
a class level <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code>.</p>
<p>To use a module global with triggering <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> (for example if one
wants to use a lazy loaded submodule) one can access it as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sys</span><span class="o">.</span><span class="n">modules</span><span class="p">[</span><span class="vm">__name__</span><span class="p">]</span><span class="o">.</span><span class="n">some_global</span>
</pre></div>
</div>
<p>or as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">.</span> <span class="kn">import</span> <span class="n">some_global</span>
</pre></div>
</div>
<p>Note that the latter sets the module attribute, thus <code class="docutils literal notranslate"><span class="pre">__getattr__</span></code> will be
called only once.</p>
</section>
<section id="references">
<h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="id3" role="doc-footnote">
<dt class="label" id="id3">[<a href="#id1">2</a>]</dt>
<dd>The reference implementation
(<a class="reference external" href="https://github.com/ilevkivskyi/cpython/pull/3/files">https://github.com/ilevkivskyi/cpython/pull/3/files</a>)</aside>
<aside class="footnote brackets" id="id4" role="doc-footnote">
<dt class="label" id="id4">[<a href="#id2">3</a>]</dt>
<dd>Reserved classes of identifiers
(<a class="reference external" href="https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers">https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers</a>)</aside>
</aside>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document has been placed in the public domain.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0562.rst">https://github.com/python/peps/blob/main/peps/pep-0562.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0562.rst">2024-10-31 21:24:44 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a></li>
<li><a class="reference internal" href="#backwards-compatibility-and-impact-on-performance">Backwards compatibility and impact on performance</a></li>
<li><a class="reference internal" href="#discussion">Discussion</a></li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0562.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>