python-peps/pep-0487/index.html

579 lines
53 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 487 Simpler customisation of class creation | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0487/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 487 Simpler customisation of class creation | peps.python.org'>
<meta property="og:description" content="Currently, customising class creation requires the use of a custom metaclass. This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts.">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0487/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="Currently, customising class creation requires the use of a custom metaclass. This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts.">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 487</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 487 Simpler customisation of class creation</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Martin Teichmann &lt;lkb.teichmann&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Created<span class="colon">:</span></dt>
<dd class="field-even">27-Feb-2015</dd>
<dt class="field-odd">Python-Version<span class="colon">:</span></dt>
<dd class="field-odd">3.6</dd>
<dt class="field-even">Post-History<span class="colon">:</span></dt>
<dd class="field-even">27-Feb-2015, 05-Feb-2016, 24-Jun-2016, 02-Jul-2016, 13-Jul-2016</dd>
<dt class="field-odd">Replaces<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="../pep-0422/">422</a></dd>
<dt class="field-even">Resolution<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2016-July/145629.html">Python-Dev message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#background">Background</a></li>
<li><a class="reference internal" href="#proposal">Proposal</a></li>
<li><a class="reference internal" href="#key-benefits">Key Benefits</a><ul>
<li><a class="reference internal" href="#easier-inheritance-of-definition-time-behaviour">Easier inheritance of definition time behaviour</a></li>
<li><a class="reference internal" href="#reduced-chance-of-metaclass-conflicts">Reduced chance of metaclass conflicts</a></li>
</ul>
</li>
<li><a class="reference internal" href="#new-ways-of-using-classes">New Ways of Using Classes</a><ul>
<li><a class="reference internal" href="#subclass-registration">Subclass registration</a></li>
<li><a class="reference internal" href="#trait-descriptors">Trait descriptors</a></li>
</ul>
</li>
<li><a class="reference internal" href="#implementation-details">Implementation Details</a></li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#backward-compatibility-issues">Backward compatibility issues</a></li>
<li><a class="reference internal" href="#rejected-design-options">Rejected Design Options</a><ul>
<li><a class="reference internal" href="#calling-the-hook-on-the-class-itself">Calling the hook on the class itself</a></li>
<li><a class="reference internal" href="#other-variants-of-calling-the-hooks">Other variants of calling the hooks</a></li>
<li><a class="reference internal" href="#requiring-an-explicit-decorator-on-init-subclass">Requiring an explicit decorator on <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code></a></li>
<li><a class="reference internal" href="#a-more-new-like-hook">A more <code class="docutils literal notranslate"><span class="pre">__new__</span></code>-like hook</a></li>
<li><a class="reference internal" href="#adding-a-class-attribute-with-the-attribute-order">Adding a class attribute with the attribute order</a></li>
</ul>
</li>
<li><a class="reference internal" href="#history">History</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>Currently, customising class creation requires the use of a custom metaclass.
This custom metaclass then persists for the entire lifecycle of the class,
creating the potential for spurious metaclass conflicts.</p>
<p>This PEP proposes to instead support a wide range of customisation
scenarios through a new <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> hook in the class body,
and a hook to initialize attributes.</p>
<p>The new mechanism should be easier to understand and use than
implementing a custom metaclass, and thus should provide a gentler
introduction to the full power of Pythons metaclass machinery.</p>
</section>
<section id="background">
<h2><a class="toc-backref" href="#background" role="doc-backlink">Background</a></h2>
<p>Metaclasses are a powerful tool to customize class creation. They have,
however, the problem that there is no automatic way to combine metaclasses.
If one wants to use two metaclasses for a class, a new metaclass combining
those two needs to be created, typically manually.</p>
<p>This need often occurs as a surprise to a user: inheriting from two base
classes coming from two different libraries suddenly raises the necessity
to manually create a combined metaclass, where typically one is not
interested in those details about the libraries at all. This becomes
even worse if one library starts to make use of a metaclass which it
has not done before. While the library itself continues to work perfectly,
suddenly every code combining those classes with classes from another library
fails.</p>
</section>
<section id="proposal">
<h2><a class="toc-backref" href="#proposal" role="doc-backlink">Proposal</a></h2>
<p>While there are many possible ways to use a metaclass, the vast majority
of use cases falls into just three categories: some initialization code
running after class creation, the initialization of descriptors and
keeping the order in which class attributes were defined.</p>
<p>The first two categories can easily be achieved by having simple hooks
into the class creation:</p>
<ol class="arabic simple">
<li>An <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> hook that initializes
all subclasses of a given class.</li>
<li>upon class creation, a <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> hook is called on all the
attribute (descriptors) defined in the class, and</li>
</ol>
<p>The third category is the topic of another PEP, <a class="pep reference internal" href="../pep-0520/" title="PEP 520 Preserving Class Attribute Definition Order">PEP 520</a>.</p>
<p>As an example, the first use case looks as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">QuestBase</span><span class="p">:</span>
<span class="gp">... </span> <span class="c1"># this is implicitly a @classmethod (see below for motivation)</span>
<span class="gp">... </span> <span class="k">def</span> <span class="nf">__init_subclass__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">swallow</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="gp">... </span> <span class="bp">cls</span><span class="o">.</span><span class="n">swallow</span> <span class="o">=</span> <span class="n">swallow</span>
<span class="gp">... </span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init_subclass__</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">Quest</span><span class="p">(</span><span class="n">QuestBase</span><span class="p">,</span> <span class="n">swallow</span><span class="o">=</span><span class="s2">&quot;african&quot;</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">pass</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">Quest</span><span class="o">.</span><span class="n">swallow</span>
<span class="go">&#39;african&#39;</span>
</pre></div>
</div>
<p>The base class <code class="docutils literal notranslate"><span class="pre">object</span></code> contains an empty <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>
method which serves as an endpoint for cooperative multiple inheritance.
Note that this method has no keyword arguments, meaning that all
methods which are more specialized have to process all keyword
arguments.</p>
<p>This general proposal is not a new idea (it was first suggested for
inclusion in the language definition <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2001-November/018651.html">more than 10 years ago</a>, and a
similar mechanism has long been supported by <a class="reference external" href="http://docs.zope.org/zope_secrets/extensionclass.html">Zopes ExtensionClass</a>),
but the situation has changed sufficiently in recent years that
the idea is worth reconsidering for inclusion.</p>
<p>The second part of the proposal adds an <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code>
initializer for class attributes, especially if they are descriptors.
Descriptors are defined in the body of a
class, but they do not know anything about that class, they do not
even know the name they are accessed with. They do get to know their
owner once <code class="docutils literal notranslate"><span class="pre">__get__</span></code> is called, but still they do not know their
name. This is unfortunate, for example they cannot put their
associated value into their objects <code class="docutils literal notranslate"><span class="pre">__dict__</span></code> under their name,
since they do not know that name. This problem has been solved many
times, and is one of the most important reasons to have a metaclass in
a library. While it would be easy to implement such a mechanism using
the first part of the proposal, it makes sense to have one solution
for this problem for everyone.</p>
<p>To give an example of its usage, imagine a descriptor representing weak
referenced values:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">weakref</span>
<span class="k">class</span> <span class="nc">WeakAttribute</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__get__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">instance</span><span class="p">,</span> <span class="n">owner</span><span class="p">):</span>
<span class="k">return</span> <span class="n">instance</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">]()</span>
<span class="k">def</span> <span class="fm">__set__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">instance</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="n">instance</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">weakref</span><span class="o">.</span><span class="n">ref</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="c1"># this is the new initializer:</span>
<span class="k">def</span> <span class="nf">__set_name__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">owner</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
</pre></div>
</div>
<p>Such a <code class="docutils literal notranslate"><span class="pre">WeakAttribute</span></code> may, for example, be used in a tree structure
where one wants to avoid cyclic references via the parent:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">TreeNode</span><span class="p">:</span>
<span class="n">parent</span> <span class="o">=</span> <span class="n">WeakAttribute</span><span class="p">()</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">parent</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">parent</span> <span class="o">=</span> <span class="n">parent</span>
</pre></div>
</div>
<p>Note that the <code class="docutils literal notranslate"><span class="pre">parent</span></code> attribute is used like a normal attribute,
yet the tree contains no cyclic references and can thus be easily
garbage collected when out of use. The <code class="docutils literal notranslate"><span class="pre">parent</span></code> attribute magically
becomes <code class="docutils literal notranslate"><span class="pre">None</span></code> once the parent ceases existing.</p>
<p>While this example looks very trivial, it should be noted that until
now such an attribute cannot be defined without the use of a metaclass.
And given that such a metaclass can make life very hard, this kind of
attribute does not exist yet.</p>
<p>Initializing descriptors could simply be done in the
<code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> hook. But this would mean that descriptors can
only be used in classes that have the proper hook, the generic version
like in the example would not work generally. One could also call
<code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> from within the base implementation of
<code class="docutils literal notranslate"><span class="pre">object.__init_subclass__</span></code>. But given that it is a common mistake
to forget to call <code class="docutils literal notranslate"><span class="pre">super()</span></code>, it would happen too often that suddenly
descriptors are not initialized.</p>
</section>
<section id="key-benefits">
<h2><a class="toc-backref" href="#key-benefits" role="doc-backlink">Key Benefits</a></h2>
<section id="easier-inheritance-of-definition-time-behaviour">
<h3><a class="toc-backref" href="#easier-inheritance-of-definition-time-behaviour" role="doc-backlink">Easier inheritance of definition time behaviour</a></h3>
<p>Understanding Pythons metaclasses requires a deep understanding of
the type system and the class construction process. This is legitimately
seen as challenging, due to the need to keep multiple moving parts (the code,
the metaclass hint, the actual metaclass, the class object, instances of the
class object) clearly distinct in your mind. Even when you know the rules,
its still easy to make a mistake if youre not being extremely careful.</p>
<p>Understanding the proposed implicit class initialization hook only requires
ordinary method inheritance, which isnt quite as daunting a task. The new
hook provides a more gradual path towards understanding all of the phases
involved in the class definition process.</p>
</section>
<section id="reduced-chance-of-metaclass-conflicts">
<h3><a class="toc-backref" href="#reduced-chance-of-metaclass-conflicts" role="doc-backlink">Reduced chance of metaclass conflicts</a></h3>
<p>One of the big issues that makes library authors reluctant to use metaclasses
(even when they would be appropriate) is the risk of metaclass conflicts.
These occur whenever two unrelated metaclasses are used by the desired
parents of a class definition. This risk also makes it very difficult to
<em>add</em> a metaclass to a class that has previously been published without one.</p>
<p>By contrast, adding an <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> method to an existing type poses
a similar level of risk to adding an <code class="docutils literal notranslate"><span class="pre">__init__</span></code> method: technically, there
is a risk of breaking poorly implemented subclasses, but when that occurs,
it is recognised as a bug in the subclass rather than the library author
breaching backwards compatibility guarantees.</p>
</section>
</section>
<section id="new-ways-of-using-classes">
<h2><a class="toc-backref" href="#new-ways-of-using-classes" role="doc-backlink">New Ways of Using Classes</a></h2>
<section id="subclass-registration">
<h3><a class="toc-backref" href="#subclass-registration" role="doc-backlink">Subclass registration</a></h3>
<p>Especially when writing a plugin system, one likes to register new
subclasses of a plugin baseclass. This can be done as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">PluginBase</span><span class="p">:</span>
<span class="n">subclasses</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">__init_subclass__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init_subclass__</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">cls</span><span class="o">.</span><span class="n">subclasses</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span>
</pre></div>
</div>
<p>In this example, <code class="docutils literal notranslate"><span class="pre">PluginBase.subclasses</span></code> will contain a plain list of all
subclasses in the entire inheritance tree. One should note that this also
works nicely as a mixin class.</p>
</section>
<section id="trait-descriptors">
<h3><a class="toc-backref" href="#trait-descriptors" role="doc-backlink">Trait descriptors</a></h3>
<p>There are many designs of Python descriptors in the wild which, for
example, check boundaries of values. Often those “traits” need some support
of a metaclass to work. This is how this would look like with this
PEP:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Trait</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">minimum</span><span class="p">,</span> <span class="n">maximum</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">minimum</span> <span class="o">=</span> <span class="n">minimum</span>
<span class="bp">self</span><span class="o">.</span><span class="n">maximum</span> <span class="o">=</span> <span class="n">maximum</span>
<span class="k">def</span> <span class="fm">__get__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">instance</span><span class="p">,</span> <span class="n">owner</span><span class="p">):</span>
<span class="k">return</span> <span class="n">instance</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">key</span><span class="p">]</span>
<span class="k">def</span> <span class="fm">__set__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">instance</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">minimum</span> <span class="o">&lt;</span> <span class="n">value</span> <span class="o">&lt;</span> <span class="bp">self</span><span class="o">.</span><span class="n">maximum</span><span class="p">:</span>
<span class="n">instance</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s2">&quot;value not in range&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__set_name__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">owner</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">key</span> <span class="o">=</span> <span class="n">name</span>
</pre></div>
</div>
</section>
</section>
<section id="implementation-details">
<h2><a class="toc-backref" href="#implementation-details" role="doc-backlink">Implementation Details</a></h2>
<p>The hooks are called in the following order: <code class="docutils literal notranslate"><span class="pre">type.__new__</span></code> calls
the <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> hooks on the descriptor after the new class has been
initialized. Then it calls <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> on the base class, on
<code class="docutils literal notranslate"><span class="pre">super()</span></code>, to be precise. This means that subclass initializers already
see the fully initialized descriptors. This way, <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> users
can fix all descriptors again if this is needed.</p>
<p>Another option would have been to call <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> in the base
implementation of <code class="docutils literal notranslate"><span class="pre">object.__init_subclass__</span></code>. This way it would be possible
even to prevent <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> from being called. Most of the times,
however, such a prevention would be accidental, as it often happens that a call
to <code class="docutils literal notranslate"><span class="pre">super()</span></code> is forgotten.</p>
<p>As a third option, all the work could have been done in <code class="docutils literal notranslate"><span class="pre">type.__init__</span></code>.
Most metaclasses do their work in <code class="docutils literal notranslate"><span class="pre">__new__</span></code>, as this is recommended by
the documentation. Many metaclasses modify their arguments before they
pass them over to <code class="docutils literal notranslate"><span class="pre">super().__new__</span></code>. For compatibility with those kind
of classes, the hooks should be called from <code class="docutils literal notranslate"><span class="pre">__new__</span></code>.</p>
<p>Another small change should be done: in the current implementation of
CPython, <code class="docutils literal notranslate"><span class="pre">type.__init__</span></code> explicitly forbids the use of keyword arguments,
while <code class="docutils literal notranslate"><span class="pre">type.__new__</span></code> allows for its attributes to be shipped as keyword
arguments. This is weirdly incoherent, and thus it should be forbidden.
While it would be possible to retain the current behavior, it would be better
if this was fixed, as it is probably not used at all: the only use case would
be that at metaclass calls its <code class="docutils literal notranslate"><span class="pre">super().__new__</span></code> with <em>name</em>, <em>bases</em> and
<em>dict</em> (yes, <em>dict</em>, not <em>namespace</em> or <em>ns</em> as mostly used with modern
metaclasses) as keyword arguments. This should not be done. This little
change simplifies the implementation of this PEP significantly, while
improving the coherence of Python overall.</p>
<p>As a second change, the new <code class="docutils literal notranslate"><span class="pre">type.__init__</span></code> just ignores keyword
arguments. Currently, it insists that no keyword arguments are given. This
leads to a (wanted) error if one gives keyword arguments to a class declaration
if the metaclass does not process them. Metaclass authors that do want to
accept keyword arguments must filter them out by overriding <code class="docutils literal notranslate"><span class="pre">__init__</span></code>.</p>
<p>In the new code, it is not <code class="docutils literal notranslate"><span class="pre">__init__</span></code> that complains about keyword arguments,
but <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>, whose default implementation takes no arguments. In
a classical inheritance scheme using the method resolution order, each
<code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> may take out its keyword arguments until none are left,
which is checked by the default implementation of <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>.</p>
<p>For readers who prefer reading Python over English, this PEP proposes to
replace the current <code class="docutils literal notranslate"><span class="pre">type</span></code> and <code class="docutils literal notranslate"><span class="pre">object</span></code> with the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">NewType</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">3</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">ns</span> <span class="o">=</span> <span class="n">args</span>
<span class="n">init</span> <span class="o">=</span> <span class="n">ns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;__init_subclass__&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">init</span><span class="p">,</span> <span class="n">types</span><span class="o">.</span><span class="n">FunctionType</span><span class="p">):</span>
<span class="n">ns</span><span class="p">[</span><span class="s1">&#39;__init_subclass__&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="nb">classmethod</span><span class="p">(</span><span class="n">init</span><span class="p">)</span>
<span class="bp">self</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">ns</span><span class="p">)</span>
<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="n">func</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="s1">&#39;__set_name__&#39;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">if</span> <span class="n">func</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">func</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">k</span><span class="p">)</span>
<span class="nb">super</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">__init_subclass__</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">ns</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">ns</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">NewObject</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">__init_subclass__</span><span class="p">(</span><span class="bp">cls</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
</div>
</section>
<section id="reference-implementation">
<h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2>
<p>The reference implementation for this PEP is attached to
<a class="reference external" href="http://bugs.python.org/issue27366">issue 27366</a>.</p>
</section>
<section id="backward-compatibility-issues">
<h2><a class="toc-backref" href="#backward-compatibility-issues" role="doc-backlink">Backward compatibility issues</a></h2>
<p>The exact calling sequence in <code class="docutils literal notranslate"><span class="pre">type.__new__</span></code> is slightly changed, raising
fears of backwards compatibility. It should be assured by tests that common use
cases behave as desired.</p>
<p>The following class definitions (except the one defining the metaclass)
continue to fail with a <code class="docutils literal notranslate"><span class="pre">TypeError</span></code> as superfluous class arguments are passed:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyMeta</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">MyClass</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">,</span> <span class="n">otherarg</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="k">pass</span>
<span class="n">MyMeta</span><span class="p">(</span><span class="s2">&quot;MyClass&quot;</span><span class="p">,</span> <span class="p">(),</span> <span class="n">otherargs</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="kn">import</span> <span class="nn">types</span>
<span class="n">types</span><span class="o">.</span><span class="n">new_class</span><span class="p">(</span><span class="s2">&quot;MyClass&quot;</span><span class="p">,</span> <span class="p">(),</span> <span class="nb">dict</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">,</span> <span class="n">otherarg</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
<span class="n">types</span><span class="o">.</span><span class="n">prepare_class</span><span class="p">(</span><span class="s2">&quot;MyClass&quot;</span><span class="p">,</span> <span class="p">(),</span> <span class="nb">dict</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">,</span> <span class="n">otherarg</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
</pre></div>
</div>
<p>A metaclass defining only a <code class="docutils literal notranslate"><span class="pre">__new__</span></code> method which is interested in keyword
arguments now does not need to define an <code class="docutils literal notranslate"><span class="pre">__init__</span></code> method anymore, as the
default <code class="docutils literal notranslate"><span class="pre">type.__init__</span></code> ignores keyword arguments. This is nicely in line
with the recommendation to override <code class="docutils literal notranslate"><span class="pre">__new__</span></code> in metaclasses instead of
<code class="docutils literal notranslate"><span class="pre">__init__</span></code>. The following code does not fail anymore:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyMeta</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">namespace</span><span class="p">,</span> <span class="n">otherarg</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">namespace</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">MyClass</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">,</span> <span class="n">otherarg</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
</div>
<p>Only defining an <code class="docutils literal notranslate"><span class="pre">__init__</span></code> method in a metaclass continues to fail with
<code class="docutils literal notranslate"><span class="pre">TypeError</span></code> if keyword arguments are given:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyMeta</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">namespace</span><span class="p">,</span> <span class="n">otherarg</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">namespace</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">MyClass</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">,</span> <span class="n">otherarg</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
</div>
<p>Defining both <code class="docutils literal notranslate"><span class="pre">__init__</span></code> and <code class="docutils literal notranslate"><span class="pre">__new__</span></code> continues to work fine.</p>
<p>About the only thing that stops working is passing the arguments of
<code class="docutils literal notranslate"><span class="pre">type.__new__</span></code> as keyword arguments:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyMeta</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">namespace</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="o">=</span><span class="n">bases</span><span class="p">,</span>
<span class="nb">dict</span><span class="o">=</span><span class="n">namespace</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">MyClass</span><span class="p">(</span><span class="n">metaclass</span><span class="o">=</span><span class="n">MyMeta</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
</div>
<p>This will now raise <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>, but this is weird code, and easy
to fix even if someone used this feature.</p>
</section>
<section id="rejected-design-options">
<h2><a class="toc-backref" href="#rejected-design-options" role="doc-backlink">Rejected Design Options</a></h2>
<section id="calling-the-hook-on-the-class-itself">
<h3><a class="toc-backref" href="#calling-the-hook-on-the-class-itself" role="doc-backlink">Calling the hook on the class itself</a></h3>
<p>Adding an <code class="docutils literal notranslate"><span class="pre">__autodecorate__</span></code> hook that would be called on the class
itself was the proposed idea of <a class="pep reference internal" href="../pep-0422/" title="PEP 422 Simpler customisation of class creation">PEP 422</a>. Most examples work the same
way or even better if the hook is called only on strict subclasses. In general,
it is much easier to arrange to explicitly call the hook on the class in which it
is defined (to opt-in to such a behavior) than to opt-out (by remember to check for
<code class="docutils literal notranslate"><span class="pre">cls</span> <span class="pre">is</span> <span class="pre">__class</span></code> in the hook body), meaning that one does not want the hook to be
called on the class it is defined in.</p>
<p>This becomes most evident if the class in question is designed as a
mixin: it is very unlikely that the code of the mixin is to be
executed for the mixin class itself, as it is not supposed to be a
complete class on its own.</p>
<p>The original proposal also made major changes in the class
initialization process, rendering it impossible to back-port the
proposal to older Python versions.</p>
<p>When its desired to also call the hook on the base class, two mechanisms are available:</p>
<ol class="arabic simple">
<li>Introduce an additional mixin class just to hold the <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>
implementation. The original “base” class can then list the new mixin as its
first parent class.</li>
<li>Implement the desired behaviour as an independent class decorator, and apply that
decorator explicitly to the base class, and then implicitly to subclasses via
<code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code>.</li>
</ol>
<p>Calling <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> explicitly from a class decorator will generally be
undesirable, as this will also typically call <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> a second time on
the parent class, which is unlikely to be desired behaviour.</p>
</section>
<section id="other-variants-of-calling-the-hooks">
<h3><a class="toc-backref" href="#other-variants-of-calling-the-hooks" role="doc-backlink">Other variants of calling the hooks</a></h3>
<p>Other names for the hook were presented, namely <code class="docutils literal notranslate"><span class="pre">__decorate__</span></code> or
<code class="docutils literal notranslate"><span class="pre">__autodecorate__</span></code>. This proposal opts for <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> as
it is very close to the <code class="docutils literal notranslate"><span class="pre">__init__</span></code> method, just for the subclass,
while it is not very close to decorators, as it does not return the
class.</p>
<p>For the <code class="docutils literal notranslate"><span class="pre">__set_name__</span></code> hook other names have been proposed as well,
<code class="docutils literal notranslate"><span class="pre">__set_owner__</span></code>, <code class="docutils literal notranslate"><span class="pre">__set_ownership__</span></code> and <code class="docutils literal notranslate"><span class="pre">__init_descriptor__</span></code>.</p>
</section>
<section id="requiring-an-explicit-decorator-on-init-subclass">
<h3><a class="toc-backref" href="#requiring-an-explicit-decorator-on-init-subclass" role="doc-backlink">Requiring an explicit decorator on <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code></a></h3>
<p>One could require the explicit use of <code class="docutils literal notranslate"><span class="pre">&#64;classmethod</span></code> on the
<code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code> decorator. It was made implicit since theres no
sensible interpretation for leaving it out, and that case would need
to be detected anyway in order to give a useful error message.</p>
<p>This decision was reinforced after noticing that the user experience of
defining <code class="docutils literal notranslate"><span class="pre">__prepare__</span></code> and forgetting the <code class="docutils literal notranslate"><span class="pre">&#64;classmethod</span></code> method
decorator is singularly incomprehensible (particularly since <a class="pep reference internal" href="../pep-3115/" title="PEP 3115 Metaclasses in Python 3000">PEP 3115</a>
documents it as an ordinary method, and the current documentation doesnt
explicitly say anything one way or the other).</p>
</section>
<section id="a-more-new-like-hook">
<h3><a class="toc-backref" href="#a-more-new-like-hook" role="doc-backlink">A more <code class="docutils literal notranslate"><span class="pre">__new__</span></code>-like hook</a></h3>
<p>In <a class="pep reference internal" href="../pep-0422/" title="PEP 422 Simpler customisation of class creation">PEP 422</a> the hook worked more like the <code class="docutils literal notranslate"><span class="pre">__new__</span></code> method than the
<code class="docutils literal notranslate"><span class="pre">__init__</span></code> method, meaning that it returned a class instead of
modifying one. This allows a bit more flexibility, but at the cost
of much harder implementation and undesired side effects.</p>
</section>
<section id="adding-a-class-attribute-with-the-attribute-order">
<h3><a class="toc-backref" href="#adding-a-class-attribute-with-the-attribute-order" role="doc-backlink">Adding a class attribute with the attribute order</a></h3>
<p>This got its own <a class="pep reference internal" href="../pep-0520/" title="PEP 520 Preserving Class Attribute Definition Order">PEP 520</a>.</p>
</section>
</section>
<section id="history">
<h2><a class="toc-backref" href="#history" role="doc-backlink">History</a></h2>
<p>This used to be a competing proposal to <a class="pep reference internal" href="../pep-0422/" title="PEP 422 Simpler customisation of class creation">PEP 422</a> by Alyssa Coghlan and Daniel
Urban. <a class="pep reference internal" href="../pep-0422/" title="PEP 422 Simpler customisation of class creation">PEP 422</a> intended to achieve the same goals as this PEP, but with a
different way of implementation. In the meantime, <a class="pep reference internal" href="../pep-0422/" title="PEP 422 Simpler customisation of class creation">PEP 422</a> has been withdrawn
favouring this approach.</p>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document has been placed in the public domain.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0487.rst">https://github.com/python/peps/blob/main/peps/pep-0487.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0487.rst">2023-10-11 12:05:51 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#background">Background</a></li>
<li><a class="reference internal" href="#proposal">Proposal</a></li>
<li><a class="reference internal" href="#key-benefits">Key Benefits</a><ul>
<li><a class="reference internal" href="#easier-inheritance-of-definition-time-behaviour">Easier inheritance of definition time behaviour</a></li>
<li><a class="reference internal" href="#reduced-chance-of-metaclass-conflicts">Reduced chance of metaclass conflicts</a></li>
</ul>
</li>
<li><a class="reference internal" href="#new-ways-of-using-classes">New Ways of Using Classes</a><ul>
<li><a class="reference internal" href="#subclass-registration">Subclass registration</a></li>
<li><a class="reference internal" href="#trait-descriptors">Trait descriptors</a></li>
</ul>
</li>
<li><a class="reference internal" href="#implementation-details">Implementation Details</a></li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#backward-compatibility-issues">Backward compatibility issues</a></li>
<li><a class="reference internal" href="#rejected-design-options">Rejected Design Options</a><ul>
<li><a class="reference internal" href="#calling-the-hook-on-the-class-itself">Calling the hook on the class itself</a></li>
<li><a class="reference internal" href="#other-variants-of-calling-the-hooks">Other variants of calling the hooks</a></li>
<li><a class="reference internal" href="#requiring-an-explicit-decorator-on-init-subclass">Requiring an explicit decorator on <code class="docutils literal notranslate"><span class="pre">__init_subclass__</span></code></a></li>
<li><a class="reference internal" href="#a-more-new-like-hook">A more <code class="docutils literal notranslate"><span class="pre">__new__</span></code>-like hook</a></li>
<li><a class="reference internal" href="#adding-a-class-attribute-with-the-attribute-order">Adding a class attribute with the attribute order</a></li>
</ul>
</li>
<li><a class="reference internal" href="#history">History</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0487.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>