iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
python-peps/pep-0518/index.html

719 lines
47 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 518 – Specifying Minimum Build System Requirements for Python Projects | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0518/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 518 – Specifying Minimum Build System Requirements for Python Projects | peps.python.org'>
<meta property="og:description" content="This PEP specifies how Python software packages should specify what build dependencies they have in order to execute their chosen build system. As part of this specification, a new configuration file is introduced for software packages to use to specify...">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0518/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="This PEP specifies how Python software packages should specify what build dependencies they have in order to execute their chosen build system. As part of this specification, a new configuration file is introduced for software packages to use to specify...">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 518</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 518 – Specifying Minimum Build System Requirements for Python Projects</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Brett Cannon &lt;brett&#32;&#97;t&#32;python.org&gt;,
Nathaniel J. Smith &lt;njs&#32;&#97;t&#32;pobox.com&gt;,
Donald Stufft &lt;donald&#32;&#97;t&#32;stufft.io&gt;</dd>
<dt class="field-even">BDFL-Delegate<span class="colon">:</span></dt>
<dd class="field-even">Alyssa Coghlan</dd>
<dt class="field-odd">Discussions-To<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/distutils-sig&#64;python.org/">Distutils-SIG list</a></dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Topic<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="../topic/packaging/">Packaging</a></dd>
<dt class="field-odd">Created<span class="colon">:</span></dt>
<dd class="field-odd">10-May-2016</dd>
<dt class="field-even">Post-History<span class="colon">:</span></dt>
<dd class="field-even">10-May-2016,
11-May-2016,
13-May-2016</dd>
<dt class="field-odd">Resolution<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/distutils-sig/2016-May/028969.html">Distutils-SIG message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#file-format">File Format</a></li>
<li><a class="reference internal" href="#build-system-table">build-system table</a></li>
<li><a class="reference internal" href="#tool-table">tool table</a></li>
<li><a class="reference internal" href="#json-schema">JSON Schema</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul>
<li><a class="reference internal" href="#a-semantic-version-key">A semantic version key</a></li>
<li><a class="reference internal" href="#a-more-nested-namespace">A more nested namespace</a></li>
<li><a class="reference internal" href="#other-table-names">Other table names</a></li>
<li><a class="reference internal" href="#other-file-formats">Other file formats</a><ul>
<li><a class="reference internal" href="#overview-of-file-formats-considered">Overview of file formats considered</a></li>
<li><a class="reference internal" href="#json">JSON</a></li>
<li><a class="reference internal" href="#yaml">YAML</a></li>
<li><a class="reference internal" href="#configparser">configparser</a></li>
<li><a class="reference internal" href="#python-literals">Python literals</a></li>
</ul>
</li>
<li><a class="reference internal" href="#sticking-with-setup-cfg">Sticking with <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code></a></li>
<li><a class="reference internal" href="#other-file-names">Other file names</a></li>
</ul>
</li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>This PEP specifies how Python software packages should specify what
build dependencies they have in order to execute their chosen build
system. As part of this specification, a new configuration file is
introduced for software packages to use to specify their build
dependencies (with the expectation that the same configuration file
will be used for future configuration details).</p>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<p>When Python first developed its tooling for building distributions of
software for projects, distutils <a class="footnote-reference brackets" href="#distutils" id="id1">[1]</a> was the chosen
solution. As time went on, setuptools <a class="footnote-reference brackets" href="#setuptools" id="id2">[2]</a> gained popularity
to add some features on top of distutils. Both used the concept of a
<code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file that project maintainers executed to build
distributions of their software (as well as users to install said
distribution).</p>
<p>Using an executable file to specify build requirements under distutils
isn’t an issue as distutils is part of Python’s standard library.
Having the build tool as part of Python means that a <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> has
no external dependency that a project maintainer needs to worry about
to build a distribution of their project. There was no need to specify
any dependency information as the only dependency is Python.</p>
<p>But when a project chooses to use setuptools, the use of an executable
file like <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> becomes an issue. You can’t execute a
<code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file without knowing its dependencies, but currently
there is no standard way to know what those dependencies are in an
automated fashion without executing the <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file where that
information is stored. It’s a catch-22 of a file not being runnable
without knowing its own contents which can’t be known programmatically
unless you run the file.</p>
<p>Setuptools tried to solve this with a <code class="docutils literal notranslate"><span class="pre">setup_requires</span></code> argument to
its <code class="docutils literal notranslate"><span class="pre">setup()</span></code> function <a class="footnote-reference brackets" href="#setup-args" id="id3">[3]</a>. This solution has a number
of issues, such as:</p>
<ul class="simple">
<li>No tooling (besides setuptools itself) can access this information
without executing the <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>, but <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> can’t be
executed without having these items installed.</li>
<li>While setuptools itself will install anything listed in this, they
won’t be installed until <em>during</em> the execution of the <code class="docutils literal notranslate"><span class="pre">setup()</span></code>
function, which means that the only way to actually use anything
added here is through increasingly complex machinations that delay
the import and usage of these modules until later on in the
execution of the <code class="docutils literal notranslate"><span class="pre">setup()</span></code> function.</li>
<li>This cannot include <code class="docutils literal notranslate"><span class="pre">setuptools</span></code> itself nor can it include a
replacement to <code class="docutils literal notranslate"><span class="pre">setuptools</span></code>, which means that projects such as
<code class="docutils literal notranslate"><span class="pre">numpy.distutils</span></code> are largely incapable of utilizing it and
projects cannot take advantage of newer setuptools features until
their users naturally upgrade the version of setuptools to a newer
one.</li>
<li>The items listed in <code class="docutils literal notranslate"><span class="pre">setup_requires</span></code> get implicitly installed
whenever you execute the <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> but one of the common ways
that the <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> is executed is via another tool, such as
<code class="docutils literal notranslate"><span class="pre">pip</span></code>, who is already managing dependencies. This means that
a command like <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">spam</span></code> might end up having both
pip and setuptools downloading and installing packages and end
users needing to configure <em>both</em> tools (and for <code class="docutils literal notranslate"><span class="pre">setuptools</span></code>
without being in control of the invocation) to change settings
like which repository it installs from. It also means that users
need to be aware of the discovery rules for both tools, as one
may support different package formats or determine the latest
version differently.</li>
</ul>
<p>This has culminated in a situation where use of <code class="docutils literal notranslate"><span class="pre">setup_requires</span></code>
is rare, where projects tend to either simply copy and paste snippets
between <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> files or they eschew it all together in favor
of simply documenting elsewhere what they expect the user to have
manually installed prior to attempting to build or install their
project.</p>
<p>All of this has led pip <a class="footnote-reference brackets" href="#pip" id="id4">[4]</a> to simply assume that setuptools is
necessary when executing a <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file. The problem with this,
though, is it doesn’t scale if another project began to gain traction
in the community as setuptools has. It also prevents other projects
from gaining traction due to the friction required to use it with a
project when pip can’t infer the fact that something other than
setuptools is required.</p>
<p>This PEP attempts to rectify the situation by specifying a way to list
the minimal dependencies of the build system of a project in a
declarative fashion in a specific file. This allows a project to list
what build dependencies it has to go from e.g. source checkout to
wheel, while not falling into the catch-22 trap that a <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>
has where tooling can’t infer what a project needs to build itself.
Implementing this PEP will allow projects to specify what build system
they depend on upfront so that tools like pip can make sure that they
are installed in order to run the build system to build the project.</p>
<p>To provide more context and motivation for this PEP, think of the
(rough) steps required to produce a built artifact for a project:</p>
<ol class="arabic simple">
<li>The source checkout of the project.</li>
<li>Installation of the build system.</li>
<li>Execute the build system.</li>
</ol>
<p>This PEP covers step #2. <a class="pep reference internal" href="../pep-0517/" title="PEP 517 – A build-system independent format for source trees">PEP 517</a> covers step #3, including how to have
the build system dynamically specify more dependencies that the build
system requires to perform its job. The purpose of this PEP though, is
to specify the minimal set of requirements for the build system to
simply begin execution.</p>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<section id="file-format">
<h3><a class="toc-backref" href="#file-format" role="doc-backlink">File Format</a></h3>
<p>The build system dependencies will be stored in a file named
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> that is written in the TOML format <a class="footnote-reference brackets" href="#toml" id="id5">[6]</a>.</p>
<p>This format was chosen as it is human-usable (unlike JSON <a class="footnote-reference brackets" href="#id20" id="id6">[7]</a>),
it is flexible enough (unlike configparser <a class="footnote-reference brackets" href="#id22" id="id7">[9]</a>), stems
from a standard (also unlike configparser <a class="footnote-reference brackets" href="#id22" id="id8">[9]</a>), and it
is not overly complex (unlike YAML <a class="footnote-reference brackets" href="#id21" id="id9">[8]</a>). The TOML format is
already in use by the Rust community as part of their
Cargo package manager <a class="footnote-reference brackets" href="#cargo" id="id10">[14]</a> and in private email stated they have
been quite happy with their choice of TOML. A more thorough
discussion as to why various alternatives were not chosen can be read
in the <a class="reference internal" href="#other-file-formats">Other file formats</a> section. The authors do realize, though,
that choice of configuration file format is ultimately subjective and
a choice had to be made and the authors prefer TOML for this situation.</p>
<p>Below we list the tables that tools are expected to recognize/respect.
Tables not specified in this PEP are reserved for future use by other
PEPs.</p>
</section>
<section id="build-system-table">
<h3><a class="toc-backref" href="#build-system-table" role="doc-backlink">build-system table</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code> table is used to store build-related data.
Initially only one key of the table will be valid and is mandatory
for the table: <code class="docutils literal notranslate"><span class="pre">requires</span></code>. This key must have a value of a list
of strings representing <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a> dependencies required to execute the
build system (currently that means what dependencies are required to
execute a <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file).</p>
<p>For the vast majority of Python projects that rely upon setuptools,
the <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file will be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">build</span><span class="o">-</span><span class="n">system</span><span class="p">]</span>
<span class="c1"># Minimum requirements for the build system to execute.</span>
<span class="n">requires</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;setuptools&quot;</span><span class="p">,</span> <span class="s2">&quot;wheel&quot;</span><span class="p">]</span> <span class="c1"># PEP 508 specifications.</span>
</pre></div>
</div>
<p>Because the use of setuptools and wheel are so expansive in the
community at the moment, build tools are expected to use the example
configuration file above as their default semantics when a
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file is not present.</p>
<p>Tools should not require the existence of the <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code> table.
A <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file may be used to store configuration details
other than build-related data and thus lack a <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code> table
legitimately. If the file exists but is lacking the <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code>
table then the default values as specified above should be used.
If the table is specified but is missing required fields then the tool
should consider it an error.</p>
</section>
<section id="tool-table">
<h3><a class="toc-backref" href="#tool-table" role="doc-backlink">tool table</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">[tool]</span></code> table is where any tool related to your Python
project, not just build tools, can have users specify configuration
data as long as they use a sub-table within <code class="docutils literal notranslate"><span class="pre">[tool]</span></code>, e.g. the
<a class="reference external" href="https://pypi.python.org/pypi/flit">flit</a> tool would store its
configuration in <code class="docutils literal notranslate"><span class="pre">[tool.flit]</span></code>.</p>
<p>We need some mechanism to allocate names within the <code class="docutils literal notranslate"><span class="pre">tool.*</span></code>
namespace, to make sure that different projects don’t attempt to use
the same sub-table and collide. Our rule is that a project can use
the subtable <code class="docutils literal notranslate"><span class="pre">tool.$NAME</span></code> if, and only if, they own the entry for
<code class="docutils literal notranslate"><span class="pre">$NAME</span></code> in the Cheeseshop/PyPI.</p>
</section>
<section id="json-schema">
<h3><a class="toc-backref" href="#json-schema" role="doc-backlink">JSON Schema</a></h3>
<p>To provide a type-specific representation of the resulting data from
the TOML file for illustrative purposes only, the following JSON
Schema <a class="footnote-reference brackets" href="#jsonschema" id="id11">[15]</a> would match the data format:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;$schema&quot;</span><span class="p">:</span> <span class="s2">&quot;http://json-schema.org/schema#&quot;</span><span class="p">,</span>
<span class="s2">&quot;type&quot;</span><span class="p">:</span> <span class="s2">&quot;object&quot;</span><span class="p">,</span>
<span class="s2">&quot;additionalProperties&quot;</span><span class="p">:</span> <span class="n">false</span><span class="p">,</span>
<span class="s2">&quot;properties&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;build-system&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;type&quot;</span><span class="p">:</span> <span class="s2">&quot;object&quot;</span><span class="p">,</span>
<span class="s2">&quot;additionalProperties&quot;</span><span class="p">:</span> <span class="n">false</span><span class="p">,</span>
<span class="s2">&quot;properties&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;requires&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;type&quot;</span><span class="p">:</span> <span class="s2">&quot;array&quot;</span><span class="p">,</span>
<span class="s2">&quot;items&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;type&quot;</span><span class="p">:</span> <span class="s2">&quot;string&quot;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="s2">&quot;required&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;requires&quot;</span><span class="p">]</span>
<span class="p">},</span>
<span class="s2">&quot;tool&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;type&quot;</span><span class="p">:</span> <span class="s2">&quot;object&quot;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
</section>
<section id="rejected-ideas">
<h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2>
<section id="a-semantic-version-key">
<h3><a class="toc-backref" href="#a-semantic-version-key" role="doc-backlink">A semantic version key</a></h3>
<p>For future-proofing the structure of the configuration file, a
<code class="docutils literal notranslate"><span class="pre">semantics-version</span></code> key was initially proposed. Defaulting to <code class="docutils literal notranslate"><span class="pre">1</span></code>,
the idea was that if any semantics changes to previously defined keys
or tables occurred which were not backwards-compatible, then the
<code class="docutils literal notranslate"><span class="pre">semantics-version</span></code> would be incremented to a new number.</p>
<p>In the end, though, it was decided that this was a premature
optimization. The expectation is that changes to what is pre-defined
semantically in the configuration file will be rather conservative.
And in the instances where a backwards-incompatible change would have
occurred, different names can be used for the new semantics to avoid
breaking older tools.</p>
</section>
<section id="a-more-nested-namespace">
<h3><a class="toc-backref" href="#a-more-nested-namespace" role="doc-backlink">A more nested namespace</a></h3>
<p>An earlier draft of this PEP had a top-level <code class="docutils literal notranslate"><span class="pre">[package]</span></code> table. The
idea was to impose some scoping for a semantics versioning scheme
(see <a class="reference internal" href="#a-semantic-version-key">A semantic version key</a> for why that idea was rejected).
With the need for scoping removed, the point of having a top-level
table became superfluous.</p>
</section>
<section id="other-table-names">
<h3><a class="toc-backref" href="#other-table-names" role="doc-backlink">Other table names</a></h3>
<p>Another name proposed for the <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code> table was
<code class="docutils literal notranslate"><span class="pre">[build]</span></code>. The alternative name is shorter, but doesn’t convey as
much of the intention of what information is stored in the table. After
a vote on the distutils-sig mailing list, the current name won out.</p>
</section>
<section id="other-file-formats">
<h3><a class="toc-backref" href="#other-file-formats" role="doc-backlink">Other file formats</a></h3>
<p>Several other file formats were put forward for consideration, all
rejected for various reasons. Key requirements were that the format
be editable by human beings and have an implementation that can be
vendored easily by projects. This outright excluded certain formats
like XML which are not friendly towards human beings and were never
seriously discussed.</p>
<section id="overview-of-file-formats-considered">
<h4><a class="toc-backref" href="#overview-of-file-formats-considered" role="doc-backlink">Overview of file formats considered</a></h4>
<p>The key reasons for rejecting the other alternatives considered are
summarised in the following sections, while the full review (including
positive arguments in favour of TOML) can be found at <a class="footnote-reference brackets" href="#file-formats" id="id12">[16]</a>.</p>
<p>TOML was ultimately selected as it provided all the features we
were interested in, while avoiding the downsides introduced by
the alternatives.</p>
<table class="docutils align-default">
<thead>
<tr class="row-odd"><th class="head">Feature</th>
<th class="head">TOML</th>
<th class="head">YAML</th>
<th class="head">JSON</th>
<th class="head">CFG/INI</th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td>Well-defined</td>
<td>yes</td>
<td>yes</td>
<td>yes</td>
<td></td>
</tr>
<tr class="row-odd"><td>Real data types</td>
<td>yes</td>
<td>yes</td>
<td>yes</td>
<td></td>
</tr>
<tr class="row-even"><td>Reliable Unicode</td>
<td>yes</td>
<td>yes</td>
<td>yes</td>
<td></td>
</tr>
<tr class="row-odd"><td>Reliable comments</td>
<td>yes</td>
<td>yes</td>
<td></td>
<td></td>
</tr>
<tr class="row-even"><td>Easy for humans to edit</td>
<td>yes</td>
<td>??</td>
<td></td>
<td>??</td>
</tr>
<tr class="row-odd"><td>Easy for tools to edit</td>
<td>yes</td>
<td>??</td>
<td>yes</td>
<td>??</td>
</tr>
<tr class="row-even"><td>In standard library</td>
<td></td>
<td></td>
<td>yes</td>
<td>yes</td>
</tr>
<tr class="row-odd"><td>Easy for pip to vendor</td>
<td>yes</td>
<td></td>
<td>n/a</td>
<td>n/a</td>
</tr>
</tbody>
</table>
<p>(“??” in the table indicates items where most folks would be
inclined to answer “yes”, but there turn out to be a lot of
quirks and edge cases that arise in practice due to either
the lack of a clear specification, or else the underlying
file format specification being surprisingly complicated)</p>
<p>The <code class="docutils literal notranslate"><span class="pre">pytoml</span></code> TOML parser is ~300 lines of pure Python code,
so being outside the standard library didn’t count heavily
against it.</p>
<p>Python literals were also discussed as a potential format, but
weren’t considered in the file format review (since they’re not
a common pre-existing file format).</p>
</section>
<section id="json">
<h4><a class="toc-backref" href="#json" role="doc-backlink">JSON</a></h4>
<p>The JSON format <a class="footnote-reference brackets" href="#id20" id="id13">[7]</a> was initially considered but quickly
rejected. While great as a human-readable, string-based data exchange
format, the syntax does not lend itself to easy editing by a human
being (e.g. the syntax is more verbose than necessary while not
allowing for comments).</p>
<p>An example JSON file for the proposed data would be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;build&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;requires&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;setuptools&quot;</span><span class="p">,</span>
<span class="s2">&quot;wheel&gt;=0.27&quot;</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
<section id="yaml">
<h4><a class="toc-backref" href="#yaml" role="doc-backlink">YAML</a></h4>
<p>The YAML format <a class="footnote-reference brackets" href="#id21" id="id14">[8]</a> was designed to be a superset of JSON
<a class="footnote-reference brackets" href="#id20" id="id15">[7]</a> while being easier to work with by hand. There are three main
issues with YAML.</p>
<p>One is that the specification is large: 86 pages if printed on
letter-sized paper. That leaves the possibility that someone may use a
feature of YAML that works with one parser but not another. It has
been suggested to standardize on a subset, but that basically means
creating a new standard specific to this file which is not tractable
long-term.</p>
<p>Two is that YAML itself is not safe by default. The specification
allows for the arbitrary execution of code which is best avoided when
dealing with configuration data. It is of course possible to avoid
this behavior – for example, PyYAML provides a <code class="docutils literal notranslate"><span class="pre">safe_load</span></code> operation
– but if any tool carelessly uses <code class="docutils literal notranslate"><span class="pre">load</span></code> instead then they open
themselves up to arbitrary code execution. While this PEP is focused on
the building of projects which inherently involves code execution,
other configuration data such as project name and version number may
end up in the same file someday where arbitrary code execution is not
desired.</p>
<p>And finally, the most popular Python implementation of YAML is
PyYAML <a class="footnote-reference brackets" href="#pyyaml" id="id16">[10]</a> which is a large project of a few thousand lines of
code and an optional C extension module. While in and of itself this
isn’t necessarily an issue, this becomes more of a problem for
projects like pip where they would most likely need to vendor PyYAML
as a dependency so as to be fully self-contained (otherwise you end
up with your install tool needing an install tool to work). A
proof-of-concept re-working of PyYAML has been done to see how easy
it would be to potentially vendor a simpler version of the library
which shows it is a possibility.</p>
<p>An example YAML file is:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">build</span><span class="p">:</span>
<span class="n">requires</span><span class="p">:</span>
<span class="o">-</span> <span class="n">setuptools</span>
<span class="o">-</span> <span class="n">wheel</span><span class="o">&gt;=</span><span class="mf">0.27</span>
</pre></div>
</div>
</section>
<section id="configparser">
<h4><a class="toc-backref" href="#configparser" role="doc-backlink">configparser</a></h4>
<p>An INI-style configuration file based on what
configparser <a class="footnote-reference brackets" href="#id22" id="id17">[9]</a> accepts was considered. Unfortunately
there is no specification of what configparser accepts, leading to
support skew between versions. For instance, what ConfigParser in
Python 2.7 accepts is not the same as what configparser in Python 3
accepts. While one could standardize on what Python 3 accepts and
simply vendor the backport of the configparser module, that does mean
this PEP would have to codify that the backport of configparser must
be used by all project wishes to consume the metadata specified by
this PEP. This is overly restrictive and could lead to confusion if
someone is not aware of that a specific version of configparser is
expected.</p>
<p>An example INI file is:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">build</span><span class="p">]</span>
<span class="n">requires</span> <span class="o">=</span>
<span class="n">setuptools</span>
<span class="n">wheel</span><span class="o">&gt;=</span><span class="mf">0.27</span>
</pre></div>
</div>
</section>
<section id="python-literals">
<h4><a class="toc-backref" href="#python-literals" role="doc-backlink">Python literals</a></h4>
<p>Someone proposed using Python literals as the configuration format.
The file would contain one dict at the top level, with the data all
inside that dict, with sections defined by the keys. All Python
programmers would be used to the format, there would implicitly be no
third-party dependency to read the configuration data, and it can be
safe if parsed by <code class="docutils literal notranslate"><span class="pre">ast.literal_eval()</span></code> <a class="footnote-reference brackets" href="#ast-literal-eval" id="id18">[13]</a>.
Python literals can be identical to JSON, with the added benefit of
supporting trailing commas and comments. In addition, Python’s richer
data model may be useful for some future configuration needs (e.g. non-string
dict keys, floating point vs. integer values).</p>
<p>On the other hand, python literals are a Python-specific format, and
it is anticipated that these data may need to be read by packaging
tools, etc. that are not written in Python.</p>
<p>An example Python literal file for the proposed data would be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># The build configuration</span>
<span class="p">{</span><span class="s2">&quot;build&quot;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&quot;requires&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;setuptools&quot;</span><span class="p">,</span>
<span class="s2">&quot;wheel&gt;=0.27&quot;</span><span class="p">,</span> <span class="c1"># note the trailing comma</span>
<span class="c1"># &quot;numpy&gt;=1.10&quot; # a commented out data line</span>
<span class="p">]</span>
<span class="c1"># and here is an arbitrary comment.</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
</section>
<section id="sticking-with-setup-cfg">
<h3><a class="toc-backref" href="#sticking-with-setup-cfg" role="doc-backlink">Sticking with <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code></a></h3>
<p>There are two issues with <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> used by setuptools as a general
format. One is that they are <code class="docutils literal notranslate"><span class="pre">.ini</span></code> files which have issues as mentioned
in the <a class="reference internal" href="#id22">configparser</a> discussion above. The other is that the schema for
that file has never been rigorously defined and thus it’s unknown which
format would be safe to use going forward without potentially confusing
setuptools installations.</p>
</section>
<section id="other-file-names">
<h3><a class="toc-backref" href="#other-file-names" role="doc-backlink">Other file names</a></h3>
<p>Several other file names were considered and rejected (although this
is very much a bikeshedding topic, and so the decision comes down to
mostly taste).</p>
<dl class="simple">
<dt>pysettings.toml</dt><dd>Most reasonable alternative.</dd>
<dt>pypa.toml</dt><dd>While it makes sense to reference the PyPA <a class="footnote-reference brackets" href="#pypa" id="id19">[11]</a>, it is a
somewhat niche term. It’s better to have the file name make sense
without having domain-specific knowledge.</dd>
<dt>pybuild.toml</dt><dd>From the restrictive perspective of this PEP this filename makes
sense, but if any non-build metadata ever gets added to the file
then the name ceases to make sense.</dd>
<dt>pip.toml</dt><dd>Too tool-specific.</dd>
<dt>meta.toml</dt><dd>Too generic; project may want to have its own metadata file.</dd>
<dt>setup.toml</dt><dd>While keeping with traditional thanks to <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>, it does not
necessarily match what the file may contain in the future (e.g. is
knowing the name of a project inherently part of its setup?).</dd>
<dt>pymeta.toml</dt><dd>Not obvious to newcomers to programming and/or Python.</dd>
<dt>pypackage.toml &amp; pypackaging.toml</dt><dd>Name conflation of what a “package” is (project versus namespace).</dd>
<dt>pydevelop.toml</dt><dd>The file may contain details not specific to development.</dd>
<dt>pysource.toml</dt><dd>Not directly related to source code.</dd>
<dt>pytools.toml</dt><dd>Misleading as the file is (currently) aimed at project management.</dd>
<dt>dstufft.toml</dt><dd>Too person-specific. ;)</dd>
</dl>
</section>
</section>
<section id="references">
<h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="distutils" role="doc-footnote">
<dt class="label" id="distutils">[<a href="#id1">1</a>]</dt>
<dd>distutils
(<a class="reference external" href="https://docs.python.org/3/library/distutils.html#module-distutils">https://docs.python.org/3/library/distutils.html#module-distutils</a>)</aside>
<aside class="footnote brackets" id="setuptools" role="doc-footnote">
<dt class="label" id="setuptools">[<a href="#id2">2</a>]</dt>
<dd>setuptools
(<a class="reference external" href="https://pypi.python.org/pypi/setuptools">https://pypi.python.org/pypi/setuptools</a>)</aside>
<aside class="footnote brackets" id="setup-args" role="doc-footnote">
<dt class="label" id="setup-args">[<a href="#id3">3</a>]</dt>
<dd>setuptools: New and Changed setup() Keywords
(<a class="reference external" href="http://pythonhosted.org/setuptools/setuptools.html#new-and-changed-setup-keywords">http://pythonhosted.org/setuptools/setuptools.html#new-and-changed-setup-keywords</a>)</aside>
<aside class="footnote brackets" id="pip" role="doc-footnote">
<dt class="label" id="pip">[<a href="#id4">4</a>]</dt>
<dd>pip
(<a class="reference external" href="https://pypi.python.org/pypi/pip">https://pypi.python.org/pypi/pip</a>)</aside>
<aside class="footnote brackets" id="wheel" role="doc-footnote">
<dt class="label" id="wheel">[5]</dt>
<dd>wheel
(<a class="reference external" href="https://pypi.python.org/pypi/wheel">https://pypi.python.org/pypi/wheel</a>)</aside>
<aside class="footnote brackets" id="toml" role="doc-footnote">
<dt class="label" id="toml">[<a href="#id5">6</a>]</dt>
<dd>TOML
(<a class="reference external" href="https://github.com/toml-lang/toml">https://github.com/toml-lang/toml</a>)</aside>
<aside class="footnote brackets" id="id20" role="doc-footnote">
<dt class="label" id="id20">[7]<em> (<a href='#id6'>1</a>, <a href='#id13'>2</a>, <a href='#id15'>3</a>) </em></dt>
<dd>JSON
(<a class="reference external" href="http://json.org/">http://json.org/</a>)</aside>
<aside class="footnote brackets" id="id21" role="doc-footnote">
<dt class="label" id="id21">[8]<em> (<a href='#id9'>1</a>, <a href='#id14'>2</a>) </em></dt>
<dd>YAML
(<a class="reference external" href="http://yaml.org/">http://yaml.org/</a>)</aside>
<aside class="footnote brackets" id="id22" role="doc-footnote">
<dt class="label" id="id22">[9]<em> (<a href='#id7'>1</a>, <a href='#id8'>2</a>, <a href='#id17'>3</a>) </em></dt>
<dd>configparser
(<a class="reference external" href="https://docs.python.org/3/library/configparser.html#module-configparser">https://docs.python.org/3/library/configparser.html#module-configparser</a>)</aside>
<aside class="footnote brackets" id="pyyaml" role="doc-footnote">
<dt class="label" id="pyyaml">[<a href="#id16">10</a>]</dt>
<dd>PyYAML
(<a class="reference external" href="https://pypi.python.org/pypi/PyYAML">https://pypi.python.org/pypi/PyYAML</a>)</aside>
<aside class="footnote brackets" id="pypa" role="doc-footnote">
<dt class="label" id="pypa">[<a href="#id19">11</a>]</dt>
<dd>PyPA
(<a class="reference external" href="https://www.pypa.io">https://www.pypa.io</a>)</aside>
<aside class="footnote brackets" id="bazel" role="doc-footnote">
<dt class="label" id="bazel">[12]</dt>
<dd>Bazel
(<a class="reference external" href="http://bazel.io/">http://bazel.io/</a>)</aside>
<aside class="footnote brackets" id="ast-literal-eval" role="doc-footnote">
<dt class="label" id="ast-literal-eval">[<a href="#id18">13</a>]</dt>
<dd><code class="docutils literal notranslate"><span class="pre">ast.literal_eval()</span></code>
(<a class="reference external" href="https://docs.python.org/3/library/ast.html#ast.literal_eval">https://docs.python.org/3/library/ast.html#ast.literal_eval</a>)</aside>
<aside class="footnote brackets" id="cargo" role="doc-footnote">
<dt class="label" id="cargo">[<a href="#id10">14</a>]</dt>
<dd>Cargo, Rust’s package manager
(<a class="reference external" href="http://doc.crates.io/">http://doc.crates.io/</a>)</aside>
<aside class="footnote brackets" id="jsonschema" role="doc-footnote">
<dt class="label" id="jsonschema">[<a href="#id11">15</a>]</dt>
<dd>JSON Schema
(<a class="reference external" href="http://json-schema.org/">http://json-schema.org/</a>)</aside>
<aside class="footnote brackets" id="file-formats" role="doc-footnote">
<dt class="label" id="file-formats">[<a href="#id12">16</a>]</dt>
<dd>Nathaniel J. Smith’s file format review
(<a class="reference external" href="https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f">https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f</a>)</aside>
</aside>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document has been placed in the public domain.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0518.rst">https://github.com/python/peps/blob/main/peps/pep-0518.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0518.rst">2023-10-11 12:05:51 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#file-format">File Format</a></li>
<li><a class="reference internal" href="#build-system-table">build-system table</a></li>
<li><a class="reference internal" href="#tool-table">tool table</a></li>
<li><a class="reference internal" href="#json-schema">JSON Schema</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul>
<li><a class="reference internal" href="#a-semantic-version-key">A semantic version key</a></li>
<li><a class="reference internal" href="#a-more-nested-namespace">A more nested namespace</a></li>
<li><a class="reference internal" href="#other-table-names">Other table names</a></li>
<li><a class="reference internal" href="#other-file-formats">Other file formats</a><ul>
<li><a class="reference internal" href="#overview-of-file-formats-considered">Overview of file formats considered</a></li>
<li><a class="reference internal" href="#json">JSON</a></li>
<li><a class="reference internal" href="#yaml">YAML</a></li>
<li><a class="reference internal" href="#configparser">configparser</a></li>
<li><a class="reference internal" href="#python-literals">Python literals</a></li>
</ul>
</li>
<li><a class="reference internal" href="#sticking-with-setup-cfg">Sticking with <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code></a></li>
<li><a class="reference internal" href="#other-file-names">Other file names</a></li>
</ul>
</li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0518.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink