python-peps/pep-0610/index.html

661 lines
49 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 610 Recording the Direct URL Origin of installed distributions | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0610/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 610 Recording the Direct URL Origin of installed distributions | peps.python.org'>
<meta property="og:description" content="Following PEP 440, a distribution can be identified by a name and either a version, or a direct URL reference (see PEP440 Direct References). After installation, the name and version are captured in the project metadata, but currently there is no way to...">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0610/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="Following PEP 440, a distribution can be identified by a name and either a version, or a direct URL reference (see PEP440 Direct References). After installation, the name and version are captured in the project metadata, but currently there is no way to...">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 610</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 610 Recording the Direct URL Origin of installed distributions</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Stéphane Bidoul &lt;stephane.bidoul&#32;&#97;t&#32;gmail.com&gt;, Chris Jerdonek &lt;chris.jerdonek&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-even">Sponsor<span class="colon">:</span></dt>
<dd class="field-even">Alyssa Coghlan &lt;ncoghlan&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-odd">BDFL-Delegate<span class="colon">:</span></dt>
<dd class="field-odd">Pradyun Gedam &lt;pradyunsg&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-even">Discussions-To<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/recording-the-source-url-of-an-installed-distribution/1535">Discourse thread</a></dd>
<dt class="field-odd">Status<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-even">Type<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-odd">Topic<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="../topic/packaging/">Packaging</a></dd>
<dt class="field-even">Created<span class="colon">:</span></dt>
<dd class="field-even">21-Apr-2019</dd>
<dt class="field-odd">Post-History<span class="colon">:</span></dt>
<dd class="field-odd"><p></p></dd>
<dt class="field-even">Resolution<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/1535/56">Discourse message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a><ul>
<li><a class="reference internal" href="#installation-from-direct-url-references">Installation from direct URL references</a></li>
<li><a class="reference internal" href="#freezing-an-environment">Freezing an environment</a></li>
<li><a class="reference internal" href="#the-importance-of-installing-from-vcs-urls-for-application-integrators">The importance of installing from (VCS) URLs for application integrators</a></li>
<li><a class="reference internal" href="#additional-origin-metadata-available-for-vcs-urls">Additional origin metadata available for VCS URLs</a></li>
<li><a class="reference internal" href="#note-about-editable-installs">Note about “editable” installs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#registered-vcs">Registered VCS</a><ul>
<li><a class="reference internal" href="#git">Git</a></li>
<li><a class="reference internal" href="#mercurial">Mercurial</a></li>
<li><a class="reference internal" href="#bazaar">Bazaar</a></li>
<li><a class="reference internal" href="#subversion">Subversion</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#examples">Examples</a><ul>
<li><a class="reference internal" href="#example-direct-url-json">Example direct_url.json</a></li>
<li><a class="reference internal" href="#example-pip-commands-and-their-effect-on-direct-url-json">Example pip commands and their effect on direct_url.json</a></li>
</ul>
</li>
<li><a class="reference internal" href="#use-cases">Use cases</a></li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li>
<li><a class="reference internal" href="#alternatives">Alternatives</a><ul>
<li><a class="reference internal" href="#pep-426-source-url">PEP 426 source_url</a></li>
<li><a class="reference internal" href="#revision-vs-ref">revision vs ref</a></li>
</ul>
</li>
<li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<div class="pep-banner canonical-pypa-spec sticky-banner admonition attention">
<p class="admonition-title">Attention</p>
<p>This PEP is a historical document. The up-to-date, canonical spec, <a class="reference external" href="https://packaging.python.org/en/latest/specifications/direct-url/#direct-url" title="(in Python Packaging User Guide)"><span>Recording the Direct URL Origin of installed distributions</span></a>, is maintained on the <a class="reference external" href="https://packaging.python.org/en/latest/specifications/">PyPA specs page</a>.</p>
<p class="close-button">×</p>
<p>See the <a class="reference external" href="https://www.pypa.io/en/latest/specifications/#handling-fixes-and-other-minor-updates">PyPA specification update process</a> for how to propose changes.</p>
</div>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>Following <a class="pep reference internal" href="../pep-0440/" title="PEP 440 Version Identification and Dependency Specification">PEP 440</a>, a distribution can be identified by a name and either a
version, or a direct URL reference (see <a class="pep reference internal" href="../pep-0440/#direct-references" title="PEP 440 Version Identification and Dependency Specification § Direct references">PEP440 Direct References</a>).
After installation, the name and version are captured in the project metadata,
but currently there is no way to obtain details of the URL used when the
distribution was identified by a direct URL reference.</p>
<p>This proposal defines
additional metadata, to be added to the installed distribution by the
installation front end, which records the Direct URL Origin for use by
consumers which introspect the database of installed packages (see <a class="pep reference internal" href="../pep-0376/" title="PEP 376 Database of Installed Python Distributions">PEP 376</a>).</p>
</section>
<section id="motivation">
<h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2>
<p>The original motivation of this PEP was to permit tools with a “freeze”
operation allowing a Python environment to be recreated to work in a broader
range of situations.</p>
<p>Specifically, the PEP originated from the desire to address <a class="reference external" href="https://github.com/pypa/pip/issues/609">pip issue #609</a>:
i.e. improving the behavior of <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">freeze</span></code> in the presence of distributions
installed from direct URL references. It follows a
<a class="reference external" href="https://discuss.python.org/t/pip-freeze-vcs-urls-and-pep-517-feat-editable-installs/1473">thread on discuss.python.org</a> about the best course of action to implement
it.</p>
<section id="installation-from-direct-url-references">
<h3><a class="toc-backref" href="#installation-from-direct-url-references" role="doc-backlink">Installation from direct URL references</a></h3>
<p>Python installers such as pip are capable of downloading and installing
distributions from package indexes. They are also capable of downloading
and installing source code from requirements specifying arbitrary URLs of
source archives and Version Control Systems (VCS) repositories,
as standardized in <a class="pep reference internal" href="../pep-0440/#direct-references" title="PEP 440 Version Identification and Dependency Specification § Direct references">PEP440 Direct References</a>.</p>
<p>In other words, two relevant installation modes exist.</p>
<ol class="arabic simple">
<li>the package to install is specified as a name and version specifier:</li>
</ol>
<blockquote>
<div>In this case, the installer looks in a package index (or optionally
using <code class="docutils literal notranslate"><span class="pre">--find-links</span></code> in the case of pip) to find the distribution to install.</div></blockquote>
<ol class="arabic simple" start="2">
<li>The package to install is specified as a direct URL reference:</li>
</ol>
<blockquote>
<div>In this case, the installer downloads whatever is specified by the URL
(typically a wheel, a source archive or a VCS repository) and installs it.<p>In this mode, installers typically download the source code in a
temporary directory, invoke the <a class="pep reference internal" href="../pep-0517/" title="PEP 517 A build-system independent format for source trees">PEP 517</a> build backend to produce a wheel
if needed, install the wheel, and delete the temporary directory.</p>
<p>After installation, no trace of the URL the user requested to download the
package is left on the user system.</p>
</div></blockquote>
</section>
<section id="freezing-an-environment">
<h3><a class="toc-backref" href="#freezing-an-environment" role="doc-backlink">Freezing an environment</a></h3>
<p>Pip also sports a command named <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">freeze</span></code> which examines the Database of
Installed Python Distributions to generate a list of requirements. The main
goal of this command is to help users generating a list of requirements that
will later allow the re-installation the same environment with the highest
possible fidelity.</p>
<p>As of pip version 19.3, the <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">freeze</span></code> command outputs a <code class="docutils literal notranslate"><span class="pre">name==version</span></code>
line for each installed
distribution (except for editable installs). To achieve the goal of
reinstalling the same environment, this requires the (name, version)
tuple to refer to an immutable version of the
distribution. The immutability is guaranteed by package indexes
such as Warehouse. The package index to use is typically known from
environmental or command line parameters of the installer.</p>
<p>This freeze mechanism therefore works fine for installation mode 1 (i.e.
when the package to install was specified as a name plus version specifier).</p>
<p>For installation mode 2, i.e. when the package to install was specified as a
direct URL reference, the <code class="docutils literal notranslate"><span class="pre">name==version</span></code> tuple is obviously not sufficient
to reinstall the same distribution and users of the freeze command expect it
to output the URL that was originally requested.</p>
<p>The reasoning above is equally applicable to tools, other than <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">freeze</span></code>,
that would attempt to generate a <code class="docutils literal notranslate"><span class="pre">Pipfile.lock</span></code> or any other similar format
from the Database of Installed Python Distributions. Unless specified
otherwise, “freeze” is used in this document as a generic term for such
an operation.</p>
</section>
<section id="the-importance-of-installing-from-vcs-urls-for-application-integrators">
<h3><a class="toc-backref" href="#the-importance-of-installing-from-vcs-urls-for-application-integrators" role="doc-backlink">The importance of installing from (VCS) URLs for application integrators</a></h3>
<p>For an application integrator, it is important to be able to reliably install
and freeze unreleased version of python distributions.
For instance when a developer needs to deploy an unreleased patched version
of a dependency, it is common to install the dependency directly from a VCS
branch that has the patch, while waiting for the maintainer to release an
updated version.</p>
<p>In such cases, it is important for “freeze” to pin the exact VCS
reference (commit-hash if available) that was installed, in order to create
reproducible builds with the highest possible fidelity.</p>
</section>
<section id="additional-origin-metadata-available-for-vcs-urls">
<h3><a class="toc-backref" href="#additional-origin-metadata-available-for-vcs-urls" role="doc-backlink">Additional origin metadata available for VCS URLs</a></h3>
<p>For VCS URLs, there is additional origin information available only at
install time useful for introspection and certain workflows. For example,
when installing a revision from a VCS URL, a tool can determine if the
revision corresponds to a branch, tag or (in the case of Git) a ref. This
information can be used when introspecting the Database of Installed Distributions
to communicate to users more information about what version was installed
(e.g. whether a branch or tag was installed and, if so, the name of the
branch or tag). This also permits one to know whether a <a class="pep reference internal" href="../pep-0440/" title="PEP 440 Version Identification and Dependency Specification">PEP 440</a> direct
reference URL can be constructed using the tag form, as only tags have the
semantics of immutability.</p>
<p>In cases where the revision is mutable (e.g. branches and Git refs), knowing
this information enables workflows where users can e.g. update to the latest
version of a branch they are tracking, or update to the latest version of a
pull request they are reviewing locally. In contrast, when the revision is a
tag, tools can know in advance (e.g. without network calls) that no update is
needed.</p>
<p>As with the URL itself, if this information isnt recorded at install time
when the VCS repository is available, it would otherwise be lost.</p>
</section>
<section id="note-about-editable-installs">
<h3><a class="toc-backref" href="#note-about-editable-installs" role="doc-backlink">Note about “editable” installs</a></h3>
<p>The editable installation mode of pip roughly lets a user insert a
local directory in sys.path for development purpose. This mode is somewhat
abused to work around the fact that a non-editable install from a VCS URL
loses track of the origin after installation.
Indeed, editable installs implicitly record the VCS origin in the checkout
directory, so the information can be recovered when running “freeze”.</p>
<p>The use of this workaround, although useful, is fragile, creates confusion
about the purpose of the editable mode, and works only when the distribution
can be installed with setuptools (i.e. it is not usable with other <a class="pep reference internal" href="../pep-0517/" title="PEP 517 A build-system independent format for source trees">PEP 517</a>
build backends).</p>
<p>When this PEP is implemented, it will not be necessary anymore to use
editable installs for the purpose of making pip freeze work correctly with
VCS references.</p>
</section>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<p>This PEP specifies a new <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code> metadata file in the
<code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory of an installed distribution.</p>
<p>The fields specified are sufficient to reproduce the source archive and <a class="reference external" href="https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support">VCS
URLs supported by pip</a>. They are also sufficient to reproduce
<a class="pep reference internal" href="../pep-0440/#direct-references" title="PEP 440 Version Identification and Dependency Specification § Direct references">PEP440 Direct References</a>,
as well as <a class="reference external" href="https://github.com/pypa/pipfile">Pipfile and Pipfile.lock</a> entries. Finally, they
are sufficient to record the branch, tag, and/or Git ref origin of the
installed version that is already available for editable installs by virtue
of a VCS checkout being present.</p>
<p>Since at least three different ways already exist to encode this type of
information, this PEP uses a dictionary format, so as not to make any
assumption on how a direct
URL reference must ultimately be encoded in a requirement or lockfile. See also
the <a class="reference internal" href="#alternatives">Alternatives</a> section below for more discussion about this choice.</p>
<p>Information has been taken from Rubys bundler manual to verify it has similar
capabilities and inform the selection and naming of fields in this
specifications.</p>
<p>The JSON format allows for the addition of additional fields in the future.</p>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<p>This PEP specifies a <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code> file in the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory
of an installed distribution, to record the Direct URL Origin of the distribution.</p>
<p>The canonical source for the name and semantics of this metadata file is
the <a class="reference external" href="https://packaging.python.org/specifications/direct-url">Recording the Direct URL Origin of installed distributions</a> document.</p>
<p>This file MUST be created by installers when installing a distribution
from a requirement specifying a direct URL reference (including a VCS URL).</p>
<p>This file MUST NOT be created when installing a distribution from an other
type of requirement (i.e. name plus version specifier).</p>
<p>This JSON file MUST be a dictionary, compliant with <span class="target" id="index-0"></span><a class="rfc reference external" href="https://datatracker.ietf.org/doc/html/rfc8259.html"><strong>RFC 8259</strong></a> and UTF-8
encoded.</p>
<p>If present, it MUST contain at least two fields. The first one is <code class="docutils literal notranslate"><span class="pre">url</span></code>, with
type <code class="docutils literal notranslate"><span class="pre">string</span></code>. Depending on what <code class="docutils literal notranslate"><span class="pre">url</span></code> refers to, the second field MUST be
one of <code class="docutils literal notranslate"><span class="pre">vcs_info</span></code> (if <code class="docutils literal notranslate"><span class="pre">url</span></code> is a VCS reference), <code class="docutils literal notranslate"><span class="pre">archive_info</span></code> (if
<code class="docutils literal notranslate"><span class="pre">url</span></code> is a source archives or a wheel), or <code class="docutils literal notranslate"><span class="pre">dir_info</span></code> (if <code class="docutils literal notranslate"><span class="pre">url</span></code> is a
local directory). These info fields have a (possibly empty) subdictionary as
value, with the possible keys defined below.</p>
<p><code class="docutils literal notranslate"><span class="pre">url</span></code> MUST be stripped of any sensitive authentication information,
for security reasons.</p>
<p>The user:password section of the URL MAY however
be composed of environment variables, matching the following regular
expression:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>\$\{[A-Za-z0-9-_]+\}(:\$\{[A-Za-z0-9-_]+\})?
</pre></div>
</div>
<p>Additionally, the user:password section of the URL MAY be a
well-known, non security sensitive string. A typical example is <code class="docutils literal notranslate"><span class="pre">git</span></code>
in the case of an URL such as <code class="docutils literal notranslate"><span class="pre">ssh://git&#64;gitlab.com</span></code>.</p>
<p>When <code class="docutils literal notranslate"><span class="pre">url</span></code> refers to a VCS repository, the <code class="docutils literal notranslate"><span class="pre">vcs_info</span></code> key MUST be present
as a dictionary with the following keys:</p>
<ul class="simple">
<li>A <code class="docutils literal notranslate"><span class="pre">vcs</span></code> key (type <code class="docutils literal notranslate"><span class="pre">string</span></code>) MUST be present, containing the name of the VCS
(i.e. one of <code class="docutils literal notranslate"><span class="pre">git</span></code>, <code class="docutils literal notranslate"><span class="pre">hg</span></code>, <code class="docutils literal notranslate"><span class="pre">bzr</span></code>, <code class="docutils literal notranslate"><span class="pre">svn</span></code>). Other VCSs SHOULD be registered by
writing a PEP to amend this specification.
The <code class="docutils literal notranslate"><span class="pre">url</span></code> value MUST be compatible with the corresponding VCS,
so an installer can hand it off without transformation to a
checkout/download command of the VCS.</li>
<li>A <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> key (type <code class="docutils literal notranslate"><span class="pre">string</span></code>) MAY be present naming a
branch/tag/ref/commit/revision/etc (in a format compatible with the VCS)
to install.</li>
<li>A <code class="docutils literal notranslate"><span class="pre">commit_id</span></code> key (type <code class="docutils literal notranslate"><span class="pre">string</span></code>) MUST be present, containing the
exact commit/revision number that was installed.
If the VCS supports commit-hash
based revision identifiers, such commit-hash MUST be used as
<code class="docutils literal notranslate"><span class="pre">commit_id</span></code> in order to reference the immutable
version of the source code that was installed.</li>
<li>If the installer could discover additional information about
the requested revision, it MAY add a <code class="docutils literal notranslate"><span class="pre">resolved_revision</span></code> and/or
<code class="docutils literal notranslate"><span class="pre">resolved_revision_type</span></code> field. If no revision was provided in
the requested URL, <code class="docutils literal notranslate"><span class="pre">resolved_revision</span></code> MAY contain the default branch
that was installed, and <code class="docutils literal notranslate"><span class="pre">resolved_revision_type</span></code> will be <code class="docutils literal notranslate"><span class="pre">branch</span></code>.
If the installer determines that <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> was a tag, it MAY
add <code class="docutils literal notranslate"><span class="pre">resolved_revision_type</span></code> with value <code class="docutils literal notranslate"><span class="pre">tag</span></code>.</li>
</ul>
<p>When <code class="docutils literal notranslate"><span class="pre">url</span></code> refers to a source archive or a wheel, the <code class="docutils literal notranslate"><span class="pre">archive_info</span></code> key
MUST be present as a dictionary with the following key:</p>
<ul class="simple">
<li>A <code class="docutils literal notranslate"><span class="pre">hash</span></code> key (type <code class="docutils literal notranslate"><span class="pre">string</span></code>) SHOULD be present, with value
<code class="docutils literal notranslate"><span class="pre">&lt;hash-algorithm&gt;=&lt;expected-hash&gt;</span></code>.
It is RECOMMENDED that only hashes which are unconditionally provided by
the latest version of the standard librarys <code class="docutils literal notranslate"><span class="pre">hashlib</span></code> module be used for
source archive hashes. At time of writing, that list consists of md5,
sha1, sha224, sha256, sha384, and sha512.</li>
</ul>
<p>When <code class="docutils literal notranslate"><span class="pre">url</span></code> refers to a local directory, the <code class="docutils literal notranslate"><span class="pre">dir_info</span></code> key MUST be
present as a dictionary with the following key:</p>
<ul class="simple">
<li><code class="docutils literal notranslate"><span class="pre">editable</span></code> (type: <code class="docutils literal notranslate"><span class="pre">boolean</span></code>): <code class="docutils literal notranslate"><span class="pre">true</span></code> if the distribution was installed
in editable mode, <code class="docutils literal notranslate"><span class="pre">false</span></code> otherwise. If absent, default to <code class="docutils literal notranslate"><span class="pre">false</span></code>.</li>
</ul>
<p>When <code class="docutils literal notranslate"><span class="pre">url</span></code> refers to a local directory, it MUST have the <code class="docutils literal notranslate"><span class="pre">file</span></code> scheme
and be compliant with <span class="target" id="index-1"></span><a class="rfc reference external" href="https://datatracker.ietf.org/doc/html/rfc8089.html"><strong>RFC 8089</strong></a>. In particular, the path component must
be absolute. Symbolic links SHOULD be preserved when making relative
paths absolute.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>When the requested URL has the <a class="reference external" href="file://">file://</a> scheme and points to a local directory that happens to contain a
VCS checkout, installers MUST NOT attempt to infer any VCS information and
therefore MUST NOT output any VCS related information (such as <code class="docutils literal notranslate"><span class="pre">vcs_info</span></code>)
in <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code>.</p>
</div>
<p>A top-level <code class="docutils literal notranslate"><span class="pre">subdirectory</span></code> field MAY be present containing a directory path,
relative to the root of the VCS repository, source archive or local directory,
to specify where <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> or <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> is located.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>As a general rule, installers should as much as possible preserve the
information that was provided in the requested URL when generating
<code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code>. For example, user:password environment variables
should be preserved and <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> should reflect the revision that was
provided in the requested URL as faithfully as possible. This information is
however <em>enriched</em> with more precise data, such as <code class="docutils literal notranslate"><span class="pre">commit_id</span></code>.</p>
</div>
<section id="registered-vcs">
<h3><a class="toc-backref" href="#registered-vcs" role="doc-backlink">Registered VCS</a></h3>
<p>This section lists the registered VCSs; expanded, VCS-specific information
on how to use the <code class="docutils literal notranslate"><span class="pre">vcs</span></code>, <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code>, and other fields of
<code class="docutils literal notranslate"><span class="pre">vcs_info</span></code>; and in
some cases additional VCS-specific fields.
Tools MAY support other VCSs although it is RECOMMENDED to register
them by writing a PEP to amend this specification. The <code class="docutils literal notranslate"><span class="pre">vcs</span></code> field SHOULD be the command name
(lowercased). Additional fields that would be necessary to
support such VCS SHOULD be prefixed with the VCS command name.</p>
<section id="git">
<h4><a class="toc-backref" href="#git" role="doc-backlink">Git</a></h4>
<p>Home page</p>
<blockquote>
<div><a class="reference external" href="https://git-scm.com/">https://git-scm.com/</a></div></blockquote>
<p>vcs command</p>
<blockquote>
<div>git</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">vcs</span></code> field</p>
<blockquote>
<div>git</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> field</p>
<blockquote>
<div>A tag name, branch name, Git ref, commit hash, shortened commit hash,
or other commit-ish.</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">commit_id</span></code> field</p>
<blockquote>
<div>A commit hash (40 hexadecimal characters sha1).</div></blockquote>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Installers can use the <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">show-ref</span></code> and <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">symbolic-ref</span></code> commands
to determine if the <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> corresponds to a Git ref.
In turn, a ref beginning with <code class="docutils literal notranslate"><span class="pre">refs/tags/</span></code> corresponds to a tag, and
a ref beginning with <code class="docutils literal notranslate"><span class="pre">refs/remotes/origin/</span></code> after cloning corresponds
to a branch.</p>
</div>
</section>
<section id="mercurial">
<h4><a class="toc-backref" href="#mercurial" role="doc-backlink">Mercurial</a></h4>
<p>Home page</p>
<blockquote>
<div><a class="reference external" href="https://www.mercurial-scm.org/">https://www.mercurial-scm.org/</a></div></blockquote>
<p>vcs command</p>
<blockquote>
<div>hg</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">vcs</span></code> field</p>
<blockquote>
<div>hg</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> field</p>
<blockquote>
<div>A tag name, branch name, changeset ID, shortened changeset ID.</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">commit_id</span></code> field</p>
<blockquote>
<div>A changeset ID (40 hexadecimal characters).</div></blockquote>
</section>
<section id="bazaar">
<h4><a class="toc-backref" href="#bazaar" role="doc-backlink">Bazaar</a></h4>
<p>Home page</p>
<blockquote>
<div><a class="reference external" href="https://bazaar.canonical.com/">https://bazaar.canonical.com/</a></div></blockquote>
<p>vcs command</p>
<blockquote>
<div>bzr</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">vcs</span></code> field</p>
<blockquote>
<div>bzr</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> field</p>
<blockquote>
<div>A tag name, branch name, revision id.</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">commit_id</span></code> field</p>
<blockquote>
<div>A revision id.</div></blockquote>
</section>
<section id="subversion">
<h4><a class="toc-backref" href="#subversion" role="doc-backlink">Subversion</a></h4>
<p>Home page</p>
<blockquote>
<div><a class="reference external" href="https://subversion.apache.org/">https://subversion.apache.org/</a></div></blockquote>
<p>vcs command</p>
<blockquote>
<div>svn</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">vcs</span></code> field</p>
<blockquote>
<div>svn</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> field</p>
<blockquote>
<div><code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> must be compatible with <code class="docutils literal notranslate"><span class="pre">svn</span> <span class="pre">checkout</span></code> <code class="docutils literal notranslate"><span class="pre">--revision</span></code> option.
In Subversion, branch or tag is part of <code class="docutils literal notranslate"><span class="pre">url</span></code>.</div></blockquote>
<p><code class="docutils literal notranslate"><span class="pre">commit_id</span></code> field</p>
<blockquote>
<div>Since Subversion does not support globally unique identifiers,
this field is the Subversion revision number in the corresponding
repository.</div></blockquote>
</section>
</section>
</section>
<section id="examples">
<h2><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h2>
<section id="example-direct-url-json">
<h3><a class="toc-backref" href="#example-direct-url-json" role="doc-backlink">Example direct_url.json</a></h3>
<p>Source archive:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="s2">&quot;https://github.com/pypa/pip/archive/1.3.1.zip&quot;</span><span class="p">,</span>
<span class="s2">&quot;archive_info&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;hash&quot;</span><span class="p">:</span> <span class="s2">&quot;sha256=2dc6b5a470a1bde68946f263f1af1515a2574a150a30d6ce02c6ff742fcc0db8&quot;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Git URL with tag and commit-hash:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="s2">&quot;https://github.com/pypa/pip.git&quot;</span><span class="p">,</span>
<span class="s2">&quot;vcs_info&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;vcs&quot;</span><span class="p">:</span> <span class="s2">&quot;git&quot;</span><span class="p">,</span>
<span class="s2">&quot;requested_revision&quot;</span><span class="p">:</span> <span class="s2">&quot;1.3.1&quot;</span><span class="p">,</span>
<span class="s2">&quot;resolved_revision_type&quot;</span><span class="p">:</span> <span class="s2">&quot;tag&quot;</span><span class="p">,</span>
<span class="s2">&quot;commit_id&quot;</span><span class="p">:</span> <span class="s2">&quot;7921be1537eac1e97bc40179a57f0349c2aee67d&quot;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Local directory:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="s2">&quot;file:///home/user/project&quot;</span><span class="p">,</span>
<span class="s2">&quot;dir_info&quot;</span><span class="p">:</span> <span class="p">{}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Local directory installed in editable mode:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="s2">&quot;file:///home/user/project&quot;</span><span class="p">,</span>
<span class="s2">&quot;dir_info&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;editable&quot;</span><span class="p">:</span> <span class="n">true</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
<section id="example-pip-commands-and-their-effect-on-direct-url-json">
<h3><a class="toc-backref" href="#example-pip-commands-and-their-effect-on-direct-url-json" role="doc-backlink">Example pip commands and their effect on direct_url.json</a></h3>
<p>Commands that generate a <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code>:</p>
<ul class="simple">
<li>pip install <a class="reference external" href="https://example.com/app-1.0.tgz">https://example.com/app-1.0.tgz</a></li>
<li>pip install <a class="reference external" href="https://example.com/app-1.0.whl">https://example.com/app-1.0.whl</a></li>
<li>pip install “git+https://example.com/repo/app.git#egg=app&amp;subdirectory=setup”</li>
<li>pip install ./app</li>
<li>pip install <a class="reference external" href="file:///home/user/app">file:///home/user/app</a></li>
<li>pip install editable “git+https://example.com/repo/app.git#egg=app&amp;subdirectory=setup”
(in which case, <code class="docutils literal notranslate"><span class="pre">url</span></code> will be the local directory where the git repository has been
cloned to, and <code class="docutils literal notranslate"><span class="pre">dir_info</span></code> will be present with <code class="docutils literal notranslate"><span class="pre">&quot;editable&quot;:</span> <span class="pre">true</span></code> and no
<code class="docutils literal notranslate"><span class="pre">vcs_info</span></code> will be set)</li>
<li>pip install -e ./app</li>
</ul>
<p>Commands that <em>do not</em> generate a <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code></p>
<ul class="simple">
<li>pip install app</li>
<li>pip install app no-index find-links <a class="reference external" href="https://example.com/">https://example.com/</a></li>
</ul>
</section>
</section>
<section id="use-cases">
<h2><a class="toc-backref" href="#use-cases" role="doc-backlink">Use cases</a></h2>
<p>“Freezing” an environment</p>
<blockquote>
<div>Tools, such as <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">freeze</span></code>, which generate requirements from the Database
of Installed Python Distributions SHOULD exploit <code class="docutils literal notranslate"><span class="pre">direct_url.json</span></code>
if it is present, and give it priority over the Version metadata in order
to generate a higher fidelity output. In the presence of a <code class="docutils literal notranslate"><span class="pre">vcs</span></code> direct URL reference,
the <code class="docutils literal notranslate"><span class="pre">commit_id</span></code> field SHOULD be used in priority in order to provide
the highest possible fidelity to the originally installed version. If
supported by their requirement format, tools are encouraged also to output
the <code class="docutils literal notranslate"><span class="pre">tag</span></code> value if present, as it has immutable semantics.
Tools MAY choose another approach, depending on the needs of their users.<p>Note the initial iteration of this PEP does not attempt to make environments
that include editable installs or installs from local directories
reproducible, but it does attempt to make them readily identifiable. By
locating the local project directory via the <code class="docutils literal notranslate"><span class="pre">url</span></code> and <code class="docutils literal notranslate"><span class="pre">dir_info</span></code> fields
of this specification, tools can implement any strategy that fits their use
cases.</p>
</div></blockquote>
</section>
<section id="backwards-compatibility">
<h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2>
<p>Since this PEP specifies a new file in the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory,
there are no backwards compatibility implications.</p>
</section>
<section id="alternatives">
<h2><a class="toc-backref" href="#alternatives" role="doc-backlink">Alternatives</a></h2>
<section id="pep-426-source-url">
<h3><a class="toc-backref" href="#pep-426-source-url" role="doc-backlink">PEP 426 source_url</a></h3>
<p>The now withdrawn <a class="pep reference internal" href="../pep-0426/" title="PEP 426 Metadata for Python Software Packages 2.0">PEP 426</a> specifies a <code class="docutils literal notranslate"><span class="pre">source_url</span></code> metadata entry.
It is also implemented in <a class="reference external" href="https://distlib.readthedocs.io">distlib</a>.</p>
<p>It was intended for a slightly different purpose, for use in sdists.</p>
<p>This format lacks support for the <code class="docutils literal notranslate"><span class="pre">subdirectory</span></code> option of pip requirement
URLs. The same limitation is present in <a class="pep reference internal" href="../pep-0440/#direct-references" title="PEP 440 Version Identification and Dependency Specification § Direct references">PEP440 Direct References</a>.</p>
<p>It also lacks explicit support for <a class="reference external" href="https://pip.pypa.io/en/stable/reference/pip_install/#id10">environment variables in the user:password
part of URLs</a>.</p>
<p>The introduction of a key/value extensibility mechanism and support
for environment variables for user:password in <a class="pep reference internal" href="../pep-0440/" title="PEP 440 Version Identification and Dependency Specification">PEP 440</a>, would be necessary
for use in this PEP.</p>
</section>
<section id="revision-vs-ref">
<h3><a class="toc-backref" href="#revision-vs-ref" role="doc-backlink">revision vs ref</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">requested_revision</span></code> key was retained over <code class="docutils literal notranslate"><span class="pre">requested_ref</span></code> as it is a more generic term
across various VCS and <code class="docutils literal notranslate"><span class="pre">ref</span></code> has a specific meaning for <code class="docutils literal notranslate"><span class="pre">git</span></code>.</p>
</section>
</section>
<section id="acknowledgements">
<h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2>
<p>Various people helped make this PEP a reality. Paul F. Moore provided the
essence of the abstract. Alyssa Coghlan suggested the direct_url name.</p>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0610.rst">https://github.com/python/peps/blob/main/peps/pep-0610.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0610.rst">2023-10-11 12:05:51 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a><ul>
<li><a class="reference internal" href="#installation-from-direct-url-references">Installation from direct URL references</a></li>
<li><a class="reference internal" href="#freezing-an-environment">Freezing an environment</a></li>
<li><a class="reference internal" href="#the-importance-of-installing-from-vcs-urls-for-application-integrators">The importance of installing from (VCS) URLs for application integrators</a></li>
<li><a class="reference internal" href="#additional-origin-metadata-available-for-vcs-urls">Additional origin metadata available for VCS URLs</a></li>
<li><a class="reference internal" href="#note-about-editable-installs">Note about “editable” installs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#registered-vcs">Registered VCS</a><ul>
<li><a class="reference internal" href="#git">Git</a></li>
<li><a class="reference internal" href="#mercurial">Mercurial</a></li>
<li><a class="reference internal" href="#bazaar">Bazaar</a></li>
<li><a class="reference internal" href="#subversion">Subversion</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#examples">Examples</a><ul>
<li><a class="reference internal" href="#example-direct-url-json">Example direct_url.json</a></li>
<li><a class="reference internal" href="#example-pip-commands-and-their-effect-on-direct-url-json">Example pip commands and their effect on direct_url.json</a></li>
</ul>
</li>
<li><a class="reference internal" href="#use-cases">Use cases</a></li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li>
<li><a class="reference internal" href="#alternatives">Alternatives</a><ul>
<li><a class="reference internal" href="#pep-426-source-url">PEP 426 source_url</a></li>
<li><a class="reference internal" href="#revision-vs-ref">revision vs ref</a></li>
</ul>
</li>
<li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0610.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>