2020-07-08 05:56:33 -04:00
|
|
|
|
PEP: 625
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Title: Filename of a Source Distribution
|
2020-07-08 05:56:33 -04:00
|
|
|
|
Author: Tzu-ping Chung <uranusjr@gmail.com>,
|
|
|
|
|
Paul Moore <p.f.moore@gmail.com>
|
|
|
|
|
Discussions-To: https://discuss.python.org/t/draft-pep-file-name-of-a-source-distribution/4686
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Status: Deferred
|
2020-07-08 05:56:33 -04:00
|
|
|
|
Type: Standards Track
|
2022-06-14 17:22:20 -04:00
|
|
|
|
Topic: Packaging
|
2020-07-08 05:56:33 -04:00
|
|
|
|
Content-Type: text/x-rst
|
|
|
|
|
Created: 08-Jul-2020
|
|
|
|
|
Post-History: 08-Jul-2020
|
|
|
|
|
|
|
|
|
|
Abstract
|
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
This PEP describes a standard naming scheme for a Source Distribution, also
|
2022-07-09 06:27:18 -04:00
|
|
|
|
known as an *sdist*. An sdist is distinct from an arbitrary archive file
|
|
|
|
|
containing source code of Python packages, and can be used to communicate
|
|
|
|
|
information about the distribution to packaging tools.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
A standard sdist specified here is a gzipped tar file with a specially
|
2022-07-09 06:27:18 -04:00
|
|
|
|
formatted filename and the usual ``.tar.gz`` suffix. This PEP does not specify
|
|
|
|
|
the contents of the tarball, as that is covered in other specifications.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
**Note**: This PEP has been deferred until :pep:`643` has seen wider adoption
|
|
|
|
|
(in particular, until Metadata 2.2 is accepted on PyPI, and a number of common
|
|
|
|
|
backends have implemented it).
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
Motivation
|
|
|
|
|
==========
|
|
|
|
|
|
|
|
|
|
An sdist is a Python package distribution that contains "source code" of the
|
|
|
|
|
Python package, and requires a build step to be turned into a wheel on
|
|
|
|
|
installation. This format is often considered as an unbuilt counterpart of a
|
|
|
|
|
:pep:`427` wheel, and given special treatments in various parts of the
|
|
|
|
|
packaging ecosystem.
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
The content of an sdist is specified in :pep:`517` and :pep:`643`, but currently
|
|
|
|
|
the filename of the sdist is incompletely specified, meaning that consumers
|
|
|
|
|
of the format must download and process the sdist to confirm the name and
|
|
|
|
|
version of the distribution included within.
|
|
|
|
|
|
|
|
|
|
Installers currently rely on heuristics to infer the name and/or version from
|
|
|
|
|
the filename, to help the installation process. pip, for example, parses the
|
|
|
|
|
filename of an sdist from a :pep:`503` index, to obtain the distribution's
|
|
|
|
|
project name and version for dependency resolution purposes. But due to the
|
|
|
|
|
lack of specification, the installer does not have any guarantee as to the
|
|
|
|
|
correctness of the inferred data, and must verify it at some point by locally
|
|
|
|
|
building the distribution metadata.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2020-07-15 07:08:23 -04:00
|
|
|
|
This build step is awkward for a certain class of operations, when the user
|
2020-07-08 05:56:33 -04:00
|
|
|
|
does not expect the build process to occur. `pypa/pip#8387`_ describes an
|
|
|
|
|
example. The command ``pip download --no-deps --no-binary=numpy numpy`` is
|
|
|
|
|
expected to only download an sdist for numpy, since we do not need to check
|
|
|
|
|
for dependencies, and both the name and version are available by introspecting
|
2022-07-09 06:27:18 -04:00
|
|
|
|
the downloaded filename. pip, however, cannot assume the downloaded archive
|
2020-07-15 07:08:23 -04:00
|
|
|
|
follows the convention, and must build and check the metadata. For a :pep:`518`
|
2020-07-08 05:56:33 -04:00
|
|
|
|
project, this means running the ``prepare_metadata_for_build_wheel`` hook
|
|
|
|
|
specified in :pep:`517`, which incurs significant overhead.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rationale
|
|
|
|
|
=========
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
By creating a special filename scheme for the sdist format, this PEP frees up
|
2020-07-08 05:56:33 -04:00
|
|
|
|
tools from the time-consuming metadata verification step when they only need
|
2022-07-09 06:27:18 -04:00
|
|
|
|
the metadata available in the filename.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
This PEP also serves as the formal specification to the long-standing
|
2022-07-09 06:27:18 -04:00
|
|
|
|
filename convention used by the current sdist implementations. The filename
|
2020-07-08 05:56:33 -04:00
|
|
|
|
contains the distribution name and version, to aid tools identifying a
|
2020-07-15 07:08:23 -04:00
|
|
|
|
distribution without needing to download, unarchive the file, and perform
|
2020-07-08 05:56:33 -04:00
|
|
|
|
costly metadata generation for introspection, if all the information they need
|
2022-07-09 06:27:18 -04:00
|
|
|
|
is available in the filename.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Specification
|
|
|
|
|
=============
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
The name of an sdist should be ``{distribution}-{version}.tar.gz``.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
* ``distribution`` is the name of the distribution as defined in :pep:`345`,
|
2022-07-09 06:27:18 -04:00
|
|
|
|
and normalised as described in `the wheel spec`_ e.g. ``'pip'``,
|
|
|
|
|
``'flit_core'``.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
* ``version`` is the version of the distribution as defined in :pep:`440`,
|
2022-07-09 06:27:18 -04:00
|
|
|
|
e.g. ``20.2``, and normalised according to the rules in that PEP.
|
|
|
|
|
|
|
|
|
|
An sdist must be a gzipped tar archive in pax format, that is able to be
|
|
|
|
|
extracted by the standard library ``tarfile`` module with the open flag
|
|
|
|
|
``'r:gz'``.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Code that produces an sdist file MUST give the file a name that matches this
|
|
|
|
|
specification. The specification of the ``build_sdist`` hook from :pep:`517` is
|
|
|
|
|
extended to require this naming convention.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Code that processes sdist files MAY determine the distribution name and version
|
|
|
|
|
by simply parsing the filename, and is not required to verify that information
|
|
|
|
|
by generating or reading the metadata from the sdist contents.
|
|
|
|
|
|
|
|
|
|
Conforming sdist files can be recognised by the presence of the ``.tar.gz``
|
|
|
|
|
suffix and a *single* hyphen in the filename. Note that some legacy files may
|
|
|
|
|
also match these criteria, but this is not expected to be an issue in practice.
|
|
|
|
|
See the "Backwards Compatibility" section of this document for more details.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Backwards Compatibility
|
|
|
|
|
=======================
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
The new filename scheme is a subset of the current informal naming
|
|
|
|
|
convention for sdist files, so tools that create or publish files conforming
|
|
|
|
|
to this standard will be readable by older tools that only understand the
|
|
|
|
|
previous naming conventions.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Tools that consume sdist filenames would technically not be able to determine
|
|
|
|
|
whether a file is using the new standard or a legacy form. However, a review
|
|
|
|
|
of the filenames on PyPI determined that 37% of files are obviously legacy
|
|
|
|
|
(because they contain multiple or no hyphens) and of the remainder, parsing
|
|
|
|
|
according to this PEP gives the correct answer in all but 0.004% of cases.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Currently, tools that consume sdists should, if they are to be fully correct,
|
|
|
|
|
treat the name and version parsed from the filename as provisional, and verify
|
|
|
|
|
them by downloading the file and generating the actual metadata (or reading it,
|
|
|
|
|
if the sdist conforms to :pep:`643`). Tools supporting this specification can
|
|
|
|
|
treat the name and version from the filename as definitive. In theory, this
|
|
|
|
|
could risk mistakes if a legacy filename is assumed to conform to this PEP,
|
|
|
|
|
but in practice the chance of this appears to be vanishingly small.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rejected Ideas
|
|
|
|
|
==============
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Rely on the specification for sdist metadata
|
|
|
|
|
--------------------------------------------
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Since this PEP was first written, :pep:`643` has been accepted, defining a
|
|
|
|
|
trustworthy, standard sdist metadata format. This allows distribution metadata
|
|
|
|
|
(and in particular name and version) to be determined statically.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
This is not considered sufficient, however, as in a number of significant
|
|
|
|
|
cases (for example, reading filenames from a package index) the application
|
|
|
|
|
only has access to the filename, and reading metadata would involve a
|
|
|
|
|
potentially costly download.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
Use a dedicated file extension
|
|
|
|
|
------------------------------
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
The original version of this PEP proposed a filename of
|
|
|
|
|
``{distribution}-{version}.sdist``. This has the advantage of being explicit,
|
|
|
|
|
as well as allowing a future change to the storage format without needing a
|
|
|
|
|
further change of the file naming convention.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
However, there are significant compatibility issues with a new extension. Index
|
|
|
|
|
servers may currently disallow unknown extensions, and if we introduced a new
|
|
|
|
|
one, it is not clear how to handle cases like a legacy index trying to mirror an
|
|
|
|
|
index that hosts new-style sdists. Is it acceptable to only partially mirror,
|
|
|
|
|
omitting sdists for newer versions of projects? Also, build backends that produce
|
|
|
|
|
the new format would be incompaible with index servers that only accept the old
|
|
|
|
|
format, and as there is often no way for a user to request an older version of a
|
|
|
|
|
backend when doing a build, this could make it impossible to build and upload
|
|
|
|
|
sdists.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
Augment a currently common sdist naming scheme
|
|
|
|
|
----------------------------------------------
|
|
|
|
|
|
|
|
|
|
A scheme ``{distribution}-{version}.sdist.tar.gz`` was raised during the
|
|
|
|
|
initial discussion. This was abandoned due to backwards compatibility issues
|
|
|
|
|
with currently available installation tools. pip 20.1, for example, would
|
|
|
|
|
parse ``distribution-1.0.sdist.tar.gz`` as project ``distribution`` with
|
2020-07-15 07:08:23 -04:00
|
|
|
|
version ``1.0.sdist``. This would cause the sdist to be downloaded, but fail to
|
2020-07-08 05:56:33 -04:00
|
|
|
|
install due to inconsistent metadata.
|
|
|
|
|
|
2022-07-09 06:27:18 -04:00
|
|
|
|
The main advantage of this proposal was that it is easier for tools to
|
|
|
|
|
recognise the new-style naming. But this is not a particularly significant
|
|
|
|
|
benefit, given that all sdists with a single hyphen in the name are parsed
|
|
|
|
|
the same way under the old and new rules.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Open Issues
|
|
|
|
|
===========
|
|
|
|
|
|
|
|
|
|
The contents of an sdist are required to contain a single top-level directory
|
|
|
|
|
named ``{name}-{version}``. Currently no normalisation rules are required
|
|
|
|
|
for the components of this name. Should this PEP require that the same normalisation
|
|
|
|
|
rules are applied here as for the filename? Note that in practice, it is likely
|
|
|
|
|
that tools will create the two names using the same code, so normalisation is
|
|
|
|
|
likely to happen naturally, even if it is not explicitly required.
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
References
|
|
|
|
|
==========
|
|
|
|
|
|
|
|
|
|
.. _`pypa/pip#8387`: https://github.com/pypa/pip/issues/8387
|
2022-07-09 06:27:18 -04:00
|
|
|
|
.. _`the wheel spec`: https://packaging.python.org/en/latest/specifications/binary-distribution-format/
|
2020-07-08 05:56:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Copyright
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
This document is placed in the public domain or under the CC0-1.0-Universal
|
|
|
|
|
license, whichever is more permissive.
|
|
|
|
|
|
|
|
|
|
|
2022-01-16 17:33:05 -05:00
|
|
|
|
|
|
|
|
|
..
|
2020-07-08 05:56:33 -04:00
|
|
|
|
Local Variables:
|
|
|
|
|
mode: indented-text
|
|
|
|
|
indent-tabs-mode: nil
|
|
|
|
|
sentence-end-double-space: t
|
|
|
|
|
fill-column: 70
|
|
|
|
|
coding: utf-8
|
|
|
|
|
End:
|