209 lines
8.1 KiB
ReStructuredText
209 lines
8.1 KiB
ReStructuredText
PEP: 643
|
|
Title: Metadata for Package Source Distributions
|
|
Author: Paul Moore <p.f.moore@gmail.com>
|
|
BDFL-Delegate: Paul Ganssle <paul@ganssle.io>
|
|
Discussions-To: https://discuss.python.org/t/pep-643-metadata-for-package-source-distributions/5577
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 24-Oct-2020
|
|
Post-History: 24-Oct-2020, 01-Nov-2020, 02-Nov-2020
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
Python package metadata is stored in the distribution file in a standard
|
|
format, defined in the `Core Metadata Specification`_. However, for
|
|
source distributions, while the format of the data is defined, there has
|
|
traditionally been a lot of inconsistency in what data is recorded in
|
|
the sdist. See `here
|
|
<https://discuss.python.org/t/why-isnt-source-distribution-metadata-trustworthy-can-we-make-it-so/2620>`_
|
|
for a discussion of this issue.
|
|
|
|
As a result, metadata consumers are unable to rely on the data available
|
|
from source distributions, and need to use the (costly) :pep:`517` build
|
|
mechanisms to extract medatata.
|
|
|
|
This PEP defines a standard that allows build backends to reliably store
|
|
package metadata in the source distribution, while still retaining the
|
|
necessary flexibility to handle metadata fields that have to be calculated
|
|
at build time. It further defines a set of metadata values that must be
|
|
fixed when the sdist is created, ensuring that consumers have a minimum
|
|
"core" of metadata they can be sure is available.
|
|
|
|
|
|
Motivation
|
|
==========
|
|
|
|
There are a number of issues with the way that metadata is currently
|
|
stored in source distributions:
|
|
|
|
* The details of how to store metadata, while standardised, are not
|
|
easy to find.
|
|
* The specification requires an old metadata version, and has not been
|
|
updated in line with changes to the core metadata spec.
|
|
* There is no way in the spec to distinguish between "this field has been
|
|
omitted because its value will not be known until build time" and "this
|
|
field does not have a value".
|
|
* The core metadata specification allows most fields to be optional,
|
|
meaning that the previous issue affects nearly every metadata field.
|
|
|
|
This PEP proposes an update to the metadata specification to allow
|
|
recording of fields which are expected to be "filled in later", and
|
|
updates the sdist specification to clarify that backends should record
|
|
sdist metadata using that version of the spec (or later). It restricts
|
|
which fields can be "filled in later", so that a core set of metadata is
|
|
available, and reliable, when read from a sdist.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
:pep:`621` proposes a mechanism for users to specify metadata in
|
|
``pyproject.toml``. As part of that mechanism, a way was needed to say
|
|
that a particular field is defined dynamically by the backend. During
|
|
discussions on the PEP, it became clear that the same type of mechanism
|
|
would address the issue of distinguishing between "not known yet" and
|
|
"definitely has no value" in sdists. This PEP defines the ``Dynamic``
|
|
metadata field by analogy with the ``dynamic`` field in :pep:`621`.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
This PEP defines the relationship between metadata values specified in
|
|
a sdist, and the corresponding values in wheels built from that sdist.
|
|
It requires build backends to clearly mark any fields which will *not*
|
|
simply be copied unchanged from the sdist to the wheel.
|
|
|
|
In addition, this PEP makes the `PyPA Specifications`_ document the
|
|
canonical location for the specification of the sdist format (collecting
|
|
the information in :pep:`517` and in this PEP).
|
|
|
|
A new field, ``Dynamic``, will be added to the `Core Metadata Specification`_.
|
|
This field will be multiple use, and will be allowed to contain the name
|
|
of another core metadata field. The ``Dynamic`` metadata item is only
|
|
allowed in source distribution metadata.
|
|
|
|
If tools that read metadata encounter the ``Dynamic`` field anywhere except
|
|
in a sdist, they MAY fail with an error. If they do not fail, they MUST
|
|
ignore the field, and may report a warning.
|
|
|
|
If a field is marked as ``Dynamic``, there is no restriction placed on
|
|
its value in a wheel built from the sdist. A field which is marked as
|
|
``Dynamic``, MUST NOT have an explicit value in the sdist.
|
|
|
|
If a field is *not* marked as ``Dynamic``, then the value of the field
|
|
in any wheel built from the sdist MUST match the value in the sdist.
|
|
If the field is not in the sdist, and not marked as ``Dynamic``, then it
|
|
MUST NOT be present in the wheel.
|
|
|
|
Build backends MUST ensure that these rules are followed, and MUST
|
|
report an error if they are unable to do so.
|
|
|
|
The following fields are the only ones allowed to be marked as ``Dynamic``:
|
|
|
|
* ``Platform``
|
|
* ``Supported-Platform``
|
|
* ``Requires-Dist``
|
|
* ``Requires-External``
|
|
* ``Provides-Extra``
|
|
* ``Provides-Dist``
|
|
* ``Obsoletes-Dist``
|
|
|
|
As it adds a new metadata field, this PEP updates the core metadata
|
|
format to version 2.2.
|
|
|
|
Source distributions MUST use the latest version of the core metadata
|
|
specification (which will be version 2.2 or later).
|
|
|
|
Build backends SHOULD encourage projects to avoid using ``Dynamic``,
|
|
preferring to use environment markers on static values to adapt to
|
|
details of the install location.
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
As this proposal increments the core metadata version, it is compatible
|
|
with existing sdists, which will use an older metadata version. Tools
|
|
can determine whether a sdist conforms to this PEP by checking the
|
|
metadata version.
|
|
|
|
|
|
Security Implications
|
|
=====================
|
|
|
|
As this specification is purely for the storage of data that is intended
|
|
to be publicly available, there are no security implications.
|
|
|
|
|
|
How to Teach This
|
|
=================
|
|
|
|
This is a data storage format for project metadata, and so will not
|
|
typically be visible to end users. There is therefore no need to teach
|
|
users how to use the format. Developers wanting to reference the
|
|
metadata will be able to find the details in the `PyPA Specifications`_.
|
|
|
|
|
|
Rejected Ideas
|
|
==============
|
|
|
|
1. Rather than marking fields as ``Dynamic``, fields should be assumed
|
|
to be dynamic unless explicitly marked as ``Static``.
|
|
|
|
This is logically equivalent to the current proposal, but it implies
|
|
that fields being dynamic is the norm. Packaging tools can be much
|
|
more efficient in the presence of metadata that is known to be static,
|
|
so the PEP chooses to make dynamic fields the exception, and require
|
|
backends to "opt in" to making a field dynamic.
|
|
|
|
2. Rather than having a ``Dynamic`` field, add a special value that
|
|
indicates that a field is "not yet defined".
|
|
|
|
Again, this is logically equivalent to the current proposal. It makes
|
|
"being dynamic" an explicit choice, but requires a special value. As
|
|
some fields can contain arbitrary text, choosing a such a value is
|
|
somewhat awkward (although likely not a problem in practice). There
|
|
does not seem to be enough benefit to this approach to make it worth
|
|
using instead of the proposed mechanism.
|
|
|
|
3. Allow ``Requires-Python`` to be ``Dynamic``, as it cannot include environment
|
|
markers to tailor the requirement to the target environment.
|
|
|
|
Currently, no projects on PyPI have a ``Requires-Python`` value that varies
|
|
between different wheels for the same version, so there is no practical
|
|
need for this flexibility at present. If a genuine use case is identified
|
|
later, the specification can be changed to allow ``Rquires-Python`` to be
|
|
dynamic at that time.
|
|
|
|
4. Allow ``Dynamic`` to be used in wheels and/or installed distributions.
|
|
|
|
There is no obvious value to allowing this, and it seems like it is simply
|
|
adding complexity for no real reason. Allowing this could be done in a
|
|
follow-up proposal if there turned out to be sufficient benefit.
|
|
|
|
5. Allow a field to be marked as ``Dynamic``, but *also* have a value in the
|
|
sdist metadata.
|
|
|
|
There appears to be no use case for allowing this. If a use case is
|
|
identified in the future, the specification can be updated at that time.
|
|
|
|
Open Issues
|
|
===========
|
|
|
|
None
|
|
|
|
References
|
|
==========
|
|
|
|
.. _Core Metadata Specification: https://packaging.python.org/specifications/core-metadata/
|
|
.. _PyPA Specifications: https://packaging.python.org/specifications/
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|