220 lines
8.5 KiB
ReStructuredText
220 lines
8.5 KiB
ReStructuredText
PEP: 643
|
|
Title: Metadata for Package Source Distributions
|
|
Author: Paul Moore <p.f.moore@gmail.com>
|
|
BDFL-Delegate: Paul Ganssle <paul@ganssle.io>
|
|
Discussions-To: https://discuss.python.org/t/pep-643-metadata-for-package-source-distributions/5577
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 24-Oct-2020
|
|
Post-History: 24-Oct-2020
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
Python package metadata is stored in the distribution file in a standard
|
|
format, defined in the `Core Metadata Specification`_. However, for
|
|
source distributions, while the format of the data is defined, there has
|
|
traditionally been a lot of inconsistency in what data is recorded in
|
|
the sdist. See `here
|
|
<https://discuss.python.org/t/why-isnt-source-distribution-metadata-trustworthy-can-we-make-it-so/2620>`_
|
|
for a discussion of this issue.
|
|
|
|
As a result, metadata consumers are unable to rely on the data available
|
|
from source distributions, and need to use the (costly) :pep:`517` build
|
|
mechanisms to extract medatata.
|
|
|
|
This PEP defines a standard that allows build backends to reliably store
|
|
package metadata in the source distribution, while still retaining the
|
|
necessary flexibility to handle metadata fields that have to be calculated
|
|
at build time. It further defines a set of metadata values that must be
|
|
fixed when the sdist is created, ensuring that consumers have a minimum
|
|
"core" of metadata they can be sure is available.
|
|
|
|
|
|
Motivation
|
|
==========
|
|
|
|
There are a number of issues with the way that metadata is currently
|
|
stored in source distributions:
|
|
|
|
* The details of how to store metadata, while standardised, are not
|
|
easy to find.
|
|
* The specification requires an old metadata version, and has not been
|
|
updated in line with changes to the core metadata spec.
|
|
* There is no way in the spec to distinguish between "this field has been
|
|
omitted because its value will not be known until build time" and "this
|
|
field does not have a value".
|
|
* The core metadata specification allows most fields to be optional,
|
|
meaning that the previous issue affects nearly every metadata field.
|
|
|
|
This PEP proposes an update to the metadata specification to allow
|
|
recording of fields which are expected to be "filled in later", and
|
|
updates the sdist specification to clarify that backends should record
|
|
sdist metadata using that version of the spec (or later). It restricts
|
|
which fields can be "filled in later", so that a core set of metadata is
|
|
available, and reliable, when read from a sdist.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
:pep:`621` proposes a mechanism for users to specify metadata in
|
|
``pyproject.toml``. As part of that mechanism, a way was needed to say
|
|
that a particular field is defined dynamically by the backend. During
|
|
discussions on the PEP, it became clear that the same type of mechanism
|
|
would address the issue of distinguishing between "not known yet" and
|
|
"definitely has no value" in sdists. This PEP defines the ``Dynamic``
|
|
metadata field by analogy with the ``dynamic`` field in :pep:`621`.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
This PEP defines the relationship between metadata values specified in
|
|
a sdist, and the corresponding values in wheels built from that sdist.
|
|
It requires build backends to clearly mark any fields which will *not*
|
|
simply be copied unchanged from the sdist to the wheel.
|
|
|
|
A new field, ``Dynamic``, will be added to the `Core Metadata Specification`_.
|
|
This field will be multiple use, and will be allowed to contain the name
|
|
of another core metadata field. The ``Dynamic`` metadata item is only
|
|
allowed in source distribution metadata.
|
|
|
|
If a field is marked as ``Dynamic``, there is no restriction placed on
|
|
its value in a wheel built from the sdist. A field which is marked as
|
|
``Dynamic``, MUST NOT have an explicit value in the sdist.
|
|
|
|
If a field is *not* marked as ``Dynamic``, then the value of the field
|
|
in any wheel built from the sdist MUST match the value in the sdist.
|
|
If the field is not in the sdist, and not marked as ``Dynamic``, then it
|
|
MUST NOT be present in the wheel.
|
|
|
|
Build backends MUST ensure that these rules are followed, and MUST
|
|
report an error if they are unable to do so.
|
|
|
|
The following fields MAY NOT be marked as ``Dynamic``:
|
|
|
|
* ``Name``
|
|
* ``Version``
|
|
* ``Summary``
|
|
* ``Description``
|
|
* ``Requires-Python``
|
|
* ``License``
|
|
* ``Author``
|
|
* ``Author-email``
|
|
* ``Maintainer``
|
|
* ``Maintainer-email``
|
|
* ``Keywords``
|
|
* ``Classifier``
|
|
* ``Project-URL``
|
|
|
|
As it adds a new metadata field, this PEP updates the core metadata
|
|
format to version 2.2.
|
|
|
|
Source distributions MUST use the latest version of the core metadata
|
|
specification (which will be version 2.2 or later).
|
|
|
|
The ``Requires-Python`` field for a project may vary by target platform,
|
|
but is not allowed to be declared as ``Dynamic`` in the sdist metadata.
|
|
To handle this situation, build backends MUST use environment markers on
|
|
the ``Requires-Python`` field to allow that metadata to remain common
|
|
across the sdist and all wheel archives, rather than generating platform
|
|
dependent ``Requires-Python`` metadata as part of the wheel build
|
|
process. Build backends SHOULD also use this approach for other
|
|
metadata fields that may vary by target platform (e.g. dependency
|
|
declarations).
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
As this proposal increments the core metadata version, it is compatible
|
|
with existing sdists, which will use an older metadata version. Tools
|
|
can determine whether a sdist conforms to this PEP by checking the
|
|
metadata version.
|
|
|
|
|
|
Security Implications
|
|
=====================
|
|
|
|
As this specification is purely for the storage of data that is intended
|
|
to be publicly available, there are no security implications.
|
|
|
|
|
|
How to Teach This
|
|
=================
|
|
|
|
This is a data storage format for project metadata, and so will not
|
|
typically be visible to end users. There is therefore no need to teach
|
|
users how to use the format. Developers wanting to reference the
|
|
metadata will be able to find the details in the `PyPA Specifications`_.
|
|
|
|
|
|
Rejected Ideas
|
|
==============
|
|
|
|
1. Rather than marking fields as ``Dynamic``, fields should be assumed
|
|
to be dynamic unless explicitly marked as ``Static``.
|
|
|
|
This is logically equivalent to the current proposal, but it implies
|
|
that fields being dynamic is the norm. Packaging tools can be much
|
|
more efficient in the presence of metadata that is known to be static,
|
|
so the PEP chooses to make dynamic fields the exception, and require
|
|
backends to "opt in" to making a field dynamic.
|
|
|
|
2. Rather than having a ``Dynamic`` field, add a special value that
|
|
indicates that a field is "not yet defined".
|
|
|
|
Again, this is logically equivalent to the current proposal. It makes
|
|
"being dynamic" an explicit choice, but requires a special value. As
|
|
some fields can contain arbitrary text, choosing a such a value is
|
|
somewhat awkward (although likely not a problem in practice). There
|
|
does not seem to be enough benefit to this approach to make it worth
|
|
using instead of the proposed mechanism.
|
|
|
|
Open Issues
|
|
===========
|
|
|
|
1. Should we allow ``Dynamic`` to be used in wheels and/or installed
|
|
distributions?
|
|
|
|
``Dynamic`` has no obvious meaning in either of these situations, and
|
|
the PEP therefore disallows it. However, backends may find it useful
|
|
to simply copy the field across, and it may have some usefulness in
|
|
recording "other wheels built from the source this came from may have
|
|
different values". However, the value seems marginal, and the added
|
|
complexity involved in explaining the feature does not seem worth it.
|
|
|
|
Allowing this could be done in a follow-up proposal if there turned
|
|
out to be sufficient benefit.
|
|
|
|
2. If a field is marked as ``Dynamic``, but has a value in the sdist
|
|
metadata, how should that be interpreted?
|
|
|
|
The simplest answer is to just not allow dynamic fields to have a
|
|
value in the sdist at all. For now, this is what the PEP proposes.
|
|
But is there benefit in having a value which tools can take as a
|
|
"hint" for what the value in the wheel will be?
|
|
|
|
3. Should this PEP change the canonical source for the sdist
|
|
specification to the `PyPA Specifications`_ document?
|
|
|
|
It would be beneficial to collect all of the details of the sdist
|
|
format in one place. However, distribution formats are not currently
|
|
collected there, and making the move would extend the impact of this
|
|
PEP significantly.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. _Core Metadata Specification: https://packaging.python.org/specifications/core-metadata/
|
|
.. _PyPA Specifications: https://packaging.python.org/specifications/
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|