Major update to metadata 2.0 draft specs

- split many fields out to standard extensions
- support for local versions in PEP 440
- addressed several other issues noted in
  https://bitbucket.org/pypa/pypi-metadata-formats/issues
This commit is contained in:
Nick Coghlan 2013-12-21 23:52:04 +10:00
parent c1c59c5be4
commit e8266719af
3 changed files with 434 additions and 802 deletions

File diff suppressed because it is too large Load Diff

View File

@ -9,7 +9,7 @@ Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 18 Mar 2013
Post-History: 30 Mar 2013, 27 May 2013, 20 Jun 2013
Post-History: 30 Mar 2013, 27 May 2013, 20 Jun 2013, 21 Dec 2013
Replaces: 386
@ -91,6 +91,15 @@ Public version identifiers are separated into up to four segments:
Any given release will be a "final release", "pre-release", "post-release" or
"developmental release" as defined in the following sections.
All numeric components MUST be non-negative integers.
All numeric components MUST be interpreted and ordered according to their
numeric value, not as text strings.
All numeric components MAY be zero. Except as described below for the
release segment, a numeric component of zero has no special significance
aside from always being the lowest possible value in the version ordering.
.. note::
Some hard to read version identifiers are permitted by this scheme in
@ -103,6 +112,41 @@ Any given release will be a "final release", "pre-release", "post-release" or
sections.
Local version identifiers
-------------------------
Local version identifiers MUST comply with the following scheme::
<public version identifier>[-N[.N]+]
Local version identifiers are used to denote fully API compatible patched
versions of upstream projects. These are created by application developers
and system integrators when upgrading to a new upstream release would be too
disruptive to the application or other integrated system (such as a Linux
distribution).
Local version identifiers may be used anywhere a public version identifier
is expected.
Local version identifiers MUST NOT include leading or trailing whitespace.
Numeric components in the integrator suffix are interpreted in the same way
as the numeric components of the release segment.
The additional segment after the hyphen is referred to as the "integrator
suffix", and makes it possible to differentiate upstream releases from
potentially altered rebuilds by downstream integrators. The inclusion of an
integrator suffix does not affect the kind of a release, but indicates that
it may not contain the exact same code as the corresponding upstream release.
Public index servers SHOULD NOT allow the use of local version identifiers
in uploaded distributions. Local version identifiers are intended as a tool
for software integrators rather than publishers.
Distributions using a local version identifier SHOULD provide the
``python.integrator`` extension metadata (as defined in :pep:`459`).
Source labels
-------------
@ -146,12 +190,6 @@ of each component of the release segment in turn. When comparing release
segments with different numbers of components, the shorter segment is
padded out with additional zeroes as necessary.
Date based release numbers are declared to be incompatible with this scheme,
as their use is not consistent with the expected API versioning semantics
described below. Accordingly, automated tools SHOULD report an error
when encountering a leading release component greater than or equal
to ``1980``.
While any number of additional components after the first are permitted
under this scheme, the most common variants are to use two components
("major.minor") or three components ("major.minor.micro").
@ -169,6 +207,7 @@ For example::
1.1
2.0
2.0.1
...
A release series is any set of final release numbers that start with a
common prefix. For example, ``3.3.1``, ``3.3.5`` and ``3.3.9.45`` are all
@ -181,6 +220,21 @@ part of the ``3.3`` release series.
form to ``X.Y.0`` when comparing it to any release segment that includes
three components.
Date based release segments are also permitted, and are treated differently
in some cases when used in version specifiers. Any version identifier where
the leading component in the release segment is greater than or equal to
``1980`` is considered to be a date based release.
An example of a date based release scheme using the year and month of the
release::
2012.04
2012.07
2012.10
2013.01
2013.06
...
Pre-releases
------------
@ -359,6 +413,18 @@ and post-releases for minor corrections::
1.1.dev1
...
Date based releases, using an incrementing serial within each year, skipping
zero::
2012.1
2012.2
2012.3
...
2012.15
2013.1
2013.2
...
Summary of permitted suffixes and relative ordering
---------------------------------------------------
@ -423,6 +489,19 @@ The following example covers many of the possible combinations::
1.0.post456
1.1.dev1
The integrator suffix of local version identifiers that share a common
public version identifier prefix MUST be sorted in the same order as
Python's tuple sorting when the integrator suffix is parsed as follows
(this is the same definition as is used for the release segment)::
tuple(map(int, integrator_suffix.split(".")))
All integrator suffixes involved in the comparison MUST be converted to a
consistent length by padding shorter segments with zeroes as needed.
All local version identifiers (even the ``-0`` suffix) are sorted *after*
the corresponding unqualified public version identifier.
Version ordering across different metadata versions
---------------------------------------------------
@ -485,7 +564,7 @@ Semantic versions containing a hyphen (pre-releases - clause 10) or a
plus sign (builds - clause 11) are *not* compatible with this PEP
and are not permitted in the public version field.
One possible mechanism to translate such semantic versioning based build
One possible mechanism to translate such semantic versioning based source
labels to compatible public versions is to use the ``.devN`` suffix to
specify the appropriate version order.
@ -505,20 +584,6 @@ uniquely identify such releases for publication, while the source label is
used to record the original DVCS based version label.
Date based versions
~~~~~~~~~~~~~~~~~~~
As with other incompatible version schemes, date based versions can be
stored in the source label field. Translating them to a compliant
public version is straightforward: use a leading ``"0."`` prefix in the
public version label, with the date based version number as the remaining
components in the release segment.
This has the dual benefit of allowing subsequent migration to version
numbering based on API compatibility, as well as triggering more appropriate
version comparison semantics.
Olson database versioning
~~~~~~~~~~~~~~~~~~~~~~~~~
@ -526,13 +591,13 @@ The ``pytz`` project inherits its versioning scheme from the corresponding
Olson timezone database versioning scheme: the year followed by a lowercase
character indicating the version of the database within that year.
This can be translated to a compliant 3-part version identifier as
``0.<year>.<serial>``, where the serial starts at zero (for the '<year>a'
This can be translated to a compliant public version identifier as
``<year>.<serial>``, where the serial starts at zero (for the '<year>a'
release) and is incremented with each subsequent database update within the
year.
As with other translated version identifiers, the corresponding Olson
database version would be recorded in the source label field.
database version could be recorded in the source label field.
Version specifiers
@ -546,7 +611,8 @@ commas. For example::
The comparison operator (or lack thereof) determines the kind of version
clause:
* No operator: equivalent to ``~=``
* No operator: equivalent to ``>=`` for date based releases, and to ``~=``
otherwise
* ``~=``: `Compatible release`_ clause
* ``==``: `Version matching`_ clause
* ``!=``: `Version exclusion`_ clause
@ -578,6 +644,10 @@ to be compatible with the specified version.
The specified version identifier must be in the standard format described in
`Version scheme`_.
Automated tools SHOULD report an error when this operator is used in
conjunction with a date based version identifier, as it assumes the use
of semantic API versioning.
For a given release identifier ``V.N``, the compatible release clause is
approximately equivalent to the pair of comparison clauses::
@ -627,11 +697,16 @@ and a version identifier.
The specified version identifier must be in the standard format described in
`Version scheme`_, but a trailing ``.*`` is permitted as described below.
If the specified version identifier is a public version identifier (no
integrator suffix), then the integrator suffix of any candidate versions
MUST be ignored when matching versions.
By default, the version matching operator is based on a strict equality
comparison: the specified version must be exactly the same as the requested
version. The *only* substitution performed is the zero padding of the
release segment to ensure the release segments are compared with the same
length.
length (and similarly for the integrator suffix, if matching against a
specified local version identifier).
Whether or not strict version matching is appropriate depends on the specific
use case for the version specifier. Automated tools SHOULD at least issue
@ -670,6 +745,10 @@ The allowed version identifiers and comparison semantics are the same as
those of the `Version matching`_ operator, except that the sense of any
match is inverted.
If the specified version identifier is a public version identifier (no
integrator suffix), then the integrator suffix of any candidate versions
MUST be ignored when excluding versions.
For example, given the version ``1.1.post1``, the following clauses would
match or not as shown::
@ -692,6 +771,10 @@ The inclusive ordered comparison operators are ``<=`` and ``>=``.
As with version matching, the release segment is zero padded as necessary to
ensure the release segments are compared with the same length.
Local version identifiers are handled according to the combination of their
handling by the version matching operator and the consistent ordering
defined by the standard version scheme.
Exclusive ordered comparison
----------------------------
@ -711,6 +794,10 @@ The exclusive ordered comparison ``< V`` MUST NOT match a pre-release of
the given version, even if acceptance of pre-releases is enabled as
described in the section below.
Local version identifiers are handled according to the combination of their
handling by the version exclusion operator and the consistent ordering
defined by the standard version scheme.
Handling of pre-releases
------------------------
@ -788,13 +875,14 @@ Alternatively, a prebuilt archive may also be referenced::
pip (from file:///localbuilds/pip-1.3.1-py33-none-any.whl)
All direct references that do not refer to a local file URL SHOULD
specify a secure transport mechanism (such as ``https``), include an
expected hash value in the URL for verification purposes, or both. If an
insecure transport is specified without any hash information, with hash
information that the tool doesn't understand, or with a selected hash
algorithm that the tool considers too weak to trust, automated tools
SHOULD at least emit a warning and MAY refuse to rely on the URL.
All direct references that do not refer to a local file URL SHOULD specify
a secure transport mechanism (such as ``https``) AND include an expected
hash value in the URL for verification purposes. If a direct reference is
specified without any hash information, with hash information that the
tool doesn't understand, or with a selected hash algorithm that the tool
considers too weak to trust, automated tools SHOULD at least emit a warning
and MAY refuse to rely on the URL. If such a direct reference also uses an
insecure transport, automated tools SHOULD NOT rely on the URL.
It is RECOMMENDED that only hashes which are unconditionally provided by
the latest version of the standard library's ``hashlib`` module be used
@ -806,18 +894,33 @@ For source archive and wheel references, an expected hash value may be
specified by including a ``<hash-algorithm>=<expected-hash>`` entry as
part of the URL fragment.
Version control references, the ``VCS+protocol`` scheme SHOULD be
used to identify both the version control system and the secure transport.
For version control references, the ``VCS+protocol`` scheme SHOULD be
used to identify both the version control system and the secure transport,
and a version control system with hash based commit identifiers SHOULD be
used. Automated tools MAY omit warnings about missing hashes for version
control systems that do not provide hash based commit identifiers.
To support version control systems that do not support including commit or
To handle version control systems that do not support including commit or
tag references directly in the URL, that information may be appended to the
end of the URL using the ``@<tag>`` notation.
end of the URL using the ``@<commit-hash>`` or the ``@<tag>#<commit-hash>``
notation.
.. note::
This isn't *quite* the same as the existing VCS reference notation
supported by pip. Firstly, the distribution name is moved in front rather
than embedded as part of the URL. Secondly, the commit hash is included
even when retrieving based on a tag, in order to meet the requirement
above that *every* link should include a hash to make things harder to
forge (creating a malicious repo with a particular tag is easy, creating
one with a specific *hash*, less so).
Remote URL examples::
pip (from https://github.com/pypa/pip/archive/1.3.1.zip)
pip (from http://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686)
pip (from git+https://github.com/pypa/pip.git@1.3.1)
pip (from https://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686)
pip (from git+https://github.com/pypa/pip.git@7921be1537eac1e97bc40179a57f0349c2aee67d)
pip (from git+https://github.com/pypa/pip.git@1.3.1#7921be1537eac1e97bc40179a57f0349c2aee67d)
Updating the versioning specification
@ -840,7 +943,11 @@ Summary of differences from \PEP 386
on DVCS hashes
* Added the "direct reference" concept as a standard notation for direct
references to resources (rather than each tool needing to invents its own)
references to resources (rather than each tool needing to invent its own)
* Added the "local version identifier" and "integrator suffix" concepts to
allow system integrators to indicate patched builds in a way that is
supported by the upstream tools
* Added the "compatible release" clause
@ -853,7 +960,7 @@ Summary of differences from \PEP 386
* Explicit exclusion of leading or trailing whitespace
* Explicit criterion for the exclusion of date based versions
* Explicit support for date based versions
* Implicitly exclude pre-releases unless they're already present or
needed to satisfy a dependency
@ -1039,6 +1146,29 @@ and the desired pre- and post-release handling semantics for ``<`` and ``>``
ordered comparison clauses.
Support for date based version identifiers
------------------------------------------
Excluding date based versions caused significant problems in migrating
``pytz`` to the new metadata standards. It also caused concerns for the
OpenStack developers, as they use a date based versioning scheme and would
like to be able to migrate to the new metadata standards without changing
it.
The approach now adopted in the PEP is to:
* consider a leading release segment component greater than or equal to
``1980`` to denote a "date based release"
* using ``>=`` rather than ``~=`` as the default comparison operator for
version specifier clauses based on a date based release
* recommend reporting an error if ``~=`` is used with a date based release
This approach means that date based version identifiers should "just work"
for ``pytz`` and any other projects with stable APIs, and at least be usable
(through the use of appropriate version specifiers on the consumer side) for
projects with less stable APIs.
Adding direct references
------------------------
@ -1057,6 +1187,38 @@ external services have the effect of slowing down installation operations,
as well as reducing PyPI's own apparent reliability.
Adding local version identifiers
--------------------------------
It's a fact of life that downstream integrators often need to backport
upstream bug fixes to older versions. It's one of the services that gets
Linux distro vendors paid, and application developers may also apply patches
they need to bundled dependencies.
Historically, this practice has been invisible to cross-platform language
specific distribution tools - the reported "version" in the upstream
metadata is the same as for the unmodified code. This inaccuracy then
can then cause problems when attempting to work with a mixture of integrator
provided code and unmodified upstream code, or even just attempting to
identify exactly which version of the software is installed.
The introduction of local version identifiers and the "integrator suffix"
into the versioning scheme, with the corresponding ``python.integrator``
metadata extension allows this kind of activity to be represented
accurately, which should improve interoperability between the upstream
tools and various integrated platforms.
The exact scheme chosen is largely modelled on the existing behaviour of
``pkg_resources.parse_version`` and ``pkg_resources.parse_requirements``,
with the main distinction being that where ``pkg_resources`` currently always
takes the suffix into account when comparing versions for exact matches,
the PEP requires that the integrator suffix of the candidate version be
ignored when no integrator suffix is present in the version specifier clause.
This change is designed to ensure that an integrator provided version like
``pip 1.5-1`` will still satisfy a version specifier like ``pip (== 1.1)``.
References
==========

View File

@ -10,7 +10,7 @@ Type: Standards Track
Content-Type: text/x-rst
Requires: 426
Created: 11 Nov 2013
Post-History:
Post-History: 21 Dec 2013
Abstract
@ -49,6 +49,10 @@ The currently defined standard extensions are:
* ``python.exports``
* ``python.metadata_hooks``
All standard extensions are currently at version ``1.0``, and thus the
``extension_metadata`` field may be omitted without losing access to any
functionality.
The ``details`` extension
=========================
@ -502,9 +506,9 @@ previously installed, distribution.
Example::
"commands": {
"wrap_console": [{"wrapwithpython": "chair:run_cli"}],
"wrap_gui": [{"wrapwithpythonw": "chair:run_gui"}],
"prebuilt": ["notawrapper"]
"wrap_console": [{"chair": "chair:run_cli"}],
"wrap_gui": [{"chair-gui": "chair:run_gui"}],
"prebuilt": ["reduniforms"]
}
@ -532,18 +536,18 @@ Hook signatures
The currently defined metadata hooks are:
* ``install``: run after a relevant distribution has been installed,
* ``postinstall``: run after a relevant distribution has been installed,
upgraded or downgraded on the current system. May also be run as part
of a system state resync operation. If the hook is not defined, it
indicates no distribution specific actions are needed following
installation.
* ``uninstall``: run after a relevant distribution has been completely
* ``postuninstall``: run after a relevant distribution has been completely
removed from the current system. If the hook is not defined, it indicates
no distribution specific actions are needed following uninstallation.
The required signatures of these hooks are as follows::
def install(current_meta, previous_meta=None):
def postinstall(current_meta, previous_meta=None):
"""Run following installation or upgrade of a relevant distribution
*current_meta* is the distribution metadata for the version now
@ -554,7 +558,7 @@ The required signatures of these hooks are as follows::
resynchronisation of the system state).
"""
def uninstall(previous_meta):
def postuninstall(previous_meta):
"""Run after complete uninstallation of a relevant distribution
*previous_meta* is the distribution metadata for the version that
@ -568,8 +572,8 @@ Export group hooks are named after the export group of interest::
"export_groups": {
"ComfyChair.plugins": {
"install": "ComfyChair.plugins:install_hook",
"uininstall": "ComfyChair.plugins:uninstall_hook"
"postinstall": "ComfyChair.plugins:install_hook",
"postuininstall": "ComfyChair.plugins:uninstall_hook"
}
}
@ -588,16 +592,16 @@ Extension hooks are named after the metadata extension of interest::
"extensions": {
"python.exports": {
"install": "pip.export_group_hooks:run_install_hooks",
"uininstall": "pip.export_group_hooks:run_uninstall_hooks"
"postinstall": "pip.export_group_hooks:run_install_hooks",
"postuininstall": "pip.export_group_hooks:run_uninstall_hooks"
}
"python.commands": {
"install": "pip.command_hook:install_wrapper_scripts",
"postinstall": "pip.command_hook:install_wrapper_scripts",
}
}
(Note: this is just an example, but the intent is that pip *could* implement
that functionality that way if it wanted to.
that functionality that way if it wanted to).
A trailing ".*" may be used to request prefix matching rather than
requiring an exact match on the extension name.
@ -610,7 +614,8 @@ Guidelines for metadata hook invocation
Metadata hooks are likely to run with elevated privileges, this needs
to be considered carefully (e.g. by *requiring* that metadata hook
installation be opt in).
installation be opt in when using the standard tools and running with
elevated privileges).
The given parameter names are considered part of the hook signature.
Installation tools MUST call metadata hooks solely with keyword arguments.