From f659c90323ba8aa97afd33b5cba26db656f0be74 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Fri, 8 Aug 2014 20:58:55 +1000 Subject: [PATCH] Update PEP 440 based on pip proof of concept --- pep-0440.txt | 195 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 121 insertions(+), 74 deletions(-) diff --git a/pep-0440.txt b/pep-0440.txt index ee1277838..4d29da1a6 100644 --- a/pep-0440.txt +++ b/pep-0440.txt @@ -11,7 +11,7 @@ Type: Standards Track Content-Type: text/x-rst Created: 18 Mar 2013 Post-History: 30 Mar 2013, 27 May 2013, 20 Jun 2013, - 21 Dec 2013, 28 Jan 2014 + 21 Dec 2013, 28 Jan 2014, 08 Aug 2014 Replaces: 386 @@ -24,13 +24,6 @@ distributions, and declaring dependencies on particular versions. This document addresses several limitations of the previous attempt at a standardized approach to versioning, as described in PEP 345 and PEP 386. -.. note:: - - This PEP was broken out of the metadata 2.0 specification in PEP 426. - - Unlike PEP 426, the notes that remain in this document are intended as - part of the final specification (except for this one). - Definitions =========== @@ -409,20 +402,22 @@ such as ``1.0+foo0100`` which is already in its normalized form. Pre-release separators ~~~~~~~~~~~~~~~~~~~~~~ -Pre-releases should allow either a ``.`` or a ``-`` separator between the +Pre-releases should allow a ``.``, ``-``, or ``_`` separator between the release segment and the pre-release segment. The normal form for this is without a separator. This allows versions such as ``1.1.a1`` or ``1.1-a1`` -which would be normalized to ``1.1a1``. +which would be normalized to ``1.1a1``. It should also allow a seperator to +be used between the pre-release signifier and the numeral. This allows versions +such as ``1.0a.1`` which would be normalized to ``1.0a1``. Pre-release spelling ~~~~~~~~~~~~~~~~~~~~ -Pre-releases allow the additional spellings of alpha, beta, and rc for a, b, -and c respectively. This allows versions such as ``1.1alpha1``, ``1.1beta2``, -or ``1.1rc3`` which normalize to ``1.1a1``, ``1.1b2``, and ``1.1c3``. In every -case the additional spelling should be considered equivalent to their normal -forms. +Pre-releases allow the additional spellings of ``alpha``, ``beta``, ``rc``, +``pre``, and ``preview`` for ``a``, ``b``, ``c``, ``c``, and ``c`` respectively. +This allows versions such as ``1.1alpha1``, ``1.1beta2``, or ``1.1rc3`` which +normalize to ``1.1a1``, ``1.1b2``, and ``1.1c3``. In every case the additional +spelling should be considered equivalent to their normal forms. Implicit pre-release number @@ -436,10 +431,21 @@ allows versions such as ``1.2a`` which is normalized to ``1.2a0``. Post release separators ~~~~~~~~~~~~~~~~~~~~~~~ -Post releases allow either a ``.`` or a ``-`` separator as well as omitting the +Post releases allow a ``.``,``-``, or ``_`` separator as well as omitting the separator all together. The normal form of this is with the ``.`` separator. This allows versions such as ``1.2-post2`` or ``1.2post2`` which normalize to -``1.2.post2``. +``1.2.post2``. Like the pre-release seperator this also allows an optional +separator between the post release signifier and the numeral. This allows +versions like ``1.2.post-2`` which would normalize to ``1.2.post2``. + + +Post release spelling +~~~~~~~~~~~~~~~~~~~~~ + +Post-releases allow the additional spellings of ``rev`` and ``r``. This allows +versions such as ``1.0-r4`` which normalizes to ``1.0.post4``. As with the +pre-releases the additional spellings should be considered equivalent to their +normal forms. Implicit post release number @@ -450,10 +456,21 @@ to be ``0``. The normal form for this is to include the ``0`` explicitly. This allows versions such as ``1.2.post`` which is normalized to ``1.2.post0``. +Implicit post releases +~~~~~~~~~~~~~~~~~~~~~~ + +Post releases allow omitting the ``post`` signifier all together. When using +this form the separator MUST be ``-`` and no other form is allowed. This allows +versions such as ``1.0-1`` to be normalized to ``1.0.post1``. This particular +normalization MUST NOT be used in conjunction with the implicit post release +number rule. In other words ``1.0-`` is *not* a valid version and it does *not* +normalize to ``1.0.post0``. + + Development release separators ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Development releases allow either a ``.`` or a ``-`` separator as well as +Development releases allow a ``.``, ``-``, or a ``_`` separator as well as omitting the separator all together. The normal form of this is with the ``.`` separator. This allows versions such as ``1.2-dev2`` or ``1.2dev2`` which normalize to ``1.2.dev2``. @@ -468,6 +485,33 @@ explicitly. This allows versions such as ``1.2.dev`` which is normalized to ``1.2.dev0``. +Local version segments +~~~~~~~~~~~~~~~~~~~~~~ + +With a local version, in addition to the use of ``.`` as a separator of +segments, the use of ``-`` and ``_`` is also acceptable. The normal form is +using the ``.`` character. This allows versions such as ``1.0+ubuntu-1`` to be +normalized to ``1.0+ubuntu.1``. + + +Preceding v character +~~~~~~~~~~~~~~~~~~~~~ + +In order to support the common version notation of ``v1.0`` versions may be +preceded by a single literal ``v`` character. This character MUST be ignored +for all purposes and should be omitted from all normalized forms of the +version. The same version with and without the ``v`` is considered equivalent. + + +Leading and Trailing Whitespace +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Leading and trailing whitespace must be silently ignored and removed from all +normalized forms of a version. This includes ``" "``, ``\t``, ``\n``, ``\r``, +``\f``, and ``\v``. This allows accidental whitespace to be handled sensibly, +such as a version like ``1.0\n`` which normalizes to ``1.0``. + + Examples of compliant version schemes ------------------------------------- @@ -1098,10 +1142,6 @@ Summary of differences from \PEP 386 * Moved the description of version specifiers into the versioning PEP -* Added the "source label" concept to better handle projects that wish to - use a non-compliant versioning scheme internally, especially those based - on DVCS hashes - * Added the "direct reference" concept as a standard notation for direct references to resources (rather than each tool needing to invent its own) @@ -1123,6 +1163,10 @@ Summary of differences from \PEP 386 * Explicit support for date based versions +* Explicit normalisation rules to improve compatibility with + existing version metadata on PyPI where it doesn't introduce + ambiguity + * Implicitly exclude pre-releases unless they're already present or needed to satisfy a dependency @@ -1133,11 +1177,10 @@ Summary of differences from \PEP 386 The rationale for major changes is given in the following sections. - Changing the version scheme --------------------------- -The key change in the version scheme in this PEP relative to that in +One key change in the version scheme in this PEP relative to that in PEP 386 is to sort top level developmental releases like ``X.Y.devN`` ahead of alpha releases like ``X.Ya1``. This is a far more logical sort order, as projects already using both development releases and alphas/betas/release @@ -1165,51 +1208,17 @@ The exclusion of leading and trailing whitespace was made explicit after a couple of projects with version identifiers differing only in a trailing ``\n`` character were found on PyPI. +Various other normalisation rules were also added as described in the +separate section on version normalisation below. + `Appendix A` shows detailed results of an analysis of PyPI distribution -version information, as collected on 19th February, 2013. This analysis -compares the behavior of the explicitly ordered version schemes defined in -this PEP and PEP 386 with the de facto standard defined by the behavior -of setuptools. These metrics are useful, as the intent of both PEPs is to -follow existing setuptools behavior as closely as is feasible, while -still throwing exceptions for unorderable versions (rather than trying -to guess an appropriate order as setuptools does). - -Overall, the percentage of compatible distributions improves from 97.7% -with PEP 386 to 98.7% with this PEP. While the number of projects affected -in practice was small, some of the affected projects are in widespread use -(such as Pinax and selenium). The surprising ordering discrepancy also -concerned developers and acted as an unnecessary barrier to adoption of -the new metadata standard, even for projects that weren't directly affected. - -The data also shows that the pre-release sorting discrepancies are seen -only when analyzing *all* versions from PyPI, rather than when analyzing -public versions. This is largely due to the fact that PyPI normally reports -only the most recent version for each project (unless maintainers -explicitly configure their project to display additional versions). However, -installers that need to satisfy detailed version constraints often need -to look at all available versions, as they may need to retrieve an older -release. - -Even this PEP doesn't completely eliminate the sorting differences relative -to setuptools: - -* Sorts differently (after translations): 38 / 28194 (0.13 %) -* Sorts differently (no translations): 2 / 28194 (0.01 %) - -The two remaining sort order discrepancies picked up by the analysis are due -to a pair of projects which have PyPI releases ending with a carriage -return, alongside releases with the same version number, only *without* the -trailing carriage return. - -The sorting discrepancies after translation relate mainly to differences -in the handling of pre-releases where the standard mechanism is considered -to be an improvement. For example, the existing pkg_resources scheme will -sort "1.1beta1" *after* "1.1b2", whereas the suggested standard translation -for "1.1beta1" is "1.1b1", which sorts *before* "1.1b2". Similarly, the -pkg_resources scheme will sort "-dev-N" pre-releases differently from -"devN" pre-releases when they occur within the same release, while the -scheme in this PEP requires normalizing both representations to ".devN" and -sorting them by the numeric component. +version information, as collected on 8th August, 2014. This analysis +compares the behavior of the explicitly ordered version scheme defined in +this PEP with the de facto standard defined by the behavior of setuptools. +These metrics are useful, as the intent of this PEP is to follow existing +setuptools behavior as closely as is feasible, while still throwing +exceptions for unorderable versions (rather than trying to guess an +appropriate order as setuptools does). A more opinionated description of the versioning scheme @@ -1381,7 +1390,42 @@ It was chosen instead of the hyphen to prevent ``pkg_resources.parse_version`` from parsing it as a prerelease, which is important for enabling a successful migration to the new, more structured, versioning scheme. The plus was chosen instead of a tilde because of the -significance of the tilde in Debian's version algorithm. +significance of the tilde in Debian's version ordering algorithm. + + +Providing explicit version normalization rules +---------------------------------------------- + +Historically, the de facto standard for parsing versions in Python has been the +``pkg_resources.parse_version`` command from the setuptools project. This does +not attempt to reject *any* version and instead tries to make something +meaningful, with varying levels of success, out of whatever it is given. It has +a few simple rules but otherwise it more or less relies largely on string +comparison. + +The normalization rules provided in this PEP exist primarily to either increase +the compatability with ``pkg_resources.parse_version``, particularly in +documented use cases such as ``rev``, ``r``, ``pre``, etc or to do something +more reasonable with versions that already exist on PyPI. + +All possible normalization rules were weighed against whether or not they were +*likely* to cause any ambiguity (e.g. while someone might devise a scheme where +``v1.0`` and ``1.0`` are considered distinct releases, the likelihood of anyone +actually doing that, much less on any scale that is noticeable, is fairly low). +They were also weighed against how ``pkg_resources.parse_version`` treated a +particular version string, especially with regards to how it was sorted. Finally +each rule was weighed against the kinds of additional versions it allowed, how +"ugly" those versions looked, how hard there were to parse (both mentally and +mechanically) and how much additional compatibility it would bring. + +The breadth of possible normalizations were kept to things that could easily +be implemented as part of the parsing of the version and not pre-parsing +transformations applied to the versions. This was done to limit the side +effects of each transformation as simple search and replace style transforms +increase the likelihood of ambiguous or "junk" versions. + +For an extended discussion on the various types of normalizations that were +considered, please see the proof of concept for PEP 440 within pip [4]_. References @@ -1399,6 +1443,9 @@ justifications for needing such a standard can be found in PEP 386. .. [3] Pessimistic version constraint http://docs.rubygems.org/read/chapter/16 +.. [4] Proof of Concept: PEP 440 within pip + https://github.com/pypa/pip/pull/1894 + Appendix A ========== @@ -1406,11 +1453,11 @@ Appendix A Metadata v2.0 guidelines versus setuptools:: $ invoke check.pep440 - Total Version Compatibility: 231807/239450 (96.81%) - Total Sorting Compatibility (Unfiltered): 43095/45505 (94.70%) - Total Sorting Compatibility (Filtered): 45481/45505 (99.95%) - Projects with No Compatible Versions: 802/45505 (1.76%) - Projects with Differing Latest Version: 1163/45505 (2.56%) + Total Version Compatibility: 245806/250521 (98.12%) + Total Sorting Compatibility (Unfiltered): 45441/47114 (96.45%) + Total Sorting Compatibility (Filtered): 47057/47114 (99.88%) + Projects with No Compatible Versions: 498/47114 (1.06%) + Projects with Differing Latest Version: 688/47114 (1.46%) Copyright