python-peps/pep-0440.txt

PEP: 440
Title: Version Identification and Dependency Specification
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>
BDFL-Delegate: Nick Coghlan <ncoghlan@gmail.com>
Discussions-To: Distutils SIG <distutils-sig@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 18 Mar 2013
Post-History: 30 Mar 2013
Replaces: 386


Abstract
========

This PEP describes a scheme for identifying versions of Python
software distributions, and declaring dependencies on particular
versions.

This document addresses several limitations of the previous attempt at
a standardised approach to versioning, as described in PEP 345 and PEP
386.

.. note::

   This PEP has been broken out of the metadata 2.0 specification in
   PEP 426 and refers to some details that will be in the *next*
   version of PEP 426.


Definitions
===========

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.

"Distributions" are deployable software components published through
an index server or otherwise made available for installation.

"Versions" are uniquely identified snapshots of a distribution.

"Distribution archives" are the packaged files which are used to
publish and distribute the software.  "Source archives" require a
build system to be available on the target system, while "binary
archives" only require that prebuilt files be moved to the correct
location on the target system.  As Python is a dynamically bound
cross-platform language, many "binary" archives will contain only pure
Python source code.

"Build tools" are automated tools intended to run on development
systems, producing source and binary distribution archives.  Build
tools may also be invoked by installation tools in order to install
software distributed as source archives rather than prebuilt binary
archives.

"Index servers" are active distribution registries which publish
version and dependency metadata and place constraints on the permitted
metadata.

"Publication tools" are automated tools intended to run on development
systems and upload source and binary distribution archives to index
servers.

"Installation tools" are automated tools intended to run on production
systems, consuming source and binary distribution archives from an
index server or other designated location and deploying them to the
target system.

"Automated tools" is a collective term covering build tools, index
servers, publication tools, installation tools and any other software
that produces or consumes distribution version and dependency
metadata.

"Projects" refers to the developers that manage the creation of a
particular distribution.


Version scheme
==============

Distribution versions are identified by both a public version
identifier, which supports all defined version comparison operations,
and a build label, which supports only strict equality comparisons.

The version scheme is used both to describe the distribution version
provided by a particular distribution archive, as well as to place
constraints on the version of dependencies needed in order to build or
run the software.


Public version identifiers
--------------------------

Public version identifiers MUST comply with the following scheme::

    N[.N]+[{a|b|c|rc}N][.postN][.devN]

Public version identifiers MUST NOT include leading or trailing
whitespace.

Public version identifiers MUST be unique within a given distribution.

Installation tools SHOULD ignore any public versions which do not
comply with this scheme.  Installation tools MAY warn the user when
non-compliant or ambiguous versions are detected.

Public version identifiers are separated into up to four segments:

* Release segment: ``N[.N]+``
* Pre-release segment: ``{a|b|c|rc}N``
* Post-release segment: ``.postN``
* Development release segment: ``.devN``

Any given version will be a "release", "pre-release", "post-release"
or "developmental release" as defined in the following sections.

.. note::

   Some hard to read version identifiers are permitted by this scheme
   in order to better accommodate the wide range of versioning
   practices across existing public and private Python projects, given
   the constraint that the package index is not yet sophisticated
   enough to allow the introduction of a simpler,
   backwards-incompatible scheme.

   Accordingly, some of the versioning practices which are technically
   permitted by the PEP are strongly discouraged for new projects.
   Where this is the case, the relevant details are noted in the
   following sections.


Build labels
------------

Build labels are text strings with minimal defined semantics.

To ensure build labels can be readily incorporated in file names and
URLs, they MUST be comprised of only ASCII alphanumerics, plus signs,
periods and hyphens.

In addition, build labels MUST be unique within a given distribution.


Releases
--------

A version identifier that consists solely of a release segment is
termed a "release".

The release segment consists of one or more non-negative integer
values, separated by dots::

    N[.N]+

Releases within a project will typically be numbered in a consistently
increasing fashion.

Comparison and ordering of release segments considers the numeric
value of each component of the release segment in turn.  When
comparing release segments with different numbers of components, the
shorter segment is padded out with additional zeroes as necessary.

Date based release numbers are technically compatible with this
scheme, but their use is not consistent with the expected API
versioning semantics described below.  Accordingly, automated tools
SHOULD at least issue a warning when encountering a leading release
component greater than or equal to ``1980`` and MAY treat this case as
an error.

While any number of additional components after the first are
permitted under this scheme, the most common variants are to use two
components ("major.minor") or three components ("major.minor.micro").

For example::

    0.9
    0.9.1
    0.9.2
    ...
    0.9.10
    0.9.11
    1.0
    1.0.1
    1.1
    2.0
    2.0.1

A release series is any set of release numbers that start with a
common prefix. For example, ``3.3.1``, ``3.3.5`` and ``3.3.9.45`` are
all part of the ``3.3`` release series.

.. note::

   ``X.Y`` and ``X.Y.0`` are not considered distinct release numbers,
   as the release segment comparison rules implicit expand the two
   component form to ``X.Y.0`` when comparing it to any release
   segment that includes three components.


Pre-releases
------------

Some projects use an "alpha, beta, release candidate" pre-release
cycle to support testing by their users prior to a full release.

If used as part of a project's development cycle, these pre-releases
are indicated by including a pre-release segment in the version
identifier::

    X.YaN  # Alpha release
    X.YbN  # Beta release
    X.YcN  # Release candidate (alternative notation: X.YrcN)
    X.Y    # Full release

A version identifier that consists solely of a release segment and a
pre-release segment is termed a "pre-release".

The pre-release segment consists of an alphabetical identifier for the
pre-release phase, along with a non-negative integer value.
Pre-releases for a given release are ordered first by phase (alpha,
beta, release candidate) and then by the numerical component within
that phase.

Build tools, publication tools and index servers SHOULD disallow the
creation of both ``c`` and ``rc`` releases for a common release
segment, but this may need to be tolerated in order to handle some
existing legacy distributions.

Installation tools SHOULD interpret all ``rc`` versions as coming
after all ``c`` versions (that is, ``rc1`` indicates a later version
than ``c2``).  Installation tools MAY warn the user when such
ambiguous versions are detected, or even reject them entirely.


Post-releases
-------------

Some projects use post-releases to address minor errors in a release
that do not affect the distributed software (for example, correcting
an error in the release notes).

If used as part of a project's development cycle, these post-releases
are indicated by including a post-release segment in the version
identifier::

    X.Y.postN    # Post-release

A version identifier that includes a post-release segment without a
developmental release segment is termed a "post-release".

The post-release segment consists of the string ``.post``, followed by
a non-negative integer value.  Post-releases are ordered by their
numerical component, immediately following the corresponding release,
and ahead of any subsequent release.

.. note::

   The use of post-releases to publish maintenance releases containing
   actual bug fixes is strongly discouraged.  In general, it is better
   to use a longer release number and increment the final component
   for each maintenance release.

Post-releases are also permitted for pre-releases::

    X.YaN.postM  # Post-release of an alpha release
    X.YbN.postM  # Post-release of a beta release
    X.YcN.postM  # Post-release of a release candidate

.. note::

   Creating post-releases of pre-releases is strongly discouraged, as
   it makes the version identifier difficult to parse for human
   readers.  In general, it is substantially clearer to simply create
   a new pre-release by incrementing the numeric component.


Developmental releases
----------------------

Some projects make regular developmental releases, and system
packagers (especially for Linux distributions) may wish to create
early releases directly from source control which do not conflict with
later project releases.

If used as part of a project's development cycle, these developmental
releases are indicated by including a developmental release segment in
the version identifier::

    X.Y.devN    # Developmental release

A version identifier that includes a developmental release segment is
termed a "developmental release".

The developmental release segment consists of the string ``.dev``,
followed by a non-negative integer value.  Developmental releases are
ordered by their numerical component, immediately before the
corresponding release (and before any pre-releases with the same
release segment), and following any previous release (including any
post-releases).

Developmental releases are also permitted for pre-releases and
post-releases::

    X.YaN.devM      # Developmental release of an alpha release
    X.YbN.devM      # Developmental release of a beta release
    X.YcN.devM      # Developmental release of a release candidate
    X.Y.postN.devM  # Developmental release of a post-release

.. note::

   Creating developmental releases of pre-releases is strongly
   discouraged, as it makes the version identifier difficult to parse
   for human readers.  In general, it is substantially clearer to
   simply create additional pre-releases by incrementing the numeric
   component.

   Developmental releases of post-releases are also strongly
   discouraged, but they may be appropriate for projects which use the
   post-release notation for full maintenance releases which may
   include code changes.


Examples of compliant version schemes
-------------------------------------

The standard version scheme is designed to encompass a wide range of
identification practices across public and private Python projects.
In practice, a single project attempting to use the full flexibility
offered by the scheme would create a situation where human users had
difficulty figuring out the relative order of versions, even though
the rules above ensure all compliant tools will order them
consistently.

The following examples illustrate a small selection of the different
approaches projects may choose to identify their releases, while still
ensuring that the "latest release" and the "latest stable release" can
be easily determined, both by human users and automated tools.

Simple "major.minor" versioning::

    0.1
    0.2
    0.3
    1.0
    1.1
    ...

Simple "major.minor.micro" versioning::

    1.1.0
    1.1.1
    1.1.2
    1.2.0
    ...

"major.minor" versioning with alpha, beta and release candidate
pre-releases::

    0.9
    1.0a1
    1.0a2
    1.0b1
    1.0c1
    1.0
    1.1a1
    ...

"major.minor" versioning with developmental releases, release
candidates and post-releases for minor corrections::

    0.9
    1.0.dev1
    1.0.dev2
    1.0.dev3
    1.0.dev4
    1.0rc1
    1.0rc2
    1.0
    1.0.post1
    1.1.dev1
    ...


Summary of permitted suffixes and relative ordering
---------------------------------------------------

.. note::

   This section is intended primarily for authors of tools that
   automatically process distribution metadata, rather than developers
   of Python distributions deciding on a versioning scheme.

The release segment of version identifiers MUST be sorted in the same
order as Python's tuple sorting when the release segment is parsed as
follows::

    tuple(map(int, release_segment.split(".")))

All release segments involved in the comparison MUST be converted to a
consistent length by padding shorter segments with zeroes as needed.

Within a numeric release (``1.0``, ``2.7.3``), the following suffixes
are permitted and MUST be ordered as shown::

   .devN, aN, bN, cN, rcN, <no suffix>, .postN

Note that `rc` will always sort after `c` (regardless of the numeric
component) although they are semantically equivalent.  Tools are free
to reject this case as ambiguous and remain in compliance with the
PEP.

Within an alpha (``1.0a1``), beta (``1.0b1``), or release candidate
(``1.0c1``, ``1.0rc1``), the following suffixes are permitted and MUST
be ordered as shown::

   .devN, <no suffix>, .postN

Within a post-release (``1.0.post1``), the following suffixes are
permitted and MUST be ordered as shown::

    .devN, <no suffix>

Note that ``devN`` and ``postN`` MUST always be preceded by a dot,
even when used immediately following a numeric version
(e.g. ``1.0.dev456``, ``1.0.post1``).

Within a pre-release, post-release or development release segment with
a shared prefix, ordering MUST be by the value of the numeric
component.

The following example covers many of the possible combinations::

    1.0.dev456
    1.0a1
    1.0a2.dev456
    1.0a12.dev456
    1.0a12
    1.0b1.dev456
    1.0b2
    1.0b2.post345.dev456
    1.0b2.post345
    1.0c1.dev456
    1.0c1
    1.0
    1.0.post456.dev34
    1.0.post456
    1.1.dev1


Version ordering across different metadata versions
---------------------------------------------------

Metadata v1.0 (PEP 241) and metadata v1.1 (PEP 314) do not specify a
standard version identification or ordering scheme.  This PEP does not
mandate any particular approach to handling such versions, but
acknowledges that the de facto standard for ordering them is the
scheme used by the ``pkg_resources`` component of ``setuptools``.

Software that automatically processes distribution metadata SHOULD
attempt to normalize non-compliant version identifiers to the standard
scheme, and ignore them if normalization fails.  As any normalization
scheme will be implementation specific, this means that projects using
non-compliant version identifiers may not be handled consistently
across different tools, even when correctly publishing the earlier
metadata versions.

For distributions currently using non-compliant version identifiers,
these filtering guidelines mean that it should be enough for the
project to simply switch to the use of compliant version identifiers
to ensure consistent handling by automated tools.

Distribution users may wish to explicitly remove non-compliant
versions from any private package indexes they control.

For metadata v1.2 (PEP 345), the version ordering described in this
PEP SHOULD be used in preference to the one defined in PEP 386.


Compatibility with other version schemes
----------------------------------------

Some projects may choose to use a version scheme which requires
translation in order to comply with the public version scheme defined
in this PEP.  In such cases, the build label can be used to record the
project specific version as an arbitrary label, while the translated
public version is published in the version field.

This allows automated distribution tools to provide consistently
correct ordering of published releases, while still allowing
developers to use the internal versioning scheme they prefer for their
projects.


Semantic versioning
~~~~~~~~~~~~~~~~~~~

`Semantic versioning`_ is a popular version identification scheme that
is more prescriptive than this PEP regarding the significance of
different elements of a release number.  Even if a project chooses not
to abide by the details of semantic versioning, the scheme is worth
understanding as it covers many of the issues that can arise when
depending on other distributions, and when publishing a distribution
that others rely on.

The "Major.Minor.Patch" (described in this PEP as "major.minor.micro")
aspects of semantic versioning (clauses 1-9 in the 2.0.0-rc-1
specification) are fully compatible with the version scheme defined in
this PEP, and abiding by these aspects is encouraged.

Semantic versions containing a hyphen (pre-releases - clause 10) or a
plus sign (builds - clause 11) are *not* compatible with this PEP and
are not permitted in the public version field.

One possible mechanism to translate such semantic versioning based
build labels to compatible public versions is to use the ``.devN``
suffix to specify the appropriate version order.

.. _Semantic versioning: http://semver.org/


DVCS based version labels
~~~~~~~~~~~~~~~~~~~~~~~~~

Many build tools integrate with distributed version control systems
like Git and Mercurial in order to add an identifying hash to the
version identifier.  As hashes cannot be ordered reliably such
versions are not permitted in the public version field.

As with semantic versioning, the public ``.devN`` suffix may be used
to uniquely identify such releases for publication, while the build
label is used to record the original DVCS based version label.


Date based versions
~~~~~~~~~~~~~~~~~~~

As with other incompatible version schemes, date based versions can be
stored in the build label field.  Translating them to a compliant
public version is straightforward: use a leading "0." prefix in the
public version label, with the date based version number as the
remaining components in the release segment.

This has the dual benefit of allowing subsequent migration to version
numbering based on API compatibility, as well as triggering more
appropriate version comparison semantics.


Version specifiers
==================

A version specifier consists of a series of version clauses, separated
by commas.  For example::

   0.9, >= 1.0, != 1.3.4.*, < 2.0

The comparison operator (or lack thereof) determines the kind of
version clause:
    
* No operator: `Compatible release`_ clause
* ``==``: `Version matching`_ clause
* ``!=``: `Version exclusion`_ clause
* ``is``: `Build reference`_ clause
* ``<``, ``>``, ``<=``, ``>=``: `Ordered comparison`_ clause

The comma (",") is equivalent to a logical **and** operator: a
candidate version must match all given version clauses in order to
match the specifier as a whole.

Whitespace between a conditional operator and the following version
identifier is optional, as is the whitespace around the commas.

When multiple candidate versions match a version specifier, the
preferred version SHOULD be the latest version as determined by the
consistent ordering defined by the standard `Version scheme`_.
Whether or not pre-releases are considered as candidate versions
SHOULD be handled as described in `Handling of pre-releases`_.


Compatible release
------------------

A compatible release clause consists solely of a version identifier
without any comparison operator.  It matches any candidate version
that is expected to be compatible with the specified version.

The specified version identifier must be in the standard format
described in `Version scheme`_.

For a given release identifier ``V.N``, the compatible release clause
is approximately equivalent to the pair of comparison clauses::

    >= V.N, == V.*

For example, the following version clauses are equivalent::

    2.2
    >= 2.2, == 2.*

    1.4.5
    >= 1.4.5, == 1.4.*

If a pre-release, post-release or developmental release is named in a
compatible release clause as ``V.N.suffix``, then the suffix is
ignored when determining the required prefix match::

    2.2.post3
    >= 2.2.post3, == 2.*

    1.4.5a4
    >= 1.4.5a4, == 1.4.*

The padding rules for release segment comparisons means that the
assumed degree of forward compatibility in a compatible release clause
can be controlled by appending additional zeroes to the version
specifier::

    2.2.0
    >= 2.2.0, == 2.2.*

    1.4.5.0
    >= 1.4.5.0, == 1.4.5.*


Version matching
----------------

A version matching clause includes the version matching operator
``==`` and a version identifier.

The specified version identifier must be in the standard format
described in `Version scheme`_, but a trailing ``.*`` is permitted as
described below.

By default, the version matching operator is based on a strict
equality comparison: the specified version must be exactly the same as
the requested version.  The *only* substitution performed is the zero
padding of the release segment to ensure the release segments are
compared with the same length.

Prefix matching may be requested instead of strict comparison, by
appending a trailing ``.*`` to the version identifier in the version
matching clause.  This means that additional trailing segments will be
ignored when determining whether or not a version identifier matches
the clause.  If the version includes only a release segment, than
trailing components in the release segment are also ignored.

For example, given the version ``1.1.post1``, the following clauses would
match or not as shown:
    
    == 1.1        # Not equal, so 1.1.post1 does not match clause
    == 1.1.post1  # Equal, so 1.1.post1 matches clause
    == 1.1.*      # Same prefix, so 1.1.post1 matches clause

.. note::

   The use of ``==`` when defining dependencies for published
   distributions is strongly discouraged as it greatly complicates the
   deployment of security fixes.  The strict version comparison
   operator is intended primarily for use when defining dependencies
   for repeatable *deployments of applications* while using a shared
   distribution index.


Version exclusion
-----------------

A version exclusion clause includes the version matching operator
``!=`` and a version identifier.

The allowed version identifiers and comparison semantics are the same
as those of the `Version matching`_ operator, except that the sense of
any match is inverted.

For example, given the version ``1.1.post1``, the following clauses
would match or not as shown:
    
    != 1.1        # Not equal, so 1.1.post1 matches clause
    != 1.1.post1  # Equal, so 1.1.post1 does not match clause
    != 1.1.*      # Same prefix, so 1.1.post1 does not match clause


Build reference
---------------

A build reference includes the build label matching operator ``is`` and
a build reference.

A build reference is a direct URI reference supplied to satisfy a
dependency.  The exact kinds of URIs and targets supported will be
determined by the specific installation tool used.

Publication tools and public index servers SHOULD NOT permit build
references in dependency specifications.

Installation tools SHOULD support the use of build references to
identify dependencies.

Automated tools MAY support the use of build labels in build reference
clauses.  They can be clearly distinguished from URI references
without ambiguity, as ``:`` and ``/`` are not permitted in build
labels.

Build label matching works solely on strict equality comparisons: the
candidate build label must be exactly the same as the build label in
the version clause.


Ordered comparison
------------------

An ordered comparison clause includes a comparison operator and a
version identifier, and will match any version where the comparison is
correct based on the relative position of the candidate version and
the specified version given the consistent ordering defined by the
standard `Version scheme`_.

The supported ordered comparison operators are ``<``, ``>``, ``<=``,
``>=``.

As with version matching, the release segment is zero padded as
necessary to ensure the release segments are compared with the same
length.

To exclude pre-releases and post-releases correctly, the comparison
clauses ``< V`` and ``> V`` MUST be interpreted as also implying the
version matching clause ``!= V.*``.


Handling of pre-releases
------------------------

Pre-releases of any kind, including developmental releases, are
implicitly excluded from all version specifiers, *unless* a
pre-release or developmental release is explicitly mentioned in one of
the clauses.  For example, these specifiers implicitly exclude all
pre-releases and development releases of later versions::

    2.2
    >= 1.0

While these specifiers would include at least some of them::

    2.2.dev0
    2.2, != 2.3b2
    >= 1.0a1
    >= 1.0c1
    >= 1.0, != 1.0b2
    >= 1.0, < 2.0.dev123

Dependency resolution tools SHOULD exclude pre-releases by default,
but SHOULD also allow users to request the following alternative
behaviours:

* accept already installed pre-releases for all version specifiers
* retrieve and install available pre-releases for all version
  specifiers

Dependency resolution tools MAY also allow the above behaviour to be
controlled on a per-distribution basis.

Post-releases and purely numeric releases receive no special treatment
- they are always included unless explicitly excluded.


Examples
--------

* ``3.1``: version 3.1 or later, but not version 4.0 or
  later. Excludes pre-releases and developmental releases.
* ``3.1.2``: version 3.1.2 or later, but not version 3.2.0 or
  later. Excludes pre-releases and developmental releases.
* ``3.1a1``: version 3.1a1 or later, but not version 4.0 or
  later. Allows pre-releases like 3.2a4 and developmental releases
  like 3.2.dev1.
* ``== 3.1``: specifically version 3.1 (or 3.1.0), excludes all
  pre-releases, post releases, developmental releases and any 3.1.x
  maintenance releases.
* ``== 3.1.*``: any version that starts with 3.1, excluding
  pre-releases and developmental releases. Equivalent to the ``3.1.0``
  compatible release clause.
* ``3.1.0, != 3.1.3``: version 3.1.0 or later, but not version 3.1.3
  and not version 3.2.0 or later. Excludes pre-releases and
  developmental releases.


Updating the versioning specification
=====================================

The versioning specification may be updated with clarifications
without requiring a new PEP or a change to the metadata version.

Actually changing the version comparison semantics still requires a
new versioning scheme and metadata version defined in new PEPs.


Open issues
===========

* The new ``is`` operator seems like a reasonable way to cleanly allow
  *deployments* to bring in non-published dependencies, while heavily
  discouraging the practice for published libraries.  However, it's a
  first draft of the idea, so feedback is definitely welcome.

* Currently, the cleanest way to specify that a project runs on Python
  2.6+ and 3.3+ is to use a clause like::
      
     Requires-Python: >= 2.6, < 4.0, != 3.0.*, != 3.1.*, != 3.2.*
     
  It would be better if there was a cleaner way to specify "this OR
  that" in a version specifier.  Perhaps something like::

     Requires-Python: (2.6) or (3.3)
  
  This would be a respectable increase in the complexity of the
  parsing for version specifiers though, even if it was only allowed
  at the top level.


Summary of differences from \PEP 386
====================================

* Moved the description of version specifiers into the versioning PEP

* added the "build label" concept to better handle projects that wish
  to use a non-compliant versioning scheme internally, especially
  those based on DVCS hashes
  
* added the "compatible release" clause

* added the "build reference" clause

* separated the two kinds of "version matching" clause (strict and
  prefix)

* changed the top level sort position of the ``.devN`` suffix

* allowed single value version numbers

* explicit exclusion of leading or trailing whitespace

* explicit criterion for the exclusion of date based versions

* implicitly exclude pre-releases unless explicitly requested

* treat post releases the same way as unqualified releases

* Discuss ordering and dependencies across metadata versions

The rationale for major changes is given in the following sections.


Adding build labels
-------------------

The new build label support is intended to make it clearer that the
constraints on public version identifiers are there primarily to aid
in the creation of reliable automated dependency analysis tools.
Projects are free to use whatever versioning scheme they like
internally, so long as they are able to translate it to something the
dependency analysis tools will understand.


Changing the version scheme
---------------------------

The key change in the version scheme in this PEP relative to that in
PEP 386 is to sort top level developmental releases like ``X.Y.devN``
ahead of alpha releases like ``X.Ya1``.  This is a far more logical
sort order, as projects already using both development releases and
alphas/betas/release candidates do not want their developmental
releases sorted in between their release candidates and their full
releases.  There is no rationale for using ``dev`` releases in that
position rather than merely creating additional release candidates.

The updated sort order also means the sorting of ``dev`` versions is
now consistent between the metadata standard and the pre-existing
behaviour of ``pkg_resources`` (and hence the behaviour of current
installation tools).

Making this change should make it easier for affected existing
projects to migrate to the latest version of the metadata standard.

Another change to the version scheme is to allow single number
versions, similar to those used by non-Python projects like Mozilla
Firefox, Google Chrome and the Fedora Linux distribution.  This is
actually expected to be more useful for version specifiers (allowing
things like the simple ``Requires-Python: 3`` rather than the more
convoluted ``Requires-Python: >= 3.0, < 4``), but it is easier to
allow it for both version specifiers and release numbers, rather than
splitting the two definitions.

The exclusion of leading and trailing whitespace was made explicit
after a couple of projects with version identifiers differing only in
a trailing ``\n`` character were found on PyPI.

The exclusion of major release numbers that looks like dates was
implied by the overall text of PEP 386, but not clear in the
definition of the version scheme.  This exclusion has been made clear
in the definition of the release component.

`Appendix A` shows detailed results of an analysis of PyPI
distribution version information, as collected on 19th February, 2013.
This analysis compares the behaviour of the explicitly ordered version
schemes defined in this PEP and PEP 386 with the de facto standard
defined by the behaviour of setuptools.  These metrics are useful, as
the intent of both PEPs is to follow existing setuptools behaviour as
closely as is feasible, while still throwing exceptions for
unorderable versions (rather than trying to guess an appropriate order
as setuptools does).

Overall, the percentage of compatible distributions improves from
97.7% with PEP 386 to 98.7% with this PEP.  While the number of
projects affected in practice was small, some of the affected projects
are in widespread use (such as Pinax and selenium).  The surprising
ordering discrepancy also concerned developers and acted as an
unnecessary barrier to adoption of the new metadata standard, even for
projects that weren't directly affected.

The data also shows that the pre-release sorting discrepancies are
seen only when analysing *all* versions from PyPI, rather than when
analysing public versions.  This is largely due to the fact that PyPI
normally reports only the most recent version for each project (unless
maintainers explicitly configure their project to display additional
versions).  However, installers that need to satisfy detailed version
constraints often need to look at all available versions, as they may
need to retrieve an older release.

Even this PEP doesn't completely eliminate the sorting differences
relative to setuptools:

* Sorts differently (after translations): 38 / 28194 (0.13 %)
* Sorts differently (no translations): 2 / 28194 (0.01 %)

The two remaining sort order discrepancies picked up by the analysis
are due to a pair of projects which have PyPI releases ending with a
carriage return, alongside releases with the same version number, only
*without* the trailing carriage return.

The sorting discrepancies after translation relate mainly to
differences in the handling of pre-releases where the standard
mechanism is considered to be an improvement. For example, the
existing pkg_resources scheme will sort "1.1beta1" *after* "1.1b2",
whereas the suggested standard translation for "1.1beta1" is "1.1b1",
which sorts *before* "1.1b2".  Similarly, the pkg_resources scheme
will sort "-dev-N" pre-releases differently from "devN" pre-releases
when they occur within the same release, while the scheme in this PEP
requires normalizing both representations to ".devN" and sorting them
by the numeric component.


A more opinionated description of the versioning scheme
-------------------------------------------------------

As in PEP 386, the primary focus is on codifying existing practices to
make them more amenable to automation, rather than demanding that
existing projects make non-trivial changes to their workflow.
However, the standard scheme allows significantly more flexibility
than is needed for the vast majority of simple Python packages (which
often don't even need maintenance releases - many users are happy with
needing to upgrade to a new feature release to get bug fixes).

For the benefit of novice developers, and for experienced developers
wishing to better understand the various use cases, the specification
now goes into much greater detail on the components of the defined
version scheme, including examples of how each component may be used
in practice.

The PEP also explicitly guides developers in the direction of semantic
versioning (without requiring it), and discourages the use of several
aspects of the full versioning scheme that have largely been included
in order to cover esoteric corner cases in the practices of existing
projects and in repackaging software for Linux distributions.


Describing version specifiers alongside the versioning scheme
-------------------------------------------------------------

The main reason to even have a standardised version scheme in the
first place is to make it easier to do reliable automated dependency
analysis.  It makes more sense to describe the primary use case for
version identifiers alongside their definition.


Changing the interpretation of version specifiers
-------------------------------------------------

The previous interpretation of version specifiers made it very easy to
accidentally download a pre-release version of a dependency.  This in
turn made it difficult for developers to publish pre-release versions
of software to the Python Package Index, as even marking the package
as hidden wasn't enough to keep automated tools from downloading it,
and also made it harder for users to obtain the test release manually
through the main PyPI web interface.

The previous interpretation also excluded post-releases from some
version specifiers for no adequately justified reason.

The updated interpretation is intended to make it difficult to
accidentally accept a pre-release version as satisfying a dependency,
while allowing pre-release versions to be explicitly requested when
needed.

The "some forward compatibility assumed" default version constraint is
taken directly from the Ruby community's "pessimistic version
constraint" operator [2]_ to allow projects to take a cautious
approach to forward compatibility promises, while still easily setting
a minimum required version for their dependencies.  It is made the
default behaviour rather than needing a separate operator in order to
explicitly discourage overspecification of dependencies by library
developers. The explicit comparison operators remain available to cope
with dependencies with unreliable or non-existent backwards
compatibility policies, as well as for legitimate use cases related to
deployment of integrated applications.

The two kinds of version matching (strict and prefix based) were
separated to make it possible to sensibly define the compatible
release clauses and the desired pre-release handling semantics for
``<`` and ``>`` ordered comparison clauses.


References
==========

The initial attempt at a standardised version scheme, along with the
justifications for needing such a standard can be found in PEP 386.

.. [1] Version compatibility analysis script
   (http://hg.python.org/peps/file/default/pep-0426/pepsort.py)

.. [2] Pessimistic version constraint
   (http://docs.rubygems.org/read/chapter/16)


Appendix A
==========

Metadata v2.0 guidelines versus setuptools (note that this analysis
was run when this PEP was still embedded as part of PEP 426)::

    $ ./pepsort.py
    Comparing PEP 426 version sort to setuptools.

    Analysing release versions
      Compatible: 24477 / 28194 (86.82 %)
      Compatible with translation: 247 / 28194 (0.88 %)
      Compatible with filtering: 84 / 28194 (0.30 %)
      No compatible versions: 420 / 28194 (1.49 %)
      Sorts differently (after translations): 0 / 28194 (0.00 %)
      Sorts differently (no translations): 0 / 28194 (0.00 %)
      No applicable versions: 2966 / 28194 (10.52 %)

    Analysing public versions
      Compatible: 25600 / 28194 (90.80 %)
      Compatible with translation: 1505 / 28194 (5.34 %)
      Compatible with filtering: 13 / 28194 (0.05 %)
      No compatible versions: 420 / 28194 (1.49 %)
      Sorts differently (after translations): 0 / 28194 (0.00 %)
      Sorts differently (no translations): 0 / 28194 (0.00 %)
      No applicable versions: 656 / 28194 (2.33 %)

    Analysing all versions
      Compatible: 24239 / 28194 (85.97 %)
      Compatible with translation: 2833 / 28194 (10.05 %)
      Compatible with filtering: 513 / 28194 (1.82 %)
      No compatible versions: 320 / 28194 (1.13 %)
      Sorts differently (after translations): 38 / 28194 (0.13 %)
      Sorts differently (no translations): 2 / 28194 (0.01 %)
      No applicable versions: 249 / 28194 (0.88 %)

Metadata v1.2 guidelines versus setuptools::

    $ ./pepsort.py 386
    Comparing PEP 386 version sort to setuptools.

    Analysing release versions
      Compatible: 24244 / 28194 (85.99 %)
      Compatible with translation: 247 / 28194 (0.88 %)
      Compatible with filtering: 84 / 28194 (0.30 %)
      No compatible versions: 648 / 28194 (2.30 %)
      Sorts differently (after translations): 0 / 28194 (0.00 %)
      Sorts differently (no translations): 0 / 28194 (0.00 %)
      No applicable versions: 2971 / 28194 (10.54 %)

    Analysing public versions
      Compatible: 25371 / 28194 (89.99 %)
      Compatible with translation: 1507 / 28194 (5.35 %)
      Compatible with filtering: 12 / 28194 (0.04 %)
      No compatible versions: 648 / 28194 (2.30 %)
      Sorts differently (after translations): 0 / 28194 (0.00 %)
      Sorts differently (no translations): 0 / 28194 (0.00 %)
      No applicable versions: 656 / 28194 (2.33 %)

    Analysing all versions
      Compatible: 23969 / 28194 (85.01 %)
      Compatible with translation: 2789 / 28194 (9.89 %)
      Compatible with filtering: 530 / 28194 (1.88 %)
      No compatible versions: 547 / 28194 (1.94 %)
      Sorts differently (after translations): 96 / 28194 (0.34 %)
      Sorts differently (no translations): 14 / 28194 (0.05 %)
      No applicable versions: 249 / 28194 (0.88 %)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: