PEP: 386 Title: Changing the version comparison module in Distutils Version: $Revision$ Last-Modified: $Date$ Author: Tarek Ziadé Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-June-2009 Abstract ======== This PEP proposes the inclusion of a new version comparison system in Distutils. Motivation ========== Distutils will soon extend the metadata standard, by including the `install_requires` field from Setuptools [#requires]_ among other changes. These changes are a work in progress in PEP 345 [#pep345]_, but validating the current PEP is mandatory to continue the work. The `install_requires` field will allow a package to define a dependency on another package and optionally restrict this dependency to a set of compatible versions. That's why Distutils needs to provide a robust standard and reference implementation to compare versions numbers. This will also provide to the community a convention for their package versioning needs. Current status ============== In Python there are no real restriction yet on how a project should manage its versions, and how they should be incremented. They are no standard either, even if they are a few conventions widely used, like having a major and a minor revision (1.1, 1.2, etc.). Developers are free to put in the `version` meta-data of their package any string they want, and push a new release at PyPI. This version will appear as the `latest` for end users. Some project are also using dates as their major version numbers, or a custom versioning standard that is sometimes quite exotic. The problem with this freedom is that the package will be harder to re-package for OS packagers, that need to have stricter conventions. The worst case is when a packager is unable to easily compare the versions he needs to package. For people that want to go further and use a tool to manage their version numbers, the two major ones are: - The current Distutils system [#distutils]_ - Setuptools [#setuptools]_ Distutils --------- Distutils currently provides a `StrictVersion` and a `LooseVersion` class that can be used to manage versions. The `LooseVersion` class is quite lax. From Distutils doc:: Version numbering for anarchists and software realists. Implements the standard interface for version number classes as described above. A version number consists of a series of numbers, separated by either periods or strings of letters. When comparing version numbers, the numeric components will be compared numerically, and the alphabetic components lexically. The following are all valid version numbers, in no particular order: 1.5.1 1.5.2b2 161 3.10a 8.02 3.4j 1996.07.12 3.2.pl0 3.1.1.6 2g6 11g 0.960923 2.2beta29 1.13++ 5.5.kw 2.0b1pl0 In fact, there is no such thing as an invalid version number under this scheme; the rules for comparison are simple and predictable, but may not always give the results you want (for some definition of "want"). This class makes any version string valid, and provides an algorithm to sort them numerically then lexically. It means that anything can be used to version your project:: >>> from distutils.version import LooseVersion as V >>> v1 = V('FunkyVersion') >>> v2 = V('GroovieVersion') >>> v1 > v2 False The `StrictVersion` class is more strict. From the doc:: Version numbering for anal retentive and software idealists. Implements the standard interface for version number classes as described above. A version number consists of two or three dot-separated numeric components, with an optional "pre-release" tag on the end. The pre-release tag consists of the letter 'a' or 'b' followed by a number. If the numeric components of two version numbers are equal, then one with a pre-release tag will always be deemed earlier (lesser) than one without. The following are valid version numbers (shown in the order that would be obtained by sorting according to the supplied cmp function): 0.4 0.4.0 (these two are equivalent) 0.4.1 0.5a1 0.5b3 0.5 0.9.6 1.0 1.0.4a3 1.0.4b1 1.0.4 The following are examples of invalid version numbers: 1 2.7.2.2 1.3.a4 1.3pl1 1.3c4 This class enforces a few rules, and makes a decent tool to work with version numbers:: >>> from distutils.version import StrictVersion as V >>> v2 = V('GroovieVersion') Traceback (most recent call last): ... ValueError: invalid version number 'GroovieVersion' >>> v2 = V('1.1') >>> v3 = V('1.3') >>> v2 < v3 True Although, it lacks a few elements to make it usable: - development releases - post-release tags - development releases of post-release tags. Notice that Distutils version classes are not really used in the community. Setuptools ---------- Setuptools provides another version comparison tool [#setuptools-version]_ which does not enforce any rule for the version, but try to provide a better algorithm to convert the strings to sortable keys, with a ``parse_version`` function. From the doc:: Convert a version string to a chronologically-sortable key This is a rough cross between Distutils' StrictVersion and LooseVersion; if you give it versions that would work with StrictVersion, then it behaves the same; otherwise it acts like a slightly-smarter LooseVersion. It is *possible* to create pathological version coding schemes that will fool this parser, but they should be very rare in practice. The returned value will be a tuple of strings. Numeric portions of the version are padded to 8 digits so they will compare numerically, but without relying on how numbers compare relative to strings. Dots are dropped, but dashes are retained. Trailing zeros between alpha segments or dashes are suppressed, so that e.g. "2.4.0" is considered the same as "2.4". Alphanumeric parts are lower-cased. The algorithm assumes that strings like "-" and any alpha string that alphabetically follows "final" represents a "patch level". So, "2.4-1" is assumed to be a branch or patch of "2.4", and therefore "2.4.1" is considered newer than "2.4-1", which in turn is newer than "2.4". Strings like "a", "b", "c", "alpha", "beta", "candidate" and so on (that come before "final" alphabetically) are assumed to be pre-release versions, so that the version "2.4" is considered newer than "2.4a1". Finally, to handle miscellaneous cases, the strings "pre", "preview", and "rc" are treated as if they were "c", i.e. as though they were release candidates, and therefore are not as new as a version string that does not contain them, and "dev" is replaced with an '@' so that it sorts lower than than any other pre-release tag. In other words, ``parse_version`` will return a tuple for each version string, that is compatible with ``StrictVersion`` but also accept arbitrary version and deal with them so they can be compared:: >>> from pkg_resources import parse_version as V >>> V('1.2') ('00000001', '00000002', '*final') >>> V('1.2b2') ('00000001', '00000002', '*b', '00000002', '*final') >>> V('FunkyVersion') ('*funkyversion', '*final') Caveats of existing systems --------------------------- The major problem with the described version comparison tools is that they are too permissive. Many of the versions on PyPI [#pypi]_ are obviously not useful versions, which makes it difficult for users to grok the versioning that a particular package was using and to provide tools on top of PyPI. Distutils classes are not really used in Python projects, but the Setuptools function is quite spread because it's used by tools like `easy_install` [#ezinstall]_, `pip` [#pip]_ or `zc.buildout` [#zc.buildout]_ to install dependencies of a given project. While Setuptools *does* provide a mechanism for comparing/sorting versions, it is much preferable if the versioning spec is such that a human can make a reasonable attempt at that sorting without having to run it against some code. Also there's a problem with the use of dates at the "major" version number (e.g. a version string "20090421") with RPMs: it means that any attempt to switch to a more typical "major.minor..." version scheme is problematic because it will always sort less than "20090421". Last, the meaning of `-` is specific to Setuptools, while it is avoided in some packaging systems like the one used by Debian or Ubuntu. The new versioning algorithm ============================ During Pycon, members of the Python, Ubuntu and Fedora community worked on a version standard that would be acceptable for everyone. It's currently called `verlib` and a prototype lives here : http://bitbucket.org/tarek/distutilsversion/src/ The pseudo-format supported is:: N.N[.N]+[abc]N[.N]+[.postN+][.devN+] Some examples probably make it clearer:: >>> from verlib import RationalVersion as V >>> (V('1.0a1') ... < V('1.0a2.dev456') ... < V('1.0a2') ... < V('1.0a2.1.dev456') ... < V('1.0a2.1') ... < V('1.0b1.dev456') ... < V('1.0b2') ... < V('1.0c1.dev456') ... < V('1.0c1') ... < V('1.0.dev456') ... < V('1.0') ... < V('1.0.post623.dev456') ... < V('1.0.post456')) True The trailing ".dev123" is for pre-releases. The ".post123" is for post-releases -- which apparently is used by a number of projects out there (e.g. Twisted [#twisted]_). For example *after* a "1.2.0" release there might be a "1.2.0-r678" release. We used "post" instead of "r" because the "r" is ambiguous as to whether it indicates a pre- or post-release. Last ".post623.dev456" is a development version of a post-release. ``verlib`` provides a ``RationalVersion`` class and a ``suggest_rational_version`` function. RationalVersion --------------- The `RationalVersion` class is used to hold a version and to compare it with others. It takes a string as an argument, that contains the representation of the version:: >>> from verlib import RationalVersion >>> version = RationalVersion('1.0') The version can be represented as a string:: >>> str(version) '1.0' Or compared with others:: >>> RationalVersion('1.0') > RationalVersion('0.9') True >>> RationalVersion('1.0') < RationalVersion('1.1') True A class method called ``from_parts`` is available if you want to create an instance by providing the parts that composes the version. Each part is a tuple and there are three parts: - the main version part - the pre-release part - the `postdev` marker part Examples :: >>> version = RationalVersion.from_parts((1, 0)) >>> str(version) '1.0' >>> version = RationalVersion.from_parts((1, 0), ('c', 4)) >>> str(version) '1.0c4' >>> version = RationalVersion.from_parts((1, 0), ('c', 4), ('dev', 34)) >>> str(version) '1.0c4.dev34' suggest_rational_version ------------------------ XXX explain here suggest_rational_version References ========== .. [#distutils] http://docs.python.org/distutils .. [#setuptools] http://peak.telecommunity.com/DevCenter/setuptools .. [#setuptools-version] http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version .. [#pypi] http://pypi.python.org/pypi .. [#pip] http://pypi.python.org/pypi/pip .. [#ezinstall] http://peak.telecommunity.com/DevCenter/EasyInstall .. [#zc.buildout] http://pypi.python.org/pypi/zc.buildout .. [#twisted] http://twistedmatrix.com/trac/ .. [#requires] http://peak.telecommunity.com/DevCenter/setuptools .. [#pep345] http://svn.python.org/projects/peps/branches/jim-update-345/pep-0345.txt Aknowledgments ============== Trent Mick, Matthias Klose, Phillip Eby, and many people at Pycon and Distutils-SIG. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: