PEP: 427 Title: The Wheel Binary Package Format 0.1 Version: $Revision$ Last-Modified: $Date$ Author: Daniel Holth Discussions-To: Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 20-Sep-2012 Post-History: Abstract ======== This PEP describes a built-package format for Python called "wheel". A wheel is a ZIP-format archive with a specially formatted file name and the ``.whl`` extension. It contains a single distribution nearly as it would be installed according to PEP 376 with a particular installation scheme. A wheel file may be installed by simply unpacking into site-packages with the standard 'unzip' tool, while preserving enough information to spread its contents out onto their final paths at any later time. Note ==== This draft PEP describes version 0.1 of the "wheel" format. When the PEP is accepted, the version will be changed to 1.0. (The major version is used to indicate potentially backwards-incompatible changes to the format.) Rationale ========= Python needs a package format that is easier to install than sdist. Python's sdist packages are defined by and require the distutils and setuptools build systems, running arbitrary code to build-and-install, and re-compile, code just so it can be installed into a new virtualenv. This system of conflating build-install is slow, hard to maintain, and hinders innovation in both build systems and installers. Wheel attempts to remedy these problems by providing a simpler interface between the build system and the installer. The wheel binary package format frees installers from having to know about the build system, saves time by amortizing compile time over many installations, and removes the need to install a build system in the target environment. Details ======= Installing a wheel 'distribution-1.0.py32.none.any.whl' ------------------------------------------------------- Wheel installation notionally consists of two phases: - Unpack. a. Parse ``distribution-1.0.dist-info/WHEEL``. b. Check that installer is compatible with Wheel-Version. Warn if minor version is greater, abort if major version is greater. c. If Root-Is-Purelib == 'true', unpack archive into purelib (site-packages). d. Else unpack archive into platlib (site-packages). - Spread. a. Unpacked archive includes ``distribution-1.0.dist-info/`` and (if there is data) ``distribution-1.0.data/``. b. Move each subtree of ``distribution-1.0.data/`` onto its destination path. Each subdirectory of ``distribution-1.0.data/`` is a key into a dict of destination directories, such as ``distribution-1.0.data/(purelib|platlib|headers|scripts|data)``. The initially supported paths are taken from ``distutils.command.install``. c. If applicable, update scripts starting with ``#!python`` to point to the correct interpreter. d. Update ``distribution-1.0.dist.info/RECORD`` with the installed paths. e. Remove empty ``distribution-1.0.data`` directory. f. Compile any installed .py to .pyc. (Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.) Recommended installer features '''''''''''''''''''''''''''''' Rewrite ``#!python``. In wheel, scripts are packaged in ``{distribution}-{version}.data/scripts/``. If the first line of a file in ``scripts/`` starts with exactly b'#!python', rewrite to point to the correct interpreter. Unix installers may need to add the +x bit to these files if the archive was created on Windows. Generate script wrappers. In wheel, scripts packaged on Unix systems will certainly not have accompanying .exe wrappers. Windows installers may want to add them during install. File Format ----------- File name convention '''''''''''''''''''' The wheel filename is ``{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl``. distribution Distribution name, e.g. 'django', 'pyramid'. version PEP-386 compliant version, e.g. 1.0. build tag Optional build number. Must start with a digit. A tie breaker if two wheels have the same version. Sort as None if unspecified, else sort the initial digits as a number, and the remainder lexicographically. language implementation and version tag E.g. 'py27', 'py2', 'py3'. abi tag E.g. 'cp33m', 'abi3', 'none'. platform tag E.g. 'linux_x86_64', 'any'. For example, ``distribution-1.0-1-py27-none-any.whl`` is the first build of a package called 'distribution', and is compatible with Python 2.7 (any Python 2.7 implementation), with no ABI (pure Python), on any CPU architecture. The last three components of the filename before the extension are called "compatibility tags." The compatibility tags express the package's basic interpreter requirements and are detailed in PEP 425. File contents ''''''''''''' #. Wheel files contain a folder {distribution}-{version}.dist-info/ with the PEP 426 metadata (Metadata version 1.3 or greater) and an additional file WHEEL with metadata about the archive itself. #. The root of a .whl is installed into one of purelib or platlib. #. Wheel files contain metadata about the wheel format itself in ``{distribution}-{version}.dist-info/WHEEL``:: Wheel-Version: 0.1 Generator: bdist_wheel 0.7 Root-Is-Purelib: true #. Wheel-Version is the version number of the Wheel specification. Generator is the name and optionally the version of the software that produced the archive. Root-Is-Purelib is true if the top level directory of the archive should be installed into purelib; otherwise the root should be installed into platlib. #. A wheel installer should warn if Wheel-Version is greater than the version it supports, and fail if Wheel-Version has a greater major version than the version it supports. #. If a .whl contains scripts, both purelib and platlib, or any other files that are not installed on ``sys.path``, they are found in ``{distribution}-{version}.data/{key}``, where ``{key}`` is an index into a dictionary of install paths. #. Wheel, being an installation format that is intended to work across multiple versions of Python, does not generally include .pyc files. #. Wheel does not contain setup.py or setup.cfg. The .dist-info directory ^^^^^^^^^^^^^^^^^^^^^^^^ #. Wheel .dist-info directories include at a minimum METADATA, WHEEL, and RECORD. #. METADATA is the PEP 426 metadata (Metadata version 1.3 or greater) #. WHEEL is the wheel metadata, specific to a build of the package. #. RECORD is a list of (almost) all the files in the wheel and their secure hashes. Unlike PEP 376, every file except RECORD, which cannot contain a hash of itself, must include its hash. The hash algorithm must be sha256 or better; specifically, md5 and sha1 are not permitted, as signed wheel files rely on the strong hashes in RECORD to validate the integrity of the archive. #. INSTALLER and REQUESTED are not included in the archive. #. RECORD.jws is used for digital signatures. It is not mentioned in RECORD. #. RECORD.p7s is reserved as a courtesy to anyone who would prefer to use s/mime signatures to secure their wheel files. It is not mentioned in RECORD. #. During extraction, wheel installers verify all the hashes in RECORD against the file contents. Apart from RECORD and its signatures, installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD. The .data directory ^^^^^^^^^^^^^^^^^^^ Any file that is not normally installed inside site-packages goes into the .data directory, named as the .dist-info directory but with the .data/ extension:: distribution-1.0.dist-info/ distribution-1.0.data/ The .data directory contains subdirectories and those subdirectories contain the scripts, headers, documentation and so forth from the distribution. During installation the contents of these subdirectories are moved onto their destination paths. Signed wheel files ------------------ Wheel files include an extended RECORD that enables digital signatures. PEP 376's RECORD is altered to include ``digestname=urlsafe_b64encode_nopad(digest)`` (urlsafe base64 encoding with no trailing = characters) as the second column instead of an md5sum. All possible entries are hashed, including any generated files such as .pyc files, but not RECORD. For example:: file.py,sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\_pNh2yI,3144 distribution-1.0.dist-info/RECORD,, RECORD.jws is not mentioned in RECORD at all. Every other file in the archive must have a correct hash in RECORD, or the installation will fail. The signature is one or more JSON Web Signature JSON Serialization (JWS-JS) signatures stored in a file RECORD.jws adjacent to RECORD. A signature-aware installer can be instructed to check for a particular Ed25519 public key by using an extended "extras" syntax.:: # request a normal optional feature "extra", and indicate # the package should be signed by a particular # urlsafe-b64encode-nopad encoded ed25519 public key: package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA] An application could distribute a requires.txt file with many such lines for all its dependencies and their public keys. By installing from this file an application's users would know whether the application's dependencies came from the correct publishers. Applications that wish to "fail open" for backwards compatibility with non-signature-aware installers should specify that their package provides the extra ``ed25519=(key)`` with no associated dependencies. Key distribution is outside the scope of this spec. Public wheel signing keys could be signed with the packager's GPG key or stored at an ``https://`` protected URL. JSON Web Signatures Extensions '''''''''''''''''''''''''''''' The Ed25519 algorithm is used as an extension to the JSON Web Signatures specification. Wheel uses ``alg="Ed25519"`` in the header. The key attribute holds the signature's public JSON Web Key. For JSON Web Key / JSON Private Key the verifying (public) key is called vk and the signing (private) key is called sk. Example header:: { "alg": "Ed25519", "typ": "JWT", "key": { "alg": "Ed25519", "vk": "tmAYCrSfj8gtJ10v3VkvW7jOndKmQIYE12hgnFu3cvk" } } A future version of wheel may omit ``typ``. Example payload:: { "hash": "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" } A future version of wheel may include timestamps in the payload or in the signature. See - http://self-issued.info/docs/draft-ietf-jose-json-web-signature.html - http://self-issued.info/docs/draft-jones-json-web-signature-json-serialization-01.html - http://self-issued.info/docs/draft-ietf-jose-json-web-key-05.html - http://self-issued.info/docs/draft-jones-jose-json-private-key-00.html Comparison to .egg ------------------ #. Wheel is an installation format; egg is importable. Wheel archives do not need to include .pyc and are less tied to a specific Python version or implementation. Wheel can install (pure Python) packages built with previous versions of Python so you don't always have to wait for the packager to catch up. #. Wheel uses .dist-info directories; egg uses .egg-info. Wheel is compatible with the new world of Python packaging and the new concepts it brings. #. Wheel has a richer file naming convention for today's multi-implementation world. A single wheel archive can indicate its compatibility with a number of Python language versions and implementations, ABIs, and system architectures. Historically the ABI has been specific to a CPython release, wheel is ready for the stable ABI. #. Wheel is lossless. The first wheel implementation bdist_wheel always generates egg-info, and then converts it to a .whl. It is also possible to convert existing eggs and bdist_wininst distributions. #. Wheel is versioned. Every wheel file contains the version of the wheel specification and the implementation that packaged it. Hopefully the next migration can simply be to Wheel 2.0. #. Wheel is a reference to the other Python. FAQ === Wheel defines a .data directory. Should I put all my data there? This specification does not have an opinion on how you should organize your code. The .data directory is just a place for any files that are not normally installed inside ``site-packages`` or on the PYTHONPATH. Copyright ========= This document has been placed into the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: