350 lines
13 KiB
Plaintext
350 lines
13 KiB
Plaintext
PEP: 427
|
|
Title: The Wheel Binary Package Format 0.1
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Daniel Holth <dholth@fastmail.fm>
|
|
BDFL-Delegate: Nick Coghlan <ncoghlan@gmail.com>
|
|
Discussions-To: <distutils-sig@python.org>
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 20-Sep-2012
|
|
Post-History:
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP describes a built-package format for Python called "wheel".
|
|
|
|
A wheel is a ZIP-format archive with a specially formatted file name
|
|
and the ``.whl`` extension. It contains a single distribution nearly
|
|
as it would be installed according to PEP 376 with a particular
|
|
installation scheme. A wheel file may be installed by simply
|
|
unpacking into site-packages with the standard 'unzip' tool, while
|
|
preserving enough information to spread its contents out onto their
|
|
final paths at any later time.
|
|
|
|
|
|
Note
|
|
====
|
|
|
|
This draft PEP describes version 0.1 of the "wheel" format. When the PEP
|
|
is accepted, the version will be changed to 1.0. (The major version
|
|
is used to indicate potentially backwards-incompatible changes to the
|
|
format.)
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
Python needs a package format that is easier to install than sdist.
|
|
Python's sdist packages are defined by and require the distutils and
|
|
setuptools build systems, running arbitrary code to build-and-install,
|
|
and re-compile, code just so it can be installed into a new
|
|
virtualenv. This system of conflating build-install is slow, hard to
|
|
maintain, and hinders innovation in both build systems and installers.
|
|
|
|
Wheel attempts to remedy these problems by providing a simpler
|
|
interface between the build system and the installer. The wheel
|
|
binary package format frees installers from having to know about the
|
|
build system, saves time by amortizing compile time over many
|
|
installations, and removes the need to install a build system in the
|
|
target environment.
|
|
|
|
|
|
Details
|
|
=======
|
|
|
|
Installing a wheel 'distribution-1.0.py32.none.any.whl'
|
|
-------------------------------------------------------
|
|
|
|
Wheel installation notionally consists of two phases:
|
|
|
|
- Unpack.
|
|
|
|
a. Parse ``distribution-1.0.dist-info/WHEEL``.
|
|
b. Check that installer is compatible with Wheel-Version. Warn if
|
|
minor version is greater, abort if major version is greater.
|
|
c. If Root-Is-Purelib == 'true', unpack archive into purelib
|
|
(site-packages).
|
|
d. Else unpack archive into platlib (site-packages).
|
|
|
|
- Spread.
|
|
|
|
a. Unpacked archive includes ``distribution-1.0.dist-info/`` and (if
|
|
there is data) ``distribution-1.0.data/``.
|
|
b. Move each subtree of ``distribution-1.0.data/`` onto its
|
|
destination path. Each subdirectory of ``distribution-1.0.data/``
|
|
is a key into a dict of destination directories, such as
|
|
``distribution-1.0.data/(purelib|platlib|headers|scripts|data)``.
|
|
The initially supported paths are taken from
|
|
``distutils.command.install``.
|
|
c. If applicable, update scripts starting with ``#!python`` to point
|
|
to the correct interpreter.
|
|
d. Update ``distribution-1.0.dist.info/RECORD`` with the installed
|
|
paths.
|
|
e. Remove empty ``distribution-1.0.data`` directory.
|
|
f. Compile any installed .py to .pyc. (Uninstallers should be smart
|
|
enough to remove .pyc even if it is not mentioned in RECORD.)
|
|
|
|
Recommended installer features
|
|
''''''''''''''''''''''''''''''
|
|
|
|
Rewrite ``#!python``.
|
|
In wheel, scripts are packaged in
|
|
``{distribution}-{version}.data/scripts/``. If the first line of
|
|
a file in ``scripts/`` starts with exactly ``b'#!python'``, rewrite to
|
|
point to the correct interpreter. Unix installers may need to add
|
|
the +x bit to these files if the archive was created on Windows.
|
|
|
|
Generate script wrappers.
|
|
In wheel, scripts packaged on Unix systems will certainly not have
|
|
accompanying .exe wrappers. Windows installers may want to add them
|
|
during install.
|
|
|
|
|
|
File Format
|
|
-----------
|
|
|
|
File name convention
|
|
''''''''''''''''''''
|
|
|
|
The wheel filename is ``{distribution}-{version}(-{build
|
|
tag})?-{python tag}-{abi tag}-{platform tag}.whl``.
|
|
|
|
distribution
|
|
Distribution name, e.g. 'django', 'pyramid'.
|
|
|
|
version
|
|
PEP-386 compliant version, e.g. 1.0.
|
|
|
|
build tag
|
|
Optional build number. Must start with a digit. A tie breaker if
|
|
two wheels have the same version. Sort as None if unspecified,
|
|
else sort the initial digits as a number, and the remainder
|
|
lexicographically.
|
|
|
|
language implementation and version tag
|
|
E.g. 'py27', 'py2', 'py3'.
|
|
|
|
abi tag
|
|
E.g. 'cp33m', 'abi3', 'none'.
|
|
|
|
platform tag
|
|
E.g. 'linux_x86_64', 'any'.
|
|
|
|
For example, ``distribution-1.0-1-py27-none-any.whl`` is the first
|
|
build of a package called 'distribution', and is compatible with
|
|
Python 2.7 (any Python 2.7 implementation), with no ABI (pure Python),
|
|
on any CPU architecture.
|
|
|
|
The last three components of the filename before the extension are
|
|
called "compatibility tags." The compatibility tags express the
|
|
package's basic interpreter requirements and are detailed in PEP 425.
|
|
|
|
|
|
File contents
|
|
'''''''''''''
|
|
|
|
The conents of a wheel file, where {distribution} is replaced with the
|
|
name of the package, e.g. ``beaglevote`` and {version} is replaced with
|
|
its version, e.g. ``1.0.0``, consist of:
|
|
|
|
#. ``/``, the root of the archive, contains all files to be installed in
|
|
``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and
|
|
``platlib`` are usually both ``site-packages``.
|
|
#. ``{distribution}-{version}.dist-info/`` contains metadata.
|
|
#. ``{distribution}-{version}.data/`` contains one subdirectory
|
|
for each non-empty install scheme key not already covered, where
|
|
the subdirectory name is an index into a dictionary of install paths
|
|
(e.g. ``data``, ``scripts``, ``include``, ``purelib`, ``platlib``).
|
|
#. Python scripts must appear in ``scripts`` and begin with exactly
|
|
``b'#!python'`` in order to enjoy script wrapper generation and
|
|
``#!python`` rewriting at install time. They may have any or no
|
|
extension.
|
|
#. ``{distribution}-{version}.dist-info/METADATA`` is Metadata version 1.3
|
|
(PEP 426) or greater format metadata.
|
|
#. ``{distribution}-{version}.dist-info/WHEEL`` is metadata about the archive
|
|
itself::
|
|
|
|
Wheel-Version: 0.1
|
|
Generator: bdist_wheel 0.7
|
|
Root-Is-Purelib: true
|
|
|
|
#. Wheel-Version is the version number of the Wheel specification.
|
|
Generator is the name and optionally the version of the software
|
|
that produced the archive. Root-Is-Purelib is true if the top level
|
|
directory of the archive should be installed into purelib;
|
|
otherwise the root should be installed into platlib.
|
|
#. A wheel installer should warn if Wheel-Version is greater than the
|
|
version it supports, and must fail if Wheel-Version has a greater
|
|
major version than the version it supports.
|
|
#. Wheel, being an installation format that is intended to work across
|
|
multiple versions of Python, does not generally include .pyc files.
|
|
#. Wheel does not contain setup.py or setup.cfg.
|
|
|
|
This version of the wheel specification is based on the distutils install
|
|
schemes and does not define how to install files to other locations.
|
|
The layout offers a superset of the functionality provided by the existing
|
|
wininst and egg binary formats.
|
|
|
|
|
|
The .dist-info directory
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
#. Wheel .dist-info directories include at a minimum METADATA, WHEEL,
|
|
and RECORD.
|
|
#. METADATA is the PEP 426 metadata (Metadata version 1.3 or greater)
|
|
#. WHEEL is the wheel metadata, specific to a build of the package.
|
|
#. RECORD is a list of (almost) all the files in the wheel and their
|
|
secure hashes. Unlike PEP 376, every file except RECORD, which
|
|
cannot contain a hash of itself, must include its hash. The hash
|
|
algorithm must be sha256 or better; specifically, md5 and sha1 are
|
|
not permitted, as signed wheel files rely on the strong hashes in
|
|
RECORD to validate the integrity of the archive.
|
|
#. INSTALLER and REQUESTED are not included in the archive.
|
|
#. RECORD.jws is used for digital signatures. It is not mentioned in
|
|
RECORD.
|
|
#. RECORD.p7s is allowed as a courtesy to anyone who would prefer to
|
|
use s/mime signatures to secure their wheel files. It is not
|
|
mentioned in RECORD and it is ignored by the official tools.
|
|
#. During extraction, wheel installers verify all the hashes in RECORD
|
|
against the file contents. Apart from RECORD and its signatures,
|
|
installation will fail if any file in the archive is not both
|
|
mentioned and correctly hashed in RECORD.
|
|
|
|
|
|
The .data directory
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
Any file that is not normally installed inside site-packages goes into
|
|
the .data directory, named as the .dist-info directory but with the
|
|
.data/ extension::
|
|
|
|
distribution-1.0.dist-info/
|
|
|
|
distribution-1.0.data/
|
|
|
|
The .data directory contains subdirectories with the scripts, headers,
|
|
documentation and so forth from the distribution. During installation the
|
|
contents of these subdirectories are moved onto their destination paths.
|
|
|
|
|
|
Signed wheel files
|
|
------------------
|
|
|
|
Wheel files include an extended RECORD that enables digital
|
|
signatures. PEP 376's RECORD is altered to include
|
|
``digestname=urlsafe_b64encode_nopad(digest)`` (urlsafe base64
|
|
encoding with no trailing = characters) as the second column instead
|
|
of an md5sum. All possible entries are hashed, including any
|
|
generated files such as .pyc files, but not RECORD. For example::
|
|
|
|
file.py,sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\_pNh2yI,3144
|
|
distribution-1.0.dist-info/RECORD,,
|
|
|
|
The signature file(s) RECORD.jws and RECORD.p7s are not mentioned in
|
|
RECORD at all since they can only be added after RECORD is generated.
|
|
Every other file in the archive must have a correct hash in RECORD,
|
|
or the installation will fail.
|
|
|
|
If JSON web signatures are used, one or more JSON Web Signature JSON
|
|
Serialization (JWS-JS) signatures may be stored in a file RECORD.jws
|
|
adjacent to RECORD. JWS is used to sign RECORD by including the SHA-256
|
|
hash of RECORD as the JWS payload::
|
|
|
|
{ "hash": "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" }
|
|
|
|
If RECORD.p7s is used, it must contain a PKCS#7 format signature of
|
|
RECORD.
|
|
|
|
A wheel installer may assume that the signature has already been checked
|
|
against RECORD, and only must verify the hashes in RECORD against the
|
|
extracted file contents.
|
|
|
|
See
|
|
|
|
- http://self-issued.info/docs/draft-ietf-jose-json-web-signature.html
|
|
- http://self-issued.info/docs/draft-jones-jose-jws-json-serialization.html
|
|
- http://self-issued.info/docs/draft-ietf-jose-json-web-key.html
|
|
- http://self-issued.info/docs/draft-jones-jose-json-private-key.html
|
|
|
|
|
|
Comparison to .egg
|
|
------------------
|
|
|
|
#. Wheel is an installation format; egg is importable. Wheel archives
|
|
do not need to include .pyc and are less tied to a specific Python
|
|
version or implementation. Wheel can install (pure Python) packages
|
|
built with previous versions of Python so you don't always have to
|
|
wait for the packager to catch up.
|
|
#. Wheel uses .dist-info directories; egg uses .egg-info. Wheel is
|
|
compatible with the new world of Python packaging and the new
|
|
concepts it brings.
|
|
#. Wheel has a richer file naming convention for today's
|
|
multi-implementation world. A single wheel archive can indicate
|
|
its compatibility with a number of Python language versions and
|
|
implementations, ABIs, and system architectures. Historically the
|
|
ABI has been specific to a CPython release, wheel is ready for the
|
|
stable ABI.
|
|
#. Wheel is lossless. The first wheel implementation bdist_wheel
|
|
always generates egg-info, and then converts it to a .whl. It is
|
|
also possible to convert existing eggs and bdist_wininst
|
|
distributions.
|
|
#. Wheel is versioned. Every wheel file contains the version of the
|
|
wheel specification and the implementation that packaged it.
|
|
Hopefully the next migration can simply be to Wheel 2.0.
|
|
#. Wheel is a reference to the other Python.
|
|
|
|
|
|
FAQ
|
|
===
|
|
|
|
Wheel defines a .data directory. Should I put all my data there?
|
|
This specification does not have an opinion on how you should organize
|
|
your code. The .data directory is just a place for any files that are
|
|
not normally installed inside ``site-packages`` or on the PYTHONPATH.
|
|
In other words, you may continue to use ``pkgutil.get_data(package,
|
|
resource)`` even though *those* files will usually not be distributed
|
|
in *wheel's* ``.data`` directory.
|
|
|
|
Why does wheel include attached signatures?
|
|
Attached signatures are more convenient than detached signatures
|
|
because they travel with the archive. Since only the individual files
|
|
are signed, the archive can be recompressed without invalidating
|
|
the signature, or individual files can be verified without having
|
|
to download the whole archive.
|
|
|
|
Appendix
|
|
========
|
|
|
|
Example urlsafe-base64-nopad implementation::
|
|
|
|
# urlsafe-base64-nopad for Python 3
|
|
import base64
|
|
|
|
def urlsafe_b64encode_nopad(data):
|
|
return base64.urlsafe_b64encode(data).rstrip(b'=')
|
|
|
|
def urlsafe_b64decode_nopad(data):
|
|
pad = b'=' * (4 - (len(data) & 3))
|
|
return base64.urlsafe_b64decode(data + pad)
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed into the public domain.
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|
|
End:
|