Apply Nathaniel's changes to PEP 513
This commit is contained in:
Chris Angelico
parent
4dea807de6
commit
6909798f26
118
pep-0513.txt
118
pep-0513.txt
|
|
@ -126,8 +126,7 @@ For these reasons, to achieve broad portability, Python wheels
|
|||
|
||||
To be eligible for the ``manylinux1`` platform tag, a Python wheel must
|
||||
therefore both (a) contain binary executables and compiled code that links
|
||||
*only* to libraries (other than the appropriate ``libpython`` library, which is
|
||||
always a permitted dependency consistent with the PEP 425 ABI tag) with SONAMEs
|
||||
*only* to libraries with SONAMEs
|
||||
included in the following list: ::
|
||||
|
||||
libpanelw.so.5
|
||||
|
|
@ -153,11 +152,7 @@ included in the following list: ::
|
|||
libglib-2.0.so.0
|
||||
|
||||
and, (b) work on a stock CentOS 5.11 [6]_ system that contains the system
|
||||
package manager's provided versions of these libraries. In addition,
|
||||
for wheels targeting CPython 3.2 and earlier (including all 2.x
|
||||
versions), there is an extra requirement that (c) the wheel be
|
||||
built against a version of CPython compiled with 4-byte unicode
|
||||
support (i.e. one where ``sys.maxunicode > 0xFFFF``).
|
||||
package manager's provided versions of these libraries.
|
||||
|
||||
Because CentOS 5 is only available for x86_64 and i686 architectures,
|
||||
these are the only architectures currently supported by the ``manylinux1``
|
||||
|
|
@ -216,6 +211,81 @@ the list above to remove libraries that have turned out to be
|
|||
problematic or add libraries that have turned out to be safe.
|
||||
|
||||
|
||||
libpythonX.Y.so.1
|
||||
-----------------
|
||||
|
||||
Note that ``libpythonX.Y.so.1`` is *not* on the list of libraries that
|
||||
a ``manylinux1`` extension is allowed to link to. Explicitly linking
|
||||
to ``libpythonX.Y.so.1`` is unnecessary in almost all cases: the way
|
||||
ELF linking works, extension modules that are loaded into the
|
||||
interpreter automatically get access to all of the interpreter's
|
||||
symbols, regardless of whether or not the extension itself is
|
||||
explicitly linked against libpython. Furthermore, explicit linking to
|
||||
libpython creates problems in the common configuration where Python is
|
||||
not built with ``--enable-shared``. In particular, on Debian and
|
||||
Ubuntu systems, ``apt install pythonX.Y`` does not even install
|
||||
``libpythonX.Y.so.1``, meaning that any wheel that *did* depend on
|
||||
``libpythonX.Y.so.1`` could fail to import.
|
||||
|
||||
There is one situation where extensions that are linked in this way
|
||||
can fail to work: if a host program (e.g., ``apache2``) uses
|
||||
``dlopen()`` to load a module (e.g., ``mod_wsgi``) that embeds the
|
||||
CPython interpreter, and the host program does *not* pass the
|
||||
``RTLD_GLOBAL`` flag to ``dlopen()``, then the embedded CPython will
|
||||
be unable to load any extension modules that do not themselves link
|
||||
explicitly to ``libpythonX.Y.so.1``. Fortunately, ``apache2`` *does*
|
||||
set the ``RTLD_GLOBAL`` flag, as do all the other programs that
|
||||
embed-CPython-via-a-dlopened-plugin that we could locate, so this does
|
||||
not seem to be a serious problem in practice. The incompatibility with
|
||||
Debian/Ubuntu is more of an issue than the theoretical incompatibility
|
||||
with a rather obscure corner case.
|
||||
|
||||
This is a rather complex and subtle issue that extends beyond
|
||||
the scope of ``manylinux1``; for more discussion see: [9]_, [10]_,
|
||||
[11]_.
|
||||
|
||||
UCS-2 vs UCS-4 builds
|
||||
---------------------
|
||||
|
||||
All versions of CPython 2.x, plus CPython 3.0-3.2 inclusive, can be
|
||||
built in two ABI-incompatible modes: builds using the
|
||||
``--enable-unicode=ucs2`` configure flag store Unicode data in UCS-2
|
||||
(or really UTF-16) format, while builds using the
|
||||
``--enable-unicode=ucs4`` configure flag store Unicode data in
|
||||
UCS-4. (CPython 3.3 and greater use a different storage method that
|
||||
always supports UCS-4.) If we want to make sure ``ucs2`` wheels don't
|
||||
get installed into ``ucs4`` CPythons and vice-versa, then something
|
||||
must be done.
|
||||
|
||||
An earlier version of this PEP included a requirement that
|
||||
``manylinux1`` wheels targeting these older CPython versions should
|
||||
always use the ``ucs4`` ABI. But then, in between the PEP's initial
|
||||
acceptance and its implementation, ``pip`` and ``wheel`` gained
|
||||
first-class support for tracking and checking this aspect of ABI
|
||||
compatibility for the relevant CPython versions, which is a better
|
||||
solution. So we now allow the ``manylinux1`` platform tags to be used
|
||||
in combination with any ABI tag. However, to maintain compatibility it
|
||||
is crucial to ensure that all ``manylinux1`` wheels include a
|
||||
non-trivial abi tag. For example, a wheel built against a ``ucs4``
|
||||
CPython might have a name like::
|
||||
|
||||
PKG-VERSION-cp27-cp27mu-manylinux1_x86_64.whl
|
||||
^^^^^^ Good!
|
||||
|
||||
While a wheel built against the ``ucs2`` ABI might have a name like::
|
||||
|
||||
PKG-VERSION-cp27-cp27m-manylinux1_x86_64.whl
|
||||
^^^^^ Okay!
|
||||
|
||||
But you should never have a wheel with a name like::
|
||||
|
||||
PKG-VERSION-cp27-none-manylinux1_x86_64.whl
|
||||
^^^^ BAD! Don't do this!
|
||||
|
||||
We note for information that the ``ucs4`` ABI appears to be much more
|
||||
widespread among Linux CPython distributors.
|
||||
|
||||
|
||||
Compilation of Compliant Wheels
|
||||
===============================
|
||||
|
||||
|
|
@ -237,7 +307,7 @@ Docker Image
|
|||
|
||||
The first tool is a Docker image based on CentOS 5.11, which is recommended as
|
||||
an easy to use self-contained build box for compiling ``manylinux1`` wheels
|
||||
[9]_. Compiling on a more recently-released linux distribution will generally
|
||||
[12]_. Compiling on a more recently-released linux distribution will generally
|
||||
introduce dependencies on too-new versioned symbols. The image comes with a
|
||||
full compiler suite installed (``gcc``, ``g++``, and ``gfortran`` 4.8.2) as
|
||||
well as the latest releases of Python and ``pip``.
|
||||
|
|
@ -245,7 +315,7 @@ well as the latest releases of Python and ``pip``.
|
|||
Auditwheel
|
||||
----------
|
||||
|
||||
The second tool is a command line executable called ``auditwheel`` [10]_ that
|
||||
The second tool is a command line executable called ``auditwheel`` [13]_ that
|
||||
may aid in package maintainers in dealing with third-party external
|
||||
dependencies.
|
||||
|
||||
|
|
@ -285,7 +355,7 @@ While we acknowledge many approaches for dealing with third-party library
|
|||
dependencies within ``manylinux1`` wheels, we recognize that the ``manylinux1``
|
||||
policy encourages bundling external dependencies, a practice
|
||||
which runs counter to the package management policies of many linux
|
||||
distributions' system package managers [11]_, [12]_. The primary purpose of
|
||||
distributions' system package managers [14]_, [15]_. The primary purpose of
|
||||
this is cross-distro compatibility. Furthermore, ``manylinux1`` wheels on PyPI
|
||||
occupy a different niche than the Python packages available through the
|
||||
system package manager.
|
||||
|
|
@ -352,8 +422,6 @@ think otherwise.
|
|||
We know of four main sources of potential incompatibility that are
|
||||
likely to arise in practice:
|
||||
|
||||
* "Narrow" unicode builds of Python 3.2 and earlier (including all
|
||||
versions of Python 2)
|
||||
* Eventually, in the future, there may exist distributions that break
|
||||
compatibility with this profile (e.g., if one of the libraries in
|
||||
the profile changes its ABI in a backwards-incompatible way)
|
||||
|
|
@ -361,8 +429,7 @@ likely to arise in practice:
|
|||
* A linux distribution that does not use ``glibc`` (e.g. Alpine Linux, which is
|
||||
based on musl ``libc``, or Android)
|
||||
|
||||
Checking for unicode configuration compatibility is straightforward,
|
||||
but the other cases are more subtle. We propose a two-pronged
|
||||
To address these we propose a two-pronged
|
||||
approach. To handle potential future incompatibilities, we standardize
|
||||
a mechanism for a Python distributor to signal that a particular
|
||||
Python install definitely is or is not compatible with ``manylinux1``:
|
||||
|
|
@ -389,11 +456,6 @@ Specifically, the algorithm we propose is::
|
|||
if get_platform() not in ["linux-x86_64", "linux-i686"]:
|
||||
return False
|
||||
|
||||
# "wide" Unicode mode is mandatory (always true on CPython 3.3+)
|
||||
import sys
|
||||
if sys.maxunicode <= 0xFFFF:
|
||||
return False
|
||||
|
||||
# Check for presence of _manylinux module
|
||||
try:
|
||||
import _manylinux
|
||||
|
|
@ -522,13 +584,21 @@ References
|
|||
(https://groups.google.com/forum/#!topic/manylinux-discuss/-4l3rrjfr9U)
|
||||
.. [8] distutils-sig discussion
|
||||
(https://mail.python.org/pipermail/distutils-sig/2016-January/027997.html)
|
||||
.. [9] manylinux1 docker image
|
||||
(https://quay.io/repository/manylinux/manylinux)
|
||||
.. [10] auditwheel tool
|
||||
.. [9] distutils-sig discussion
|
||||
(https://mail.python.org/pipermail/distutils-sig/2016-February/028275.html)
|
||||
.. [10] github issue discussion
|
||||
(https://github.com/pypa/manylinux/issues/30)
|
||||
.. [11] python bug tracker discussion
|
||||
(https://bugs.python.org/issue21536)
|
||||
.. [12] manylinux1 docker images
|
||||
(Source: https://github.com/pypa/manylinux;
|
||||
x86-64: https://quay.io/repository/pypa/manylinux1_x86_64;
|
||||
x86-32: https://quay.io/repository/pypa/manylinux1_i686)
|
||||
.. [13] auditwheel tool
|
||||
(https://pypi.python.org/pypi/auditwheel)
|
||||
.. [11] Fedora Bundled Software Policy
|
||||
.. [14] Fedora Bundled Software Policy
|
||||
(https://fedoraproject.org/wiki/Bundled_Software_policy)
|
||||
.. [12] Debian Policy Manual -- 4.13: Convenience copies of code
|
||||
.. [15] Debian Policy Manual -- 4.13: Convenience copies of code
|
||||
(https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue