PEP 453 updates

- use the module name Antoine suggested
- list the five distinct parts of the proposal when
  describing the implementation strategy
- note the bootstrap approach makes it easy to let people opt-out
- allow distros to redirect a global ensurepip invocation to the
  system package manager
- decide the open questions in favour of the currently documented
  approaches (i.e. no changes to uninstallation, update the
  directory layout and PATH handling on Windows)
This commit is contained in:
Nick Coghlan 2013-09-22 15:57:28 +10:00
parent e5208fc437
commit 3a9eb4ef0b
1 changed files with 83 additions and 57 deletions

View File

@ -34,7 +34,7 @@ at the very least, explicitly document the fact that it is not included.
Proposal Proposal
======== ========
This PEP proposes the inclusion of an ``extractpip`` bootstrapping module in This PEP proposes the inclusion of an ``ensurepip`` bootstrapping module in
Python 3.4, as well as in the next maintenance releases of Python 3.3 and Python 3.4, as well as in the next maintenance releases of Python 3.3 and
2.7. 2.7.
@ -164,11 +164,11 @@ binary dependencies).
Explicit bootstrapping mechanism Explicit bootstrapping mechanism
================================ ================================
An additional module called ``extractpip`` will be added to the standard An additional module called ``ensurepip`` will be added to the standard
library whose purpose is to install pip and any of its dependencies into the library whose purpose is to install pip and any of its dependencies into the
appropriate location (most commonly site-packages). It will expose a appropriate location (most commonly site-packages). It will expose a
callable named ``bootstrap()`` as well as offer direct execution via callable named ``bootstrap()`` as well as offer direct execution via
``python -m extractpip``. ``python -m ensurepip``.
The bootstrap will *not* contact PyPI, but instead rely on a private copy The bootstrap will *not* contact PyPI, but instead rely on a private copy
of pip stored inside the standard library. Accordingly, only options of pip stored inside the standard library. Accordingly, only options
@ -208,15 +208,15 @@ Implementation strategy
----------------------- -----------------------
To ensure there is no need for network access when installing Python or To ensure there is no need for network access when installing Python or
creating virtual environments, the ``extractpip`` module will, as an creating virtual environments, the ``ensurepip`` module will, as an
implementation detail, include a complete private copy of pip and its implementation detail, include a complete private copy of pip and its
dependencies which will be used to extract pip and install it into the target dependencies which will be used to extract pip and install it into the target
environment. It is important to stress that this private copy of pip is environment. It is important to stress that this private copy of pip is
*only* an implementation detail and it should *not* be relied on or *only* an implementation detail and it should *not* be relied on or
assumed to exist beyond the public capabilities exposed through the assumed to exist beyond the public capabilities exposed through the
``extractpip`` module (and indirectly through ``venv``). ``ensurepip`` module (and indirectly through ``venv``).
There is not yet a reference ``extractpip`` implementation. The existing There is not yet a reference ``ensurepip`` implementation. The existing
``get-pip.py`` bootstrap script demonstrates an earlier variation of the ``get-pip.py`` bootstrap script demonstrates an earlier variation of the
general concept, but the standard library version would take advantage of general concept, but the standard library version would take advantage of
the improved distribution capabilities offered by the CPython installers the improved distribution capabilities offered by the CPython installers
@ -225,11 +225,28 @@ to include private copies of ``pip`` and ``setuptools`` as wheel files
contact PyPI (instead installing directly from the private wheel files. contact PyPI (instead installing directly from the private wheel files.
Rather than including separate code to handle the bootstrapping, the Rather than including separate code to handle the bootstrapping, the
``extractpip`` module will manipulate sys.path appropriately to allow ``ensurepip`` module will manipulate sys.path appropriately to allow
the wheel files to be used to install themselves, either into the current the wheel files to be used to install themselves, either into the current
Python installation or into a virtual environment (as determined by the Python installation or into a virtual environment (as determined by the
options passed to the bootstrap command). options passed to the bootstrap command).
It is proposed that the implementation be carried out in five separate
steps (all steps after the first are independent of each other and can be
carried out in any order):
* the first step would add the ``ensurepip`` module and the private copies
of the most recently released versions of pip and setuptools, and update
the "Installing Python Modules" documentation. This change
would be applied to Python 2.7, 3.3 and 3.4.
* the Windows installer would be updated to offer the new ``pip``
installation option for Python 2.7.6, 3.3.3 and 3.4.0.
* the Mac OS X installer would be updated to offer the new ``pip``
installation option for Python 2.7.6, 3.3.3 and 3.4.0.
* the ``venv`` module and ``pyvenv`` command would be updated to make use
of ``ensurepip`` in Python 3.4+
* the PATH handling and ``sysconfig`` directory layout on Windows would be
updated for Python 3.4+
Proposed CLI Proposed CLI
------------ ------------
@ -238,7 +255,7 @@ The proposed CLI is based on a subset of the existing ``pip install``
options:: options::
Usage: Usage:
python -m extractpip [options] python -m ensurepip [options]
General Options: General Options:
-h, --help Show help. -h, --help Show help.
@ -256,14 +273,13 @@ should have been installed automatically when installing Python or when
creating a virtual environment. creating a virtual environment.
Users that want to retrieve the latest version from PyPI, or otherwise Users that want to retrieve the latest version from PyPI, or otherwise
needing more flexibility, should invoke the extracted ``pip`` need more flexibility, should invoke the extracted ``pip`` appropriately.
appropriately.
Proposed module API Proposed module API
------------------- -------------------
The proposed ``extractpip`` module API consists of the following two The proposed ``ensurepip`` module API consists of the following two
functions:: functions::
def version(): def version():
@ -290,12 +306,12 @@ This option will be checked by default.
If the option is checked, then the installer will invoke the following If the option is checked, then the installer will invoke the following
command with the just installed Python:: command with the just installed Python::
python -m extractpip --upgrade python -m ensurepip --upgrade
This ensures that, by default, installing or updating CPython will ensure This ensures that, by default, installing or updating CPython will ensure
that the installed version of pip is at least as recent as the one included that the installed version of pip is at least as recent as the one included
with that version of CPython. If a newer version of pip has already been with that version of CPython. If a newer version of pip has already been
installed then ``python -m extractpip --upgrade`` will simply return without installed then ``python -m ensurepip --upgrade`` will simply return without
doing anything. doing anything.
@ -303,11 +319,11 @@ Installing from source
---------------------- ----------------------
While the prebuilt binary installers will be updated to run While the prebuilt binary installers will be updated to run
``python -m extractpip`` by default, no such change will be made to the ``python -m ensurepip`` by default, no such change will be made to the
``make install`` and ``make altinstall`` commands of the source ``make install`` and ``make altinstall`` commands of the source
distribution. distribution.
``extractpip`` itself (including the private copy of ``pip`` and its ``ensurepip`` itself (including the private copy of ``pip`` and its
dependencies) will still be installed normally (as it is a regular dependencies) will still be installed normally (as it is a regular
part of the standard library), only the implicit installation of pip and part of the standard library), only the implicit installation of pip and
its dependencies will be skipped. its dependencies will be skipped.
@ -315,7 +331,7 @@ its dependencies will be skipped.
Keeping the pip bootstrapping as a separate step for ``make``-based Keeping the pip bootstrapping as a separate step for ``make``-based
installations should minimize the changes CPython redistributors need to installations should minimize the changes CPython redistributors need to
make to their build processes. Avoiding the layer of indirection through make to their build processes. Avoiding the layer of indirection through
``make`` for the ``extractpip`` invocation avoids any challenges ``make`` for the ``ensurepip`` invocation avoids any challenges
associated with determining where to install the extracted ``pip``. associated with determining where to install the extracted ``pip``.
@ -372,7 +388,7 @@ but under a new "Invoking distutils directly" subsection.
Bundling CA certificates with CPython Bundling CA certificates with CPython
------------------------------------- -------------------------------------
The ``extractpip`` implementation will include the ``pip`` CA bundle along The ``ensurepip`` implementation will include the ``pip`` CA bundle along
with the rest of ``pip``. This means CPython effectively includes with the rest of ``pip``. This means CPython effectively includes
a CA bundle that is used solely by ``pip`` after it has been extracted. a CA bundle that is used solely by ``pip`` after it has been extracted.
@ -392,8 +408,8 @@ work will be complete for pip 1.5 (which is the version likely to be current
when Python 3.4.0 is released). when Python 3.4.0 is released).
This PEP proposes that, if pip still requires it as a dependency, This PEP proposes that, if pip still requires it as a dependency,
``extractpip`` will include a private copy of ``setuptools`` (in addition ``ensurepip`` will include a private copy of ``setuptools`` (in addition
to the private copy of ``extractpip``). ``python -m extractpip`` will then to the private copy of ``ensurepip``). ``python -m ensurepip`` will then
install the private copy in addition to installing ``pip`` itself. install the private copy in addition to installing ``pip`` itself.
However, this behavior is officially considered an implementation However, this behavior is officially considered an implementation
@ -403,14 +419,14 @@ provide an appropriate dependency declaration, rather than assuming
Once pip is able to run ``pip install --upgrade pip`` without needing Once pip is able to run ``pip install --upgrade pip`` without needing
``setuptools`` installed first, then the private copy of ``setuptools`` ``setuptools`` installed first, then the private copy of ``setuptools``
will be removed from ``extractpip`` in subsequent CPython releases. will be removed from ``ensurepip`` in subsequent CPython releases.
Updating the private copy of pip Updating the private copy of pip
-------------------------------- --------------------------------
In order to keep up with evolutions in packaging as well as providing users In order to keep up with evolutions in packaging as well as providing users
with as recent version a possible the ``extractpip`` module will be with as recent version a possible the ``ensurepip`` module will be
regularly updated to the latest versions of everything it bootstraps. regularly updated to the latest versions of everything it bootstraps.
After each new ``pip`` release, and again during the preparation for any After each new ``pip`` release, and again during the preparation for any
@ -419,10 +435,10 @@ of this PEP, will be run to ensure the private copies stored in the CPython
source repository have been updated to the latest versions. source repository have been updated to the latest versions.
Updating the extractpip module API and CLI Updating the ensurepip module API and CLI
------------------------------------------ -----------------------------------------
Like ``venv`` and ``pyvenv``, the ``extractpip`` module API and CLI Like ``venv`` and ``pyvenv``, the ``ensurepip`` module API and CLI
will be governed by the normal rules for the standard library: no will be governed by the normal rules for the standard library: no
new features are permitted in maintenance releases. new features are permitted in maintenance releases.
@ -450,10 +466,10 @@ than only those with the freedom to adopt Python 3.4 as soon as it is
released. released.
Open Question: Uninstallation Uninstallation
============================= ==============
No changes are currently proposed to the uninstallation process. The No changes are proposed to the uninstallation process by this PEP. The
bootstrapped pip will be installed the same way as any other pip bootstrapped pip will be installed the same way as any other pip
installed packages, and will be handled in the same way as any other installed packages, and will be handled in the same way as any other
post-install additions to the Python environment. post-install additions to the Python environment.
@ -462,20 +478,13 @@ At least on Windows, that means the bootstrapped files will be
left behind after uninstallation, since those files won't be associated left behind after uninstallation, since those files won't be associated
with the Python MSI installer. with the Python MSI installer.
.. note:: While the case can be made for the CPython installers clearing out these
directories automatically, changing that behaviour is considered outside
Perhaps the installer should be updated to clobber everything in the scope of this PEP.
site-packages and the Scripts directory when uninstalled (treating them
as "data directories" from Python's point of view), but I would prefer
not to make this PEP conditional on that change.
Open Question: Script Execution on Windows Script Execution on Windows
========================================== ===========================
.. note::
Perhaps this question should be separated out from the PEP?
While the Windows installer was updated in Python 3.3 to optionally While the Windows installer was updated in Python 3.3 to optionally
make ``python`` available on the PATH, no such change was made to make ``python`` available on the PATH, no such change was made to
@ -539,7 +548,9 @@ downstream distributors:
installing the separate pip package when a user executes ``pip`` without installing the separate pip package when a user executes ``pip`` without
it being installed. Systems that choose this option should ensure that it being installed. Systems that choose this option should ensure that
the ``pyvenv`` command still installs pip into the virtual environment the ``pyvenv`` command still installs pip into the virtual environment
by default. by default, but may modify the ``ensurepip`` module in the system Python
installation to redirect to the platform provided mechanism when
installing ``pip`` globally.
* Do not remove the bundled copy of pip. * Do not remove the bundled copy of pip.
@ -549,8 +560,8 @@ downstream distributors:
downstream distributors have already made exception to the common downstream distributors have already made exception to the common
"debundling" policy. "debundling" policy.
* This does mean that if ``pip`` needs to be updated due to a security * This does mean that if ``pip`` needs to be updated due to a security
issue, so does the bundled version in the ``extractpip`` bootstrap module issue, so does the private copy in the ``ensurepip`` bootstrap module
* However, altering the bundled version of pip to remove the embedded * However, altering the private copy of pip to remove the embedded
CA certificate bundle and rely on the system CA bundle instead is a CA certificate bundle and rely on the system CA bundle instead is a
reasonable change. reasonable change.
@ -564,14 +575,13 @@ downstream distributors:
made to the redistributed version of Python. made to the redistributed version of Python.
* Checking the version of pip that will be bootstrapped using * Checking the version of pip that will be bootstrapped using
``python -m extractpip --version`` or ``extractpip.version()``. ``python -m ensurepip --version`` or ``ensurepip.version()``.
* Installation of the bundled version of pip into a global or virtual * Installation of pip into a global or virtual python environment using
python environment using ``python -m extractpip`` or ``python -m ensurepip`` or ``ensurepip.bootstrap()``.
``extractpip.bootstrap()``.
* ``pip install --upgrade pip`` in a global installation should not affect * ``pip install --upgrade pip`` in a global installation should not affect
any already created virtual environments (but is permitted to affect any already created virtual environments (but is permitted to affect
future virtual environments, even though it will not do so in the future virtual environments, even though it will not do so when using
normal case). the upstream version of ``ensurepip``).
* ``pip install --upgrade pip`` in a virtual environment should not affect * ``pip install --upgrade pip`` in a virtual environment should not affect
the global installation. the global installation.
@ -602,7 +612,7 @@ and other related projects.
Backwards Compatibility Backwards Compatibility
----------------------- -----------------------
The public API and CLI of the ``extractpip`` module itself will fall under The public API and CLI of the ``ensurepip`` module itself will fall under
the typical backwards compatibility policy of Python for its standard the typical backwards compatibility policy of Python for its standard
library. The externally developed software that this PEP bundles does not. library. The externally developed software that this PEP bundles does not.
@ -614,7 +624,7 @@ its own 6 month release cycle rather than CPython's 18-24 month cycle.
Security Releases Security Releases
----------------- -----------------
Any security update that affects the ``extractpip`` module will be shared Any security update that affects the ``ensurepip`` module will be shared
prior to release with the Python Security Response Team prior to release with the Python Security Response Team
(security@python.org). The PSRT will then decide if the reported issue (security@python.org). The PSRT will then decide if the reported issue
warrants a security release of CPython with an updated private copy of warrants a security release of CPython with an updated private copy of
@ -692,13 +702,18 @@ tracker to use, but hopefully less than has been seen historically when
including complete public copies of third-party projects in the standard including complete public copies of third-party projects in the standard
library. library.
Finally, the approach described in this PEP avoids some technical issues The approach described in this PEP also avoids some technical issues
related to handle CPython maintenance updates when pip has been independently related to handling CPython maintenance updates when pip has been
updated to a more recent version. The proposed pip-based bootstrapping independently updated to a more recent version. The proposed pip-based
mechanism handles that automatically, since pip and the system installer bootstrapping mechanism handles that automatically, since pip and the
never get into a fight about who owns the pip installation (it is always system installer never get into a fight about who owns the pip
managed through pip, either directly, or indirectly via the getpip bootstrap installation (it is always managed through pip, either directly, or
module). indirectly via the ``ensurepip`` bootstrap module).
Finally, the separate bootstrapping step means it also easy to avoid
installing ``pip`` at all if end users so desire. This is often the case
if integrators are using system packages to handle installation of
components written in multiple languages using a common set of tools.
Defaulting to --user installation Defaulting to --user installation
@ -727,6 +742,17 @@ References
.. [#homebrew] `Homebrew <http://brew.sh/>` .. [#homebrew] `Homebrew <http://brew.sh/>`
.. [#conda] `Conda <http://www.continuum.io/blog/conda>` .. [#conda] `Conda <http://www.continuum.io/blog/conda>`
.. [1] Discussion thread 1 (distutils-sig)
(https://mail.python.org/pipermail/distutils-sig/2013-August/022529.html)
.. [2] Discussion thread 2 (distutils-sig)
(https://mail.python.org/pipermail/distutils-sig/2013-September/022702.html)
.. [3] Discussion thread 3 (python-dev)
(https://mail.python.org/pipermail/python-dev/2013-September/128723.html)
.. [4] Discussion thread 4 (python-dev)
(https://mail.python.org/pipermail/python-dev/2013-September/128780.html)
Copyright Copyright
========= =========