python-peps/pep-0439.txt

PEP: 439
Title: Inclusion of pip bootstrap in Python installation
Version: $Revision$
Last-Modified: $Date$
Author: Richard Jones <richard@python.org>
BDFL-Delegate:  Nick Coghlan <ncoghlan@gmail.com>
Discussions-To: <distutils-sig@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 18-Mar-2013
Python-Version: 3.4
Post-History: 19-Mar-2013


Abstract
========

This PEP proposes the inclusion of a pip boostrap executable in the
Python installation to simplify the use of 3rd-party modules by Python
users.

This PEP does not propose to include the pip implementation in the
Python standard library.  Nor does it propose to implement any package
management or installation mechanisms beyond those provided by PEP
427 ("The Wheel Binary Package Format 1.0") and TODO distlib PEP.


Rationale
=========

Currently the user story for installing 3rd-party Python modules is
not as simple as it could be.  It requires that all 3rd-party modules
inform the user of how to install the installer, typically via a link
to the installer.  That link may be out of date or the steps required
to perform the install of the installer may be enough of a roadblock
to prevent the user from further progress.

Large Python projects which emphasise a low barrier to entry have
shied away from depending on third party packages because of the
introduction of this potential stumbling block for new users.

With the inclusion of the package installer command in the standard
Python installation the barrier to installing additional software is
considerably reduced.  It is hoped that this will therefore increase
the likelihood that Python projects will reuse third party software.

The Python community also has an issue of complexity around the current
bootstrap procedure for pip and setuptools. They all have
their own bootstrap download file with slightly different usages and
even refer to each other in some cases. Having a single bootstrap which
is common amongst them all, with a simple usage, would be far preferable.

It is also hoped that this is reduces the number of proposals to
include more and more software in the Python standard library, and
therefore that more popular Python software is more easily upgradeable
beyond requiring Python installation upgrades.


Proposal
========

The bootstrap will install the pip implementation, setuptools by downloading
their installation files from PyPI.

This proposal affects two components of packaging: `the pip bootstrap`_ and,
thanks to easier package installation, `modifications to publishing
packages`_.

The core of this proposal is that the user experience of using pip should not
require the user to install pip.


The pip bootstrap
-----------------

The Python installation includes an executable called "pip3" (see PEP 394 for
naming rationale etc.) that attempts to import pip machinery.  If it can then
the pip command proceeds as normal.  If it cannot it will bootstrap pip by
downloading the pip implementation and setuptools wheel files. Hereafter the
installation of the "pip implementation" will imply installation of setuptools
and virtualenv.  Once installed, the pip command proceeds as normal. Once the
bootstrap process is complete the "pip3" command is no longer the bootstrap
but rather the full pip command.

A boostrap is used in the place of a the full pip code so that we don't have
to bundle pip and also pip is upgradeable outside of the regular Python
upgrade timeframe and processes.

To avoid issues with sudo we will have the bootstrap default to
installing the pip implementation to the per-user site-packages
directory defined in PEP 370 and implemented in Python 2.6/3.0.  Since
we avoid installing to the system Python we also avoid conflicting
with any other packaging system (on Linux systems, for example.) If
the user is inside a virtual environment [1]_ then the pip
implementation will be installed into that virtual environment.

The bootstrap process will proceed as follows:

1. The user system has Python (3.4+) installed.  In the "scripts"
   directory of the Python installation there is the bootstrap script
   called "pip3".
2. The user will invoke a pip command, typically "pip3 install
   <package>", for example "pip3 install Django".
3. The boostrap script will attempt to import the pip implementation.
   If this succeeds, the pip command is processed normally. Stop.
4. On failing to import the pip implementation the bootstrap notifies
   the user that it needs to "install pip". It will ask the user whether it
   should install pip as a system-wide site-packages or as a user-only
   package. This choice will also be present as a command-line option to pip
   so non-interactive use is possible.
5. The bootstrap will and contact PyPI to obtain the latest download wheel
   file (see PEP 427.)
6. Upon downloading the file it is installed using "python setup.py install".
7. The pip tool may now import the pip implementation and continues to
   process the requested user command normally.

Users may be running in an environment which cannot access the public
Internet and are relying solely on a local package repository.  They
would use the "-i" (Base URL of Python Package Index) argument to the
"pip3 install" command. This simply overrides the default index URL pointing
to PyPI.

Some users may have no Internet access suitable for fetching the pip
implementation file. These users can manually download and install the
setuptools and pip tar files. Adding specific support for this use-case is
unnecessary.

The download of the pip implementation install file will be performed
securely.  The transport from pypi.python.org will be done over HTTPS with the
CA certificate check performed. This facility will be present in Python 3.4+
using Operating System certificates (see PEP XXXX).

Beyond those arguments controlling index location and download
options, the "pip3" boostrap command may support further standard pip
options for verbosity, quietness and logging.

The "pip3" command will support two new command-line options that are used
in the boostrapping, and otherwise ignored. They control where the pip
implementation is installed:

--bootstrap
  Install to the user's packages directory. The name of this option is chosen
  to promote it as the preferred installation option.

--bootstrap-to-system
  Install to the system site-packages directory.

These command-line options will also need to be implemented, but otherwise
ignored, in the pip implementation.

Consideration should be given to defaulting pip to install packages to the
user's packages directory if pip is installed in that location.

The "--no-install" option to the "pip3" command will not affect the
bootstrapping process.


Modifications to publishing packages
------------------------------------

An additional new Python package is proposed, "pypublish", which
will be a tool for publishing packages to PyPI.  It would replace the
current "python setup.py register" and "python setup.py upload"
distutils commands.  Again because of the measured Python release
cycle and extensive existing Python installations these commands are
difficult to bugfix and extend.  Additionally it is desired that the
"register" and "upload" commands be able to be performed over HTTPS
with certificate validation.  Since shipping CA certificate keychains
with Python is not really feasible (updating the keychain is quite
difficult to manage) it is desirable that those commands, and the
accompanying keychain, be made installable and upgradeable outside of
Python itself.

The existing distutils mechanisms for package registration and upload would
remain, though with a deprecation warning.


Implementation
==============

The changes to pip required by this PEP are being tracked in that project's
issue tracker [2]_. Most notably, the addition of --bootstrap and --bootstrap-
to-system to the pip command-line.

It would be preferable that the pip and setuptools projects distribute a wheel
format download.

The required code for this implementation is the "pip3" command described
above. The additional pypublish can be developed outside of the scope of this
PEP's work.

Finally, it would be desirable that "pip3" be ported to Python 2.6+ to allow
the single command to replace existing pip, setuptools and virtualenv (which
would be added to the bootstrap) bootstrap scripts. Having that bootstrap
included in a future Python 2.7 release would also be highly desirable.


Risks
=====

The key that is used to sign the pip implementation download might be
compromised and this PEP currently proposes no mechanism for key
revocation.

There is a Perl package installer also named "pip". It is quite rare and not
commonly used. The Fedora variant of Linux has historically named Python's
"pip" as "python-pip" and Perl's "pip" as "perl-pip".  This policy has been
altered[3] so that future and upgraded Fedora installations will use the name
"pip" for Python's "pip". Existing (non-upgraded) installations will still
have the old name for the Python "pip", though the potential for confusion is
now much reduced.


References
==========

.. [1] PEP 405, Python Virtual Environments
       http://www.python.org/dev/peps/pep-0405/

.. [2] pip issue tracking work needed for this PEP
       https://github.com/pypa/pip/issues/863

.. [3] Fedora's python-pip package does not provide /usr/bin/pip
       https://bugzilla.redhat.com/show_bug.cgi?id=958377


Acknowledgments
===============

Nick Coghlan for his thoughts on the proposal and dealing with the Red
Hat issue.

Jannis Leidel and Carl Meyer for their thoughts. Marcus Smith for feedback.

Marcela Mašláňová for resolving the Fedora issue.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: