iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Additional rationale in PEP 453 

I realised after the last round of reviews that a *big* chunk of the
rationale for this PEP was assumed knowledge on distutils-sig, and
this lead directly to the high rate of replies on python-dev that
just didn't get why the requested exemption to include the new
feature in Python 2.7.6 is so important.

The PEP now attempts to do a better job of bridging that gap, as
well as further expanding on how having the ``ensurepip`` module
fully public in older releases should actually make things *less*
confusing, rather than more, since it gives us a location to
explain the alternative fallback bootstrapping methods.
This commit is contained in:
Nick Coghlan 2013-09-29 15:15:43 +10:00
parent ca2574f9d8
commit 75dcba1791
1 changed files with 181 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

207
pep-0453.txt
View File

@ -26,12 +26,14 @@ of that recommendation.
Rationale
=========
This change is considered a necessary step in the evolution of the Python
packaging ecosystem, as the community has embraced the Python Package Index
as a mechanism for distributing and installing Python software, but the
different concerns of language evolution and secure software distribution
mean that a faster feature release cycle is needed to properly support the
latter.
There are two related, but distinct rationales for the proposal in this
PEP. The first relates to the experience of new users, while the second
relates to better enabling the evolution of the broader Python packaging
ecosystem.
Improving the new user experience
---------------------------------
Currently, on systems without a platform package manager and repository,
installing a third-party Python package into a freshly installed Python
@ -130,6 +132,114 @@ provides a reasonably secure, cross platform mechanism to get access to
these utilities.
Enabling the evolution of the broader Python packaging ecosystem
----------------------------------------------------------------
As no new packaging standard can achieve widespread adoption without a
transition strategy that covers the versions of Python that are in
widespread *current* use (rather than merely future versions, like most
language features), the change proposed in this PEP is considered a
necessary step in the evolution of the Python packaging ecosystem
The broader community has embraced the Python Package Index as a mechanism
for distributing and installing Python software, but the different concerns
of language evolution and secure software distribution mean that a faster
feature release cycle that encompasses older versions is needed to properly
support the latter.
In addition, the core CPython development team have the luxury of
dropping support for earlier Python versions well before the rest of the
community, as downstream commercial redistributors pick up the task of
providing support for those versions to users that still need it, while
many third party libraries maintain compatibility with those versions as
long as they remain in widespread use.
This means that the current ``setup.py install`` based model for package
installation poses serious difficulties for the development and adoption
of new packaging standards, as, depending on how a project writes their
``setup.py`` file, the installation command (along with other operations)
may end up invoking the standard library's ``distutils`` package.
As an indicator of how this may cause problems for the broader ecosystem,
consider that the feature set of ``distutils`` in Python 2.6 was frozen
in June 2008 (with the release of Python 2.6b1), while the feature set of
``distutils`` in Python 2.7 was frozen in April 2010 (with the release of
Python 2.7b1).
By contrast, using a separate installer application like ``pip`` (which
ensures that even ``setup.py`` files that invoke ``distutils`` directly
still support the new packaging standards) makes it possible to support
new packaging standards in older versions of Python, just by upgrading
``pip``. The situation on older versions of Python is further improved by
making it easier for end users to install and upgrade newer build systems
like ``setuptools`` or improved PyPI upload utilities like ``twine``.
It is not coincidental that this proposed model of using a separate installer
program with more metadata heavy and less active distribution formats matches
that used by most operating systems (including Windows since the introduction
of the installer service and the MSI file format), as well as many other
language specific installers.
For Python 2.6, this compatibility issue is largely limited to various
enterprise Linux distributions (and their downstream derivatives). These
distributions often have even slower update cycles than CPython, so they
offer full support for versions of Python that are considered "security
fix only" versions upstream (and sometimes may even be to the point where
the core development team no longer support them at all - you can still get
commercial support for Python 2.3 if you really need it!).
In practice, the fact that tools like ``wget`` and ``curl`` are readily
available on Linux systems, that most users of Python on Linux are
already familiar with the command line, and that most Linux distributions
ship with a default configuration that makes running Python scripts easy,
means that the existing ``pip`` bootstrapping instructions for any \*nix
system are already quite straightforward. Even if ``pip`` isn't provided by
the system package manager, then using ``wget`` or ``curl`` to retrieve the
bootstrap script from www.pip-installer.org and then running it is just a
couple of shell commands that can easily be copied and pasted as necessary.
Accordingly, for any version of Python on any \*nix system, the need to
bootstrap ``pip`` in older versions isn't considered a major barrier to
adoption of new packaging standards, since it's just one more small
speedbump encountered by users of these long term stable releases. For
\*nix systems, this PEP's formal endorsement of ``pip`` as the preferred
default packaging tool is seen as more important than the underlying
technical details involved in making ``pip`` available by default, since
it shifts the nature of the conversation between the developers of ``pip``
and downstream repackagers of both ``pip`` and CPython.
For Python 2.7, on the other hand, the compatibility issue for adopting new
metadata standards is far more widespread, as it affects the python.org
binary installers for Windows and Mac OS X, as well as even relatively
fast moving \*nix platforms.
Firstly, and unlike Python 2.6, Python 2.7 is still a fully supported
upstream version, and will remain so until the release of Python 2.7.9
(currently scheduled for May 2015), at which time it is expected to enter
the usual "security fix only" mode. That means there are at least another
19 months where Python 2.7 is a deployment target for Python applications
that enjoys full upstream support. Even after the core development team
switches 2.7 to security release only mode in 2015, Python 2.7 will likely
remain a commercially supported legacy target out beyond 2020.
While Python 3 already presents a compelling alternative over Python 2 for
*new* Python applications and deployments without an existing investment
in Python 2 and without a dependency on specific Python 2 only third party
modules (a set which is getting ever smaller over time), it is going to take
longer to create compelling business cases to update existing Python 2.7
based infrastructure to Python 3, especially in situations where the culture
of automated testing is weak (or nonexistent), making it difficult to
effectively use the available migration utilities.
It is quite likely that it is this difference in perspective regarding what
it means for a version of Python to be "supported" which lies at the heart
of the long history of conflicts between the developers of Python packaging
tools and the core development team for CPython. A key goal of this PEP is
thus to better enable the two groups to collaborate more effectively, by
using the ``ensurepip`` module as the technical bridge between the two
distinct software lifecycles and deployment models.
Why pip?
--------
@ -170,7 +280,7 @@ utility).
To support that end, this PEP proposes the inclusion of an ``ensurepip``
bootstrapping module in Python 3.4 (along with the upcoming maintenance
releases of Python 2.7 and 3.3), as well as changes to the way Python
installed scripts are handled on Windows.
installed scripts are handled on Windows (for Python 3.4 only).
To clearly demarcate development responsibilities, and to avoid
inadvertently downgrading ``pip`` when updating CPython, the proposed
@ -179,10 +289,10 @@ mechanism in the standard library that is invoked automatically by the
CPython installers provided on python.org.
To ensure the smoothest possible experience for new users of Python, this
PEP also proposes that the installer changes and the ``ensurepip`` module
be backported to Python 2.7 and 3.3. It does *not* propose backporting any
changes to ``pyvenv`` (in Python 3.3) or to Windows script handling (in
either version).
PEP also proposes that the ``ensurepip`` module and the option to install
``pip`` when installing CPython be backported to Python 2.7 and 3.3. It
does *not* propose backporting any changes to ``pyvenv`` (in Python 3.3)
or to Windows script handling (in either version).
Finally, the PEP also strongly recommends that CPython redistributors and
other Python implementations ensure that ``pip`` is available by default, or
@ -436,7 +546,26 @@ updated to reference the revised module installation guide.
In Python 2.7 and 3.3, the documentation will make clear that the feature
was added in a maintenance release and users may need to upgrade in order
to take advantage of it.
to take advantage of it. Specifically, it is proposed to include the
following warning as a note in the documentation for the ``ensurepip``
module in these versions (adjust version numbers for 3.3 as appropriate):
This is an optional module, which may not be available in all
installations of Python 2.7. It is provided solely to simplify the
process of bootstrapping ``pip`` onto end user's systems. If it is
not available, please investigate the following alternatives:
* This module was first added in Python 2.7.6. If using an earlier
maintenance release, it will not be available. If upgrading to
a more recent maintenance release is not an option, consider
the alternative bootstrapping mechanisms below.
* Some platforms provide alternative mechanisms to obtain ``pip``.
In such cases, the platform documentation should provide
appropriate details.
* If upgrading to the latest maintenance release is not feasible, and
no platform specific instructions are provided, then refer to the
upstream `pip bootstrapping instructions
<http://www.pip-installer.org/en/latest/installing.html>`__.
The existing content of the module installation guide will be retained in
all versions, but under a new "Invoking distutils directly" subsection.
@ -523,8 +652,8 @@ maintenance releases". The rationale for doing so in this case is slightly
different for each of the two versions.
Rationale for the policy
------------------------
Rationale for this policy on maintenance releases
-------------------------------------------------
Python's strict "no new features in maintenance releases" was instituted
following the introduction of a number of new features over the course of
@ -574,12 +703,20 @@ obtaining ``pip`` for users of Python 2.7 and 3.3 on Windows and Mac OS X
systems, all of the existing ``pip`` bootstrapping mechanisms will still
work in cases where upgrading Python isn't a practical alternative.
As described in the documentation update proposal, the ``ensurepip``
documentation in older releases will include the text explaining how to
obtain ``pip`` if updating to the latest maintenance release isn't an
option, or if the module has been removed by a redistributor. This contrasts
significantly with the changes made during the Python 2.2 series, where they
were normal additions with no alternatives except to update to a sufficiently
recent version of Python if a library or application depended on them.
Potential consequences of permitting this exemption
---------------------------------------------------
There is some legitimate concern that approving an exemption to the
"no new features in maintenance releases" policy in this case will open the
The concern has been expressed that approving an exemption to the "no new
features in maintenance releases" policy in this case will open the
flood gates to requests for more such exemptions in the future. It is the
perspective of the PEP authors that the specific nature of this proposal
should itself serve to allay those fears.
@ -615,9 +752,10 @@ libraries in the common Python 2/3 subset means there are (as of
September 2013) still more Python 2 only libraries than there are Python 3
only libraries.
Since the primary purpose of this PEP is to aid new Python users, it is
Since one of the primary aims of this PEP is to aid new Python users, it is
contrary to its spirit to target *only* Python 3.4, when so many users in
at least the next 12-18 months are still going to be introduced to Python
at least the next 12-18 months (where Python 2.7 is still fully supported
by the core development team) are still going to be introduced to Python
2 before being introduced to Python 3.
Users first installing Python 2.7 on Windows and Mac OS X following
@ -628,9 +766,22 @@ beginning to explore the PyPI ecosystem, the bootstrapping instructions
can be simplified to "just install the latest maintenance release of
CPython".
Making ``pip`` readily available also serves to improve the usability of
Python 2.7 itself, as a number of the standard library additions in Python
3 are also available from PyPI for Python 2.
Making ``pip`` readily available also serves to ease the migration path
from Python 2 to Python 3, as a number of the standard library additions
in Python 3 are also available from PyPI for Python 2. Lowering the barrier
to adoption for these backports makes it easier for current Python 2 users
to selectively adopt backporting Python 3 versions, reducing the number of
updates needed in any eventual Python 3 migration.
Finally, this PEP solves a serious problem for the ``distutils-sig``
community, as it means we will finally have a standard mechanism decoupled
from the standard library's development lifecycle that we can reasonably
assume to be present on end user's systems (or at least readily
available) that allows us to support new packaging standards in older
versions of Python. A tentative, half-hearted endorsement from the
CPython core development team that tries to hide the existence of the
pip boostrapping support from end users is unlikely to provide quite the
same benefits.
Rationale for permitting the exemption in Python 3.3
@ -788,9 +939,8 @@ downstream distributors:
* Migrate build systems to utilize `pip`_ and `Wheel`_ wherever feasible
and avoid directly invoking ``setup.py``.
* This isn't strictly required by this PEP, but will help ensure a
smoother and more timely adoption of improved metadata formats as the
Python packaging ecosystem continues to evolve.
* This will help ensure a smoother and more timely migration to improved
metadata formats as the Python packaging ecosystem continues to evolve.
In the event that a Python redistributor chooses *not* to follow these
recommendations, we request that they explicitly document this fact and
@ -908,8 +1058,8 @@ Naming the module `_ensurepip`` in Python 2.7 and 3.3 was considered as
another means of skirting the "no new features in maintenance releases"
policy. However, similar to the proposal to only include the new
feature in the installers rather than the standard library, this feels like
relying on a technicality to "comply" with the policy, while still breaking
it in spirit.
relying on a technicality to nominally "comply" with the policy, while still
breaking it in spirit.
It is the considered opinion of the PEP authors that attempting to hide
the addition of the ``ensurepip`` module in earlier versions will only
@ -917,6 +1067,11 @@ serve to increase confusion rather than to reduce it, so the proposal
remains to be up front about the fact that the policy is being broken in
this case, and clearly documenting the rationale for doing so in this PEP.
As noted in the section describing the proposed documentation updates,
having ``ensurepip`` as a public module in these earlier versions also
provides a convenient home for the fallback bootstrapping instructions in
those cases where it *isn't* available.
Automatically contacting PyPI when bootstrapping pip
----------------------------------------------------