Add PEP 413 as a competitor to PEP 407
This commit is contained in:
Nick Coghlan
parent
05da204481
commit
cbfb3afdb3
|
|
@ -0,0 +1,371 @@
|
||||||
|
PEP: 413
|
||||||
|
Title: Faster evolution of the Python Standard Library
|
||||||
|
Version: $Revision$
|
||||||
|
Last-Modified: $Date$
|
||||||
|
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Process
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 2012-02-24
|
||||||
|
Post-History: 2012-02-24
|
||||||
|
Resolution: TBD
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
This PEP proposes the adoption of a new date-based versioning scheme for
|
||||||
|
the standard library (distinct from, but coupled to, the existing language
|
||||||
|
versioning scheme) that allows accelerated releases of the Python standard
|
||||||
|
library, while maintaining (or even slowing down) the current rate of
|
||||||
|
change in the core language definition.
|
||||||
|
|
||||||
|
Like PEP 407, it aims to adjust the current balance between measured
|
||||||
|
change that allows the broader community time to adapt and being able to
|
||||||
|
keep pace with external influences that evolve more rapidly than the current
|
||||||
|
release cycle can handle (this problem is particularly notable for
|
||||||
|
standard library elements that relate to web technologies).
|
||||||
|
|
||||||
|
However, it's more conservative in its aims than PEP 407, seeking to
|
||||||
|
restrict the increased pace of development to builtin and standard library
|
||||||
|
interfaces, without affecting the rate of change for other elements such
|
||||||
|
as the language syntax and version numbering as well as the CPython
|
||||||
|
binary API and bytecode format.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
To quote the PEP 407 abstract:
|
||||||
|
|
||||||
|
Finding a release cycle for an open-source project is a delicate exercise
|
||||||
|
in managing mutually contradicting constraints: developer manpower,
|
||||||
|
availability of release management volunteers, ease of maintenance for
|
||||||
|
users and third-party packagers, quick availability of new features (and
|
||||||
|
behavioural changes), availability of bug fixes without pulling in new
|
||||||
|
features or behavioural changes.
|
||||||
|
|
||||||
|
The current release cycle errs on the conservative side. It is adequate
|
||||||
|
for people who value stability over reactivity. This PEP is an attempt to
|
||||||
|
keep the stability that has become a Python trademark, while offering a
|
||||||
|
more fluid release of features, by introducing the notion of long-term
|
||||||
|
support versions.
|
||||||
|
|
||||||
|
I agree with the PEP 407 authors that the current release cycle of the
|
||||||
|
*standard library* is too slow to effectively cope with the pace of change
|
||||||
|
in some key programming areas (specifically, web protocols and related
|
||||||
|
technologies, including databases, templating and serialisation formats).
|
||||||
|
|
||||||
|
However, I have written this competing PEP because I believe that the
|
||||||
|
approach proposed in PEP 407 of offering full, potentially binary
|
||||||
|
incompatible releases of CPython every 6 months places too great a burden
|
||||||
|
on the wider Python ecosystem.
|
||||||
|
|
||||||
|
Under the current CPython release cycle, distributors of key binary
|
||||||
|
extensions will often support Python releases even after the CPython branches
|
||||||
|
enter "security fix only" mode (for example, Twisted currently ships binaries
|
||||||
|
for 2.5, 2.6 and 2.7, NumPy and SciPy suport those 3 along with 3.1 and 3.2,
|
||||||
|
PyGame adds a 2.4 binary release, wxPython provides both 32-bit and 64-bit
|
||||||
|
binaries for 2.6 and 2.7, etc).
|
||||||
|
|
||||||
|
If CPython were to triple (or more) its rate of releases, the developers of
|
||||||
|
those libraries (many of which are even more resource starved than CPython)
|
||||||
|
would face an unpalatable choice: either adopt the faster release cycle
|
||||||
|
themselves (up to 18 simultaneous binary releases for PyGame!), drop
|
||||||
|
older Python versions more quickly, or else tell their users to stick to the
|
||||||
|
CPython LTS releases (thus defeating the entire point of speeding up the
|
||||||
|
CPython release cycle in the first place).
|
||||||
|
|
||||||
|
Similarly, many support tools for Python (e.g. syntax highlighters) can take
|
||||||
|
quite some time to catch up with language level changes.
|
||||||
|
|
||||||
|
At a cultural level, the Python community is also accustomed to a certain
|
||||||
|
meaning for Python version numbers - they're linked to deprecation periods,
|
||||||
|
support periods, all sorts of things. PEP 407 proposes that collective
|
||||||
|
knowledge all be swept aside, without offering a compelling rationale for why
|
||||||
|
such a course of action is actually *necessary* (aside from, perhaps, making
|
||||||
|
the lives of the CPython core developers a little easier at the expense of
|
||||||
|
everyone else).
|
||||||
|
|
||||||
|
But, if we go back to the primary rationale for increasing the pace of change
|
||||||
|
(i.e. more timely support for web protocols and related technologies), we can
|
||||||
|
note that those only require *standard library* changes. That means many
|
||||||
|
(perhaps even most) of the negative effects on the wider community can be
|
||||||
|
avoided by explicitly limiting which parts of CPython are affected by the
|
||||||
|
new release cycle, and allowing other parts to evolve at their current, more
|
||||||
|
sedate, pace.
|
||||||
|
|
||||||
|
|
||||||
|
Proposal
|
||||||
|
========
|
||||||
|
|
||||||
|
This PEP proposes the addition of a new ``sys.stdlib_info`` attribute that
|
||||||
|
records a date based standard library version above and beyond the underlying
|
||||||
|
interpreter version::
|
||||||
|
|
||||||
|
sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0)
|
||||||
|
|
||||||
|
This information would also be included in the ``sys.version`` string::
|
||||||
|
|
||||||
|
Python 3.3.0 (12.08.0, default:c1a07c8092f7+, Feb 17 2012, 23:03:41)
|
||||||
|
[GCC 4.6.1]
|
||||||
|
|
||||||
|
When maintenance releases are created, *two* new versions of Python would
|
||||||
|
actually be published on python.org (using the first 3.3 maintenance release,
|
||||||
|
planned for February 2013 as an example)::
|
||||||
|
|
||||||
|
3.3.1 + 12.08.1 # Maintenance release
|
||||||
|
3.3.1 + 13.02.0 # Standard library release
|
||||||
|
|
||||||
|
A standard library release would just be the corresponding maintenance
|
||||||
|
release, with the following additional, backwards compatible changes:
|
||||||
|
|
||||||
|
* new features in pure Python modules
|
||||||
|
* new features in C extension modules (subject to PEP 399 compatibility
|
||||||
|
requirements)
|
||||||
|
* new features in language builtins (provided the C ABI remains unaffected)
|
||||||
|
|
||||||
|
A further 6 months later, the next 3.3 maintenance release would again be
|
||||||
|
accompanied by a new standard library release::
|
||||||
|
|
||||||
|
3.3.2 + 12.08.2 # Maintenance release
|
||||||
|
3.3.2 + 13.08.1 # Standard library release
|
||||||
|
|
||||||
|
Again, the standard library release would be binary compatible with the
|
||||||
|
previous language release, merely offering additional features at the
|
||||||
|
Python level.
|
||||||
|
|
||||||
|
Finally, 18 months after the release of 3.3, a new language release would
|
||||||
|
be made around the same time as the final 3.3 maintenance release:
|
||||||
|
|
||||||
|
3.3.3 + 12.08.3 # Maintenance release
|
||||||
|
3.4.0 + 14.02.0 # Language release
|
||||||
|
|
||||||
|
Language releases would then contain all the features that are not
|
||||||
|
permitted in standard library releases:
|
||||||
|
|
||||||
|
* new language syntax
|
||||||
|
* new deprecation warnings
|
||||||
|
* removal of previously deprecated features
|
||||||
|
* changes to the emitted bytecode
|
||||||
|
* changes to the AST
|
||||||
|
* any other significant changes to the compilation toolchain
|
||||||
|
* changes to the C ABI
|
||||||
|
|
||||||
|
The 3.4 release cycle would then follow a similar pattern to that for 3.3::
|
||||||
|
|
||||||
|
3.4.1 + 14.02.1 # Maintenance release
|
||||||
|
3.4.1 + 14.08.0 # Standard library release
|
||||||
|
3.4.2 + 14.02.2 # Maintenance release
|
||||||
|
3.4.2 + 15.02.0 # Standard library release
|
||||||
|
3.4.3 + 14.02.3 # Maintenance release
|
||||||
|
3.5.0 + 15.08.0 # Language release
|
||||||
|
|
||||||
|
|
||||||
|
Effects
|
||||||
|
=======
|
||||||
|
|
||||||
|
Effect on development cycle
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Similar to PEP 407, this PEP will break up the delivery of new features into
|
||||||
|
more discrete chunks. Instead of whole raft of changes landing all at once
|
||||||
|
in a language release, each language release will be limited to 6 months
|
||||||
|
worth of standard library changes, as well as any changes associated with
|
||||||
|
new syntax.
|
||||||
|
|
||||||
|
Effect on workflow
|
||||||
|
------------------
|
||||||
|
|
||||||
|
This PEP proposes the creation of a single additional branch for use in the
|
||||||
|
normal workflow. After the release of 3.3, the following branches would be
|
||||||
|
in use::
|
||||||
|
|
||||||
|
2.7 # Maintenance branch, no change
|
||||||
|
3.3 # Maintenance branch, as for 3.2
|
||||||
|
3.3-compat # New branch, backwards compatible changes
|
||||||
|
default # Language changes, standard library updates that depend on them
|
||||||
|
|
||||||
|
When working on a new feature, developers will need to decide whether or not
|
||||||
|
it is an acceptable change for a standard library release. If so, then it
|
||||||
|
should be checked in on ``3.3-compat`` and then merged to ``default``.
|
||||||
|
Otherwise it should be checked in directly to ``default``.
|
||||||
|
|
||||||
|
|
||||||
|
Effect on bugfix cycle
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
The effect on the bug fix cycle is essentially the same as that on the
|
||||||
|
workflow for new features - there is one additional branch to pass through
|
||||||
|
before the change reaches default branch.
|
||||||
|
|
||||||
|
|
||||||
|
Effect on the community
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
PEP 407 has this to say about the effects on the community:
|
||||||
|
|
||||||
|
People who value stability can just synchronize on the LTS releases which,
|
||||||
|
with the proposed figures, would give a similar support cycle (both in
|
||||||
|
duration and in stability).
|
||||||
|
|
||||||
|
I believe this statement is just plain wrong. Life isn't that simple. Instead,
|
||||||
|
developers of third party modules and frameworks will come under pressure to
|
||||||
|
support the full pace of the new release cycle with binary updates, teachers
|
||||||
|
and book authors will receive complaints that they're only covering an "old"
|
||||||
|
version of Python ("You're only using 3.3, the latest is 3.5!"), etc.
|
||||||
|
|
||||||
|
As the minor version number starts climbing 3 times faster than it has in the
|
||||||
|
past, I believe perceptions of language stability would also fall (whether
|
||||||
|
such opinions were justified or not).
|
||||||
|
|
||||||
|
I believe isolating the increased pace of change to the standard library,
|
||||||
|
and clearly delineating it with a separate date-based version number will
|
||||||
|
greatly reassure the rest of the community that no, we're not suddenly
|
||||||
|
asking them to triple their own rate of development. Instead, we're merely
|
||||||
|
going to ship standard library updates for the next language release in
|
||||||
|
three 6-monthly installments rather than delaying them all, even those that
|
||||||
|
are backwards compatible with the previously released version of Python.
|
||||||
|
|
||||||
|
The community benefits list in PEP 407 are equally applicable to this PEP,
|
||||||
|
at least as far as the standard library is concerned:
|
||||||
|
|
||||||
|
People who value reactivity and access to new features (without taking the
|
||||||
|
risk to install alpha versions or Mercurial snapshots) would get much more
|
||||||
|
value from the new release cycle than currently.
|
||||||
|
|
||||||
|
People who want to contribute new features or improvements would be more
|
||||||
|
motivated to do so, knowing that their contributions will be more quickly
|
||||||
|
available to normal users.
|
||||||
|
|
||||||
|
If the faster release cycle encourages more people to focus on contributing
|
||||||
|
to the standard library rather than proposing changes to the language
|
||||||
|
definition, I don't see that as a bad thing.
|
||||||
|
|
||||||
|
|
||||||
|
Handling News Updates
|
||||||
|
=====================
|
||||||
|
|
||||||
|
|
||||||
|
What's New?
|
||||||
|
-----------
|
||||||
|
|
||||||
|
The "What's New" documents would be split out into separate documents for
|
||||||
|
standard library releases and language releases. If the major version
|
||||||
|
number only continues to increase once every decade or so, resolving the
|
||||||
|
eventual numbering conflict can be safely deemed somebody elses problem :)
|
||||||
|
|
||||||
|
|
||||||
|
NEWS
|
||||||
|
----
|
||||||
|
|
||||||
|
Merge conflicts on the NEWS file is already a hassle. Since this PEP
|
||||||
|
proposes introduction of an additional branch into the normal workflow,
|
||||||
|
resolving this becomes even more critical. While Mercurial phases will
|
||||||
|
help to some degree, it would be good to eliminate the problem entirely.
|
||||||
|
|
||||||
|
One suggestion from Barry Warsaw is to adopt a non-conflicting
|
||||||
|
separate-files-per-change approach, similar to that used by Twisted [2_].
|
||||||
|
|
||||||
|
For this PEP, one possible layout for such an approach (adopted following
|
||||||
|
the release of 3.3.0+12.8.0 using the existing NEWS process) might look
|
||||||
|
like::
|
||||||
|
|
||||||
|
Misc/
|
||||||
|
lang_news/
|
||||||
|
3.3.1/
|
||||||
|
<files for core language changes>
|
||||||
|
3.4.0/
|
||||||
|
<files for core language changes>
|
||||||
|
stdlib_news/
|
||||||
|
12.08.1/
|
||||||
|
builtins/
|
||||||
|
<files for builtin changes>
|
||||||
|
extensions/
|
||||||
|
<files for extension module changes>
|
||||||
|
library/
|
||||||
|
<files for pure Python module changes>
|
||||||
|
documentation/
|
||||||
|
<files for documentation changes>
|
||||||
|
tests/
|
||||||
|
<files for testing changes>
|
||||||
|
13.02.0/
|
||||||
|
builtins/
|
||||||
|
<files for builtin changes>
|
||||||
|
extensions/
|
||||||
|
<files for extension module changes>
|
||||||
|
library/
|
||||||
|
<files for pure Python module changes>
|
||||||
|
documentation/
|
||||||
|
<files for documentation changes>
|
||||||
|
tests/
|
||||||
|
<files for testing changes>
|
||||||
|
NEWS # Now autogenerated from lang_news and stdlib_news
|
||||||
|
|
||||||
|
Putting the version information in the directory heirarchy isn't strictly
|
||||||
|
necessary (since the NEWS file generator could figure out from the version
|
||||||
|
history), but does make it easy for *humans* to keep the different versions
|
||||||
|
in order.
|
||||||
|
|
||||||
|
|
||||||
|
Why isn't PEP 384 enough?
|
||||||
|
=========================
|
||||||
|
|
||||||
|
PEP 384 introduced the notion of a "Stable ABI" for CPython, a limited
|
||||||
|
subset of the full C ABI that is guaranteed to remain stable. Extensions
|
||||||
|
built against the stable ABI should be able to support all subsequent
|
||||||
|
Python versions with the same binary.
|
||||||
|
|
||||||
|
This will help new projects to avoid coupling their C extension modules too
|
||||||
|
closely to a specific version of CPython. For existing modules, however,
|
||||||
|
migrating to the stable ABI can involve quite a lot of work (especially for
|
||||||
|
extension modules that define a lot of classes). With limited development
|
||||||
|
resources available, any time spent on such a change is time that could
|
||||||
|
otherwise have been spent working on features that are offer more direct
|
||||||
|
benefits to end users.
|
||||||
|
|
||||||
|
|
||||||
|
Why not separate out the standard library entirely?
|
||||||
|
===================================================
|
||||||
|
|
||||||
|
Because it's a lot of work for next to no pay-off. CPython without the
|
||||||
|
standard library is useless (the build chain won't even finish). You
|
||||||
|
can't create a standalone pure Python standard library, because too many
|
||||||
|
"modules" are actually tightly linked in to the internal details of the
|
||||||
|
respective interpreters (e.g. ``weakref``, ``gc``, ``sys``).
|
||||||
|
|
||||||
|
Creating a separate development branch that is kept compatible with the
|
||||||
|
previous feature release should provide most of the benefits of a
|
||||||
|
separate standard library repository with only a fraction of the pain.
|
||||||
|
|
||||||
|
|
||||||
|
Acknowledgements
|
||||||
|
================
|
||||||
|
|
||||||
|
Thanks go to the PEP 407 authors for starting this discussion, as well as
|
||||||
|
to those authors and Larry Hastings for initial discussions of the proposal
|
||||||
|
made in this PEP.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] PEP 407: New release cycle and introducing long-term support versions
|
||||||
|
http://www.python.org/dev/peps/pep-0407/
|
||||||
|
|
||||||
|
.. [2] Twisted's "topfiles" approach to NEWS generation
|
||||||
|
http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
||||||
Loading…
Reference in New Issue