Add PEP 413 as a competitor to PEP 407
This commit is contained in:
Nick Coghlan
parent
05da204481
commit
cbfb3afdb3
|
|
@ -0,0 +1,371 @@
|
|||
PEP: 413
|
||||
Title: Faster evolution of the Python Standard Library
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||||
Status: Draft
|
||||
Type: Process
|
||||
Content-Type: text/x-rst
|
||||
Created: 2012-02-24
|
||||
Post-History: 2012-02-24
|
||||
Resolution: TBD
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes the adoption of a new date-based versioning scheme for
|
||||
the standard library (distinct from, but coupled to, the existing language
|
||||
versioning scheme) that allows accelerated releases of the Python standard
|
||||
library, while maintaining (or even slowing down) the current rate of
|
||||
change in the core language definition.
|
||||
|
||||
Like PEP 407, it aims to adjust the current balance between measured
|
||||
change that allows the broader community time to adapt and being able to
|
||||
keep pace with external influences that evolve more rapidly than the current
|
||||
release cycle can handle (this problem is particularly notable for
|
||||
standard library elements that relate to web technologies).
|
||||
|
||||
However, it's more conservative in its aims than PEP 407, seeking to
|
||||
restrict the increased pace of development to builtin and standard library
|
||||
interfaces, without affecting the rate of change for other elements such
|
||||
as the language syntax and version numbering as well as the CPython
|
||||
binary API and bytecode format.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
To quote the PEP 407 abstract:
|
||||
|
||||
Finding a release cycle for an open-source project is a delicate exercise
|
||||
in managing mutually contradicting constraints: developer manpower,
|
||||
availability of release management volunteers, ease of maintenance for
|
||||
users and third-party packagers, quick availability of new features (and
|
||||
behavioural changes), availability of bug fixes without pulling in new
|
||||
features or behavioural changes.
|
||||
|
||||
The current release cycle errs on the conservative side. It is adequate
|
||||
for people who value stability over reactivity. This PEP is an attempt to
|
||||
keep the stability that has become a Python trademark, while offering a
|
||||
more fluid release of features, by introducing the notion of long-term
|
||||
support versions.
|
||||
|
||||
I agree with the PEP 407 authors that the current release cycle of the
|
||||
*standard library* is too slow to effectively cope with the pace of change
|
||||
in some key programming areas (specifically, web protocols and related
|
||||
technologies, including databases, templating and serialisation formats).
|
||||
|
||||
However, I have written this competing PEP because I believe that the
|
||||
approach proposed in PEP 407 of offering full, potentially binary
|
||||
incompatible releases of CPython every 6 months places too great a burden
|
||||
on the wider Python ecosystem.
|
||||
|
||||
Under the current CPython release cycle, distributors of key binary
|
||||
extensions will often support Python releases even after the CPython branches
|
||||
enter "security fix only" mode (for example, Twisted currently ships binaries
|
||||
for 2.5, 2.6 and 2.7, NumPy and SciPy suport those 3 along with 3.1 and 3.2,
|
||||
PyGame adds a 2.4 binary release, wxPython provides both 32-bit and 64-bit
|
||||
binaries for 2.6 and 2.7, etc).
|
||||
|
||||
If CPython were to triple (or more) its rate of releases, the developers of
|
||||
those libraries (many of which are even more resource starved than CPython)
|
||||
would face an unpalatable choice: either adopt the faster release cycle
|
||||
themselves (up to 18 simultaneous binary releases for PyGame!), drop
|
||||
older Python versions more quickly, or else tell their users to stick to the
|
||||
CPython LTS releases (thus defeating the entire point of speeding up the
|
||||
CPython release cycle in the first place).
|
||||
|
||||
Similarly, many support tools for Python (e.g. syntax highlighters) can take
|
||||
quite some time to catch up with language level changes.
|
||||
|
||||
At a cultural level, the Python community is also accustomed to a certain
|
||||
meaning for Python version numbers - they're linked to deprecation periods,
|
||||
support periods, all sorts of things. PEP 407 proposes that collective
|
||||
knowledge all be swept aside, without offering a compelling rationale for why
|
||||
such a course of action is actually *necessary* (aside from, perhaps, making
|
||||
the lives of the CPython core developers a little easier at the expense of
|
||||
everyone else).
|
||||
|
||||
But, if we go back to the primary rationale for increasing the pace of change
|
||||
(i.e. more timely support for web protocols and related technologies), we can
|
||||
note that those only require *standard library* changes. That means many
|
||||
(perhaps even most) of the negative effects on the wider community can be
|
||||
avoided by explicitly limiting which parts of CPython are affected by the
|
||||
new release cycle, and allowing other parts to evolve at their current, more
|
||||
sedate, pace.
|
||||
|
||||
|
||||
Proposal
|
||||
========
|
||||
|
||||
This PEP proposes the addition of a new ``sys.stdlib_info`` attribute that
|
||||
records a date based standard library version above and beyond the underlying
|
||||
interpreter version::
|
||||
|
||||
sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0)
|
||||
|
||||
This information would also be included in the ``sys.version`` string::
|
||||
|
||||
Python 3.3.0 (12.08.0, default:c1a07c8092f7+, Feb 17 2012, 23:03:41)
|
||||
[GCC 4.6.1]
|
||||
|
||||
When maintenance releases are created, *two* new versions of Python would
|
||||
actually be published on python.org (using the first 3.3 maintenance release,
|
||||
planned for February 2013 as an example)::
|
||||
|
||||
3.3.1 + 12.08.1 # Maintenance release
|
||||
3.3.1 + 13.02.0 # Standard library release
|
||||
|
||||
A standard library release would just be the corresponding maintenance
|
||||
release, with the following additional, backwards compatible changes:
|
||||
|
||||
* new features in pure Python modules
|
||||
* new features in C extension modules (subject to PEP 399 compatibility
|
||||
requirements)
|
||||
* new features in language builtins (provided the C ABI remains unaffected)
|
||||
|
||||
A further 6 months later, the next 3.3 maintenance release would again be
|
||||
accompanied by a new standard library release::
|
||||
|
||||
3.3.2 + 12.08.2 # Maintenance release
|
||||
3.3.2 + 13.08.1 # Standard library release
|
||||
|
||||
Again, the standard library release would be binary compatible with the
|
||||
previous language release, merely offering additional features at the
|
||||
Python level.
|
||||
|
||||
Finally, 18 months after the release of 3.3, a new language release would
|
||||
be made around the same time as the final 3.3 maintenance release:
|
||||
|
||||
3.3.3 + 12.08.3 # Maintenance release
|
||||
3.4.0 + 14.02.0 # Language release
|
||||
|
||||
Language releases would then contain all the features that are not
|
||||
permitted in standard library releases:
|
||||
|
||||
* new language syntax
|
||||
* new deprecation warnings
|
||||
* removal of previously deprecated features
|
||||
* changes to the emitted bytecode
|
||||
* changes to the AST
|
||||
* any other significant changes to the compilation toolchain
|
||||
* changes to the C ABI
|
||||
|
||||
The 3.4 release cycle would then follow a similar pattern to that for 3.3::
|
||||
|
||||
3.4.1 + 14.02.1 # Maintenance release
|
||||
3.4.1 + 14.08.0 # Standard library release
|
||||
3.4.2 + 14.02.2 # Maintenance release
|
||||
3.4.2 + 15.02.0 # Standard library release
|
||||
3.4.3 + 14.02.3 # Maintenance release
|
||||
3.5.0 + 15.08.0 # Language release
|
||||
|
||||
|
||||
Effects
|
||||
=======
|
||||
|
||||
Effect on development cycle
|
||||
---------------------------
|
||||
|
||||
Similar to PEP 407, this PEP will break up the delivery of new features into
|
||||
more discrete chunks. Instead of whole raft of changes landing all at once
|
||||
in a language release, each language release will be limited to 6 months
|
||||
worth of standard library changes, as well as any changes associated with
|
||||
new syntax.
|
||||
|
||||
Effect on workflow
|
||||
------------------
|
||||
|
||||
This PEP proposes the creation of a single additional branch for use in the
|
||||
normal workflow. After the release of 3.3, the following branches would be
|
||||
in use::
|
||||
|
||||
2.7 # Maintenance branch, no change
|
||||
3.3 # Maintenance branch, as for 3.2
|
||||
3.3-compat # New branch, backwards compatible changes
|
||||
default # Language changes, standard library updates that depend on them
|
||||
|
||||
When working on a new feature, developers will need to decide whether or not
|
||||
it is an acceptable change for a standard library release. If so, then it
|
||||
should be checked in on ``3.3-compat`` and then merged to ``default``.
|
||||
Otherwise it should be checked in directly to ``default``.
|
||||
|
||||
|
||||
Effect on bugfix cycle
|
||||
----------------------
|
||||
|
||||
The effect on the bug fix cycle is essentially the same as that on the
|
||||
workflow for new features - there is one additional branch to pass through
|
||||
before the change reaches default branch.
|
||||
|
||||
|
||||
Effect on the community
|
||||
-----------------------
|
||||
|
||||
PEP 407 has this to say about the effects on the community:
|
||||
|
||||
People who value stability can just synchronize on the LTS releases which,
|
||||
with the proposed figures, would give a similar support cycle (both in
|
||||
duration and in stability).
|
||||
|
||||
I believe this statement is just plain wrong. Life isn't that simple. Instead,
|
||||
developers of third party modules and frameworks will come under pressure to
|
||||
support the full pace of the new release cycle with binary updates, teachers
|
||||
and book authors will receive complaints that they're only covering an "old"
|
||||
version of Python ("You're only using 3.3, the latest is 3.5!"), etc.
|
||||
|
||||
As the minor version number starts climbing 3 times faster than it has in the
|
||||
past, I believe perceptions of language stability would also fall (whether
|
||||
such opinions were justified or not).
|
||||
|
||||
I believe isolating the increased pace of change to the standard library,
|
||||
and clearly delineating it with a separate date-based version number will
|
||||
greatly reassure the rest of the community that no, we're not suddenly
|
||||
asking them to triple their own rate of development. Instead, we're merely
|
||||
going to ship standard library updates for the next language release in
|
||||
three 6-monthly installments rather than delaying them all, even those that
|
||||
are backwards compatible with the previously released version of Python.
|
||||
|
||||
The community benefits list in PEP 407 are equally applicable to this PEP,
|
||||
at least as far as the standard library is concerned:
|
||||
|
||||
People who value reactivity and access to new features (without taking the
|
||||
risk to install alpha versions or Mercurial snapshots) would get much more
|
||||
value from the new release cycle than currently.
|
||||
|
||||
People who want to contribute new features or improvements would be more
|
||||
motivated to do so, knowing that their contributions will be more quickly
|
||||
available to normal users.
|
||||
|
||||
If the faster release cycle encourages more people to focus on contributing
|
||||
to the standard library rather than proposing changes to the language
|
||||
definition, I don't see that as a bad thing.
|
||||
|
||||
|
||||
Handling News Updates
|
||||
=====================
|
||||
|
||||
|
||||
What's New?
|
||||
-----------
|
||||
|
||||
The "What's New" documents would be split out into separate documents for
|
||||
standard library releases and language releases. If the major version
|
||||
number only continues to increase once every decade or so, resolving the
|
||||
eventual numbering conflict can be safely deemed somebody elses problem :)
|
||||
|
||||
|
||||
NEWS
|
||||
----
|
||||
|
||||
Merge conflicts on the NEWS file is already a hassle. Since this PEP
|
||||
proposes introduction of an additional branch into the normal workflow,
|
||||
resolving this becomes even more critical. While Mercurial phases will
|
||||
help to some degree, it would be good to eliminate the problem entirely.
|
||||
|
||||
One suggestion from Barry Warsaw is to adopt a non-conflicting
|
||||
separate-files-per-change approach, similar to that used by Twisted [2_].
|
||||
|
||||
For this PEP, one possible layout for such an approach (adopted following
|
||||
the release of 3.3.0+12.8.0 using the existing NEWS process) might look
|
||||
like::
|
||||
|
||||
Misc/
|
||||
lang_news/
|
||||
3.3.1/
|
||||
<files for core language changes>
|
||||
3.4.0/
|
||||
<files for core language changes>
|
||||
stdlib_news/
|
||||
12.08.1/
|
||||
builtins/
|
||||
<files for builtin changes>
|
||||
extensions/
|
||||
<files for extension module changes>
|
||||
library/
|
||||
<files for pure Python module changes>
|
||||
documentation/
|
||||
<files for documentation changes>
|
||||
tests/
|
||||
<files for testing changes>
|
||||
13.02.0/
|
||||
builtins/
|
||||
<files for builtin changes>
|
||||
extensions/
|
||||
<files for extension module changes>
|
||||
library/
|
||||
<files for pure Python module changes>
|
||||
documentation/
|
||||
<files for documentation changes>
|
||||
tests/
|
||||
<files for testing changes>
|
||||
NEWS # Now autogenerated from lang_news and stdlib_news
|
||||
|
||||
Putting the version information in the directory heirarchy isn't strictly
|
||||
necessary (since the NEWS file generator could figure out from the version
|
||||
history), but does make it easy for *humans* to keep the different versions
|
||||
in order.
|
||||
|
||||
|
||||
Why isn't PEP 384 enough?
|
||||
=========================
|
||||
|
||||
PEP 384 introduced the notion of a "Stable ABI" for CPython, a limited
|
||||
subset of the full C ABI that is guaranteed to remain stable. Extensions
|
||||
built against the stable ABI should be able to support all subsequent
|
||||
Python versions with the same binary.
|
||||
|
||||
This will help new projects to avoid coupling their C extension modules too
|
||||
closely to a specific version of CPython. For existing modules, however,
|
||||
migrating to the stable ABI can involve quite a lot of work (especially for
|
||||
extension modules that define a lot of classes). With limited development
|
||||
resources available, any time spent on such a change is time that could
|
||||
otherwise have been spent working on features that are offer more direct
|
||||
benefits to end users.
|
||||
|
||||
|
||||
Why not separate out the standard library entirely?
|
||||
===================================================
|
||||
|
||||
Because it's a lot of work for next to no pay-off. CPython without the
|
||||
standard library is useless (the build chain won't even finish). You
|
||||
can't create a standalone pure Python standard library, because too many
|
||||
"modules" are actually tightly linked in to the internal details of the
|
||||
respective interpreters (e.g. ``weakref``, ``gc``, ``sys``).
|
||||
|
||||
Creating a separate development branch that is kept compatible with the
|
||||
previous feature release should provide most of the benefits of a
|
||||
separate standard library repository with only a fraction of the pain.
|
||||
|
||||
|
||||
Acknowledgements
|
||||
================
|
||||
|
||||
Thanks go to the PEP 407 authors for starting this discussion, as well as
|
||||
to those authors and Larry Hastings for initial discussions of the proposal
|
||||
made in this PEP.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [1] PEP 407: New release cycle and introducing long-term support versions
|
||||
http://www.python.org/dev/peps/pep-0407/
|
||||
|
||||
.. [2] Twisted's "topfiles" approach to NEWS generation
|
||||
http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
||||
Loading…
Reference in New Issue