diff --git a/pep-0413.txt b/pep-0413.txt new file mode 100644 index 000000000..896c4a679 --- /dev/null +++ b/pep-0413.txt @@ -0,0 +1,371 @@ +PEP: 413 +Title: Faster evolution of the Python Standard Library +Version: $Revision$ +Last-Modified: $Date$ +Author: Nick Coghlan +Status: Draft +Type: Process +Content-Type: text/x-rst +Created: 2012-02-24 +Post-History: 2012-02-24 +Resolution: TBD + + +Abstract +======== + +This PEP proposes the adoption of a new date-based versioning scheme for +the standard library (distinct from, but coupled to, the existing language +versioning scheme) that allows accelerated releases of the Python standard +library, while maintaining (or even slowing down) the current rate of +change in the core language definition. + +Like PEP 407, it aims to adjust the current balance between measured +change that allows the broader community time to adapt and being able to +keep pace with external influences that evolve more rapidly than the current +release cycle can handle (this problem is particularly notable for +standard library elements that relate to web technologies). + +However, it's more conservative in its aims than PEP 407, seeking to +restrict the increased pace of development to builtin and standard library +interfaces, without affecting the rate of change for other elements such +as the language syntax and version numbering as well as the CPython +binary API and bytecode format. + + +Rationale +========= + +To quote the PEP 407 abstract: + + Finding a release cycle for an open-source project is a delicate exercise + in managing mutually contradicting constraints: developer manpower, + availability of release management volunteers, ease of maintenance for + users and third-party packagers, quick availability of new features (and + behavioural changes), availability of bug fixes without pulling in new + features or behavioural changes. + + The current release cycle errs on the conservative side. It is adequate + for people who value stability over reactivity. This PEP is an attempt to + keep the stability that has become a Python trademark, while offering a + more fluid release of features, by introducing the notion of long-term + support versions. + +I agree with the PEP 407 authors that the current release cycle of the +*standard library* is too slow to effectively cope with the pace of change +in some key programming areas (specifically, web protocols and related +technologies, including databases, templating and serialisation formats). + +However, I have written this competing PEP because I believe that the +approach proposed in PEP 407 of offering full, potentially binary +incompatible releases of CPython every 6 months places too great a burden +on the wider Python ecosystem. + +Under the current CPython release cycle, distributors of key binary +extensions will often support Python releases even after the CPython branches +enter "security fix only" mode (for example, Twisted currently ships binaries +for 2.5, 2.6 and 2.7, NumPy and SciPy suport those 3 along with 3.1 and 3.2, +PyGame adds a 2.4 binary release, wxPython provides both 32-bit and 64-bit +binaries for 2.6 and 2.7, etc). + +If CPython were to triple (or more) its rate of releases, the developers of +those libraries (many of which are even more resource starved than CPython) +would face an unpalatable choice: either adopt the faster release cycle +themselves (up to 18 simultaneous binary releases for PyGame!), drop +older Python versions more quickly, or else tell their users to stick to the +CPython LTS releases (thus defeating the entire point of speeding up the +CPython release cycle in the first place). + +Similarly, many support tools for Python (e.g. syntax highlighters) can take +quite some time to catch up with language level changes. + +At a cultural level, the Python community is also accustomed to a certain +meaning for Python version numbers - they're linked to deprecation periods, +support periods, all sorts of things. PEP 407 proposes that collective +knowledge all be swept aside, without offering a compelling rationale for why +such a course of action is actually *necessary* (aside from, perhaps, making +the lives of the CPython core developers a little easier at the expense of +everyone else). + +But, if we go back to the primary rationale for increasing the pace of change +(i.e. more timely support for web protocols and related technologies), we can +note that those only require *standard library* changes. That means many +(perhaps even most) of the negative effects on the wider community can be +avoided by explicitly limiting which parts of CPython are affected by the +new release cycle, and allowing other parts to evolve at their current, more +sedate, pace. + + +Proposal +======== + +This PEP proposes the addition of a new ``sys.stdlib_info`` attribute that +records a date based standard library version above and beyond the underlying +interpreter version:: + + sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0) + +This information would also be included in the ``sys.version`` string:: + + Python 3.3.0 (12.08.0, default:c1a07c8092f7+, Feb 17 2012, 23:03:41) + [GCC 4.6.1] + +When maintenance releases are created, *two* new versions of Python would +actually be published on python.org (using the first 3.3 maintenance release, +planned for February 2013 as an example):: + + 3.3.1 + 12.08.1 # Maintenance release + 3.3.1 + 13.02.0 # Standard library release + +A standard library release would just be the corresponding maintenance +release, with the following additional, backwards compatible changes: + +* new features in pure Python modules +* new features in C extension modules (subject to PEP 399 compatibility + requirements) +* new features in language builtins (provided the C ABI remains unaffected) + +A further 6 months later, the next 3.3 maintenance release would again be +accompanied by a new standard library release:: + + 3.3.2 + 12.08.2 # Maintenance release + 3.3.2 + 13.08.1 # Standard library release + +Again, the standard library release would be binary compatible with the +previous language release, merely offering additional features at the +Python level. + +Finally, 18 months after the release of 3.3, a new language release would +be made around the same time as the final 3.3 maintenance release: + + 3.3.3 + 12.08.3 # Maintenance release + 3.4.0 + 14.02.0 # Language release + +Language releases would then contain all the features that are not +permitted in standard library releases: + +* new language syntax +* new deprecation warnings +* removal of previously deprecated features +* changes to the emitted bytecode +* changes to the AST +* any other significant changes to the compilation toolchain +* changes to the C ABI + +The 3.4 release cycle would then follow a similar pattern to that for 3.3:: + + 3.4.1 + 14.02.1 # Maintenance release + 3.4.1 + 14.08.0 # Standard library release + 3.4.2 + 14.02.2 # Maintenance release + 3.4.2 + 15.02.0 # Standard library release + 3.4.3 + 14.02.3 # Maintenance release + 3.5.0 + 15.08.0 # Language release + + +Effects +======= + +Effect on development cycle +--------------------------- + +Similar to PEP 407, this PEP will break up the delivery of new features into +more discrete chunks. Instead of whole raft of changes landing all at once +in a language release, each language release will be limited to 6 months +worth of standard library changes, as well as any changes associated with +new syntax. + +Effect on workflow +------------------ + +This PEP proposes the creation of a single additional branch for use in the +normal workflow. After the release of 3.3, the following branches would be +in use:: + + 2.7 # Maintenance branch, no change + 3.3 # Maintenance branch, as for 3.2 + 3.3-compat # New branch, backwards compatible changes + default # Language changes, standard library updates that depend on them + +When working on a new feature, developers will need to decide whether or not +it is an acceptable change for a standard library release. If so, then it +should be checked in on ``3.3-compat`` and then merged to ``default``. +Otherwise it should be checked in directly to ``default``. + + +Effect on bugfix cycle +---------------------- + +The effect on the bug fix cycle is essentially the same as that on the +workflow for new features - there is one additional branch to pass through +before the change reaches default branch. + + +Effect on the community +----------------------- + +PEP 407 has this to say about the effects on the community: + + People who value stability can just synchronize on the LTS releases which, + with the proposed figures, would give a similar support cycle (both in + duration and in stability). + +I believe this statement is just plain wrong. Life isn't that simple. Instead, +developers of third party modules and frameworks will come under pressure to +support the full pace of the new release cycle with binary updates, teachers +and book authors will receive complaints that they're only covering an "old" +version of Python ("You're only using 3.3, the latest is 3.5!"), etc. + +As the minor version number starts climbing 3 times faster than it has in the +past, I believe perceptions of language stability would also fall (whether +such opinions were justified or not). + +I believe isolating the increased pace of change to the standard library, +and clearly delineating it with a separate date-based version number will +greatly reassure the rest of the community that no, we're not suddenly +asking them to triple their own rate of development. Instead, we're merely +going to ship standard library updates for the next language release in +three 6-monthly installments rather than delaying them all, even those that +are backwards compatible with the previously released version of Python. + +The community benefits list in PEP 407 are equally applicable to this PEP, +at least as far as the standard library is concerned: + + People who value reactivity and access to new features (without taking the + risk to install alpha versions or Mercurial snapshots) would get much more + value from the new release cycle than currently. + + People who want to contribute new features or improvements would be more + motivated to do so, knowing that their contributions will be more quickly + available to normal users. + +If the faster release cycle encourages more people to focus on contributing +to the standard library rather than proposing changes to the language +definition, I don't see that as a bad thing. + + +Handling News Updates +===================== + + +What's New? +----------- + +The "What's New" documents would be split out into separate documents for +standard library releases and language releases. If the major version +number only continues to increase once every decade or so, resolving the +eventual numbering conflict can be safely deemed somebody elses problem :) + + +NEWS +---- + +Merge conflicts on the NEWS file is already a hassle. Since this PEP +proposes introduction of an additional branch into the normal workflow, +resolving this becomes even more critical. While Mercurial phases will +help to some degree, it would be good to eliminate the problem entirely. + +One suggestion from Barry Warsaw is to adopt a non-conflicting +separate-files-per-change approach, similar to that used by Twisted [2_]. + +For this PEP, one possible layout for such an approach (adopted following +the release of 3.3.0+12.8.0 using the existing NEWS process) might look +like:: + + Misc/ + lang_news/ + 3.3.1/ + + 3.4.0/ + + stdlib_news/ + 12.08.1/ + builtins/ + + extensions/ + + library/ + + documentation/ + + tests/ + + 13.02.0/ + builtins/ + + extensions/ + + library/ + + documentation/ + + tests/ + + NEWS # Now autogenerated from lang_news and stdlib_news + +Putting the version information in the directory heirarchy isn't strictly +necessary (since the NEWS file generator could figure out from the version +history), but does make it easy for *humans* to keep the different versions +in order. + + +Why isn't PEP 384 enough? +========================= + +PEP 384 introduced the notion of a "Stable ABI" for CPython, a limited +subset of the full C ABI that is guaranteed to remain stable. Extensions +built against the stable ABI should be able to support all subsequent +Python versions with the same binary. + +This will help new projects to avoid coupling their C extension modules too +closely to a specific version of CPython. For existing modules, however, +migrating to the stable ABI can involve quite a lot of work (especially for +extension modules that define a lot of classes). With limited development +resources available, any time spent on such a change is time that could +otherwise have been spent working on features that are offer more direct +benefits to end users. + + +Why not separate out the standard library entirely? +=================================================== + +Because it's a lot of work for next to no pay-off. CPython without the +standard library is useless (the build chain won't even finish). You +can't create a standalone pure Python standard library, because too many +"modules" are actually tightly linked in to the internal details of the +respective interpreters (e.g. ``weakref``, ``gc``, ``sys``). + +Creating a separate development branch that is kept compatible with the +previous feature release should provide most of the benefits of a +separate standard library repository with only a fraction of the pain. + + +Acknowledgements +================ + +Thanks go to the PEP 407 authors for starting this discussion, as well as +to those authors and Larry Hastings for initial discussions of the proposal +made in this PEP. + +References +========== + +.. [1] PEP 407: New release cycle and introducing long-term support versions + http://www.python.org/dev/peps/pep-0407/ + +.. [2] Twisted's "topfiles" approach to NEWS generation + http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: