From 9a0fb2762171d61d6ba02b0f3b6e4f4162e88568 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Sun, 26 Feb 2012 12:49:49 +1000 Subject: [PATCH 1/8] Further refinements to PEP 413, save state before changing the proposed stdlib versioning scheme --- pep-0413.txt | 235 ++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 169 insertions(+), 66 deletions(-) diff --git a/pep-0413.txt b/pep-0413.txt index b8684fd71..f4d84b396 100644 --- a/pep-0413.txt +++ b/pep-0413.txt @@ -99,37 +99,67 @@ sedate, pace. Proposal ======== -This PEP proposes the addition of a new ``sys.stdlib_info`` attribute that -records a date based standard library version above and beyond the underlying -interpreter version:: +This PEP proposes the introduction of a new kind of CPython release: +"standard library releases". As with PEP 407, this will give CPython 3 kinds +of release: - sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0) +* Language release: "x.y.0" +* Maintenance release: "x.y.z" (where z > 0) +* Standard library release: "x.y (YY.MM)" -This information would also be included in the ``sys.version`` string:: +Under this scheme, an unqualified version reference (such as "3.3") would +always refer to the most recent corresponding language or maintenance +release. It will never be used without qualification to refer to a standard +library release (at least by python-dev - obviously, we can only set an +example, not force the rest of the Python ecosystem to go along with it). - Python 3.3.0 (12.08.0, default:c1a07c8092f7+, Feb 17 2012, 23:03:41) - [GCC 4.6.1] +Language releases will continue as they are now, as new versions of the +Python language definition, along with a new version of the CPython +interpreter and the Python standard library. Accordingly, a language +release may contain any and all of the following changes: -When maintenance releases are created, *two* new versions of Python would -actually be published on python.org (using the first 3.3 maintenance release, -planned for February 2013 as an example):: +* new language syntax +* new standard library changes (see below) +* new deprecation warnings +* removal of previously deprecated features +* changes to the emitted bytecode +* changes to the AST +* any other significant changes to the compilation toolchain +* changes to the core interpreter eval loop +* changes to the C ABI (although the PEP 384 stable ABI must be preserved) +* bug fixes - 3.3.1 + 12.08.1 # Maintenance release - 3.3.1 + 13.02.0 # Standard library release +Maintenance releases will also continue as they do today, being strictly +limited to bug fixes for the corresponding language release. No new features +or radical internal changes are permitted. -A standard library release would just be the corresponding maintenance -release, with the following additional, backwards compatible changes: +The new standard library releases will occur in parallel with each +maintenance release and will be qualified with a date-based version +identifier documenting their *actual* release date. Standard library +releases may include the following changes: * new features in pure Python modules * new features in C extension modules (subject to PEP 399 compatibility requirements) * new features in language builtins (provided the C ABI remains unaffected) +* bug fixes from the corresponding maintenance release + + +Release Cycle +------------- + +When maintenance releases are created, *two* new versions of Python would +actually be published on python.org (using the first 3.3 maintenance release, +planned for February 2013 as an example):: + + 3.3.1 # Maintenance release + 3.3 (13.02) # Standard library release A further 6 months later, the next 3.3 maintenance release would again be accompanied by a new standard library release:: - 3.3.2 + 12.08.2 # Maintenance release - 3.3.2 + 13.08.1 # Standard library release + 3.3.2 # Maintenance release + 3.3 (13.08) # Standard library release Again, the standard library release would be binary compatible with the previous language release, merely offering additional features at the @@ -138,30 +168,62 @@ Python level. Finally, 18 months after the release of 3.3, a new language release would be made around the same time as the final 3.3 maintenance release:: - 3.3.3 + 12.08.3 # Maintenance release - 3.4.0 + 14.02.0 # Language release + 3.3.3 # Maintenance release + 3.4.0 # Language release Language releases would then contain all the features that are not permitted in standard library releases: -* new language syntax -* new deprecation warnings -* removal of previously deprecated features -* changes to the emitted bytecode -* changes to the AST -* any other significant changes to the compilation toolchain -* changes to the core eval loop -* changes to the C ABI - The 3.4 release cycle would then follow a similar pattern to that for 3.3:: - 3.4.1 + 14.02.1 # Maintenance release - 3.4.1 + 14.08.0 # Standard library release - 3.4.2 + 14.02.2 # Maintenance release - 3.4.2 + 15.02.0 # Standard library release - 3.4.3 + 14.02.3 # Maintenance release - 3.5.0 + 15.08.0 # Language release + 3.4.1 # Maintenance release + 3.4 (14.08) # Standard library release + 3.4.2 # Maintenance release + 3.4 (15.02) # Standard library release + + 3.4.3 # Maintenance release + 3.5 # Language release + + +Programmatic Version Identification +----------------------------------- + +To expose the new release details programmatically, this PEP proposes the +addition of a new ``sys.stdlib_info`` attribute that records the date based +standard library version above and beyond the underlying interpreter +version. Using the initial Python 3.3 release as an example:: + + sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0) + +This information would also be included in the ``sys.version`` string:: + + Python 3.3.0 (12.08.0, default, Feb 17 2012, 23:03:41) + [GCC 4.6.1] + + +Security Fixes and Other "Out of Cycle" Releases +------------------------------------------------ + +For maintenance releases the process of handling out-of-cycle releases (for +example, to fix a security issue or resolve a critical bug in a new release), +remains the same as it is now: the minor version number is incremented and a +new release is made incorporating the required bug fixes, as well as any +other bug fixes that have been committed since the previous release. + +For standard library releases, things are slightly more complicated, as the +corresponding source control branch may contain new features in addition to +bug fixes. If such features exist, then, to provide a comprehensible release, +it would be necessary to: + +1. Change the release version number to the date of the current month. +2. Update the What's New, NEWS and documentation to refer to the new release + number. +3. Make the new release. + +(Note: due to this problem with out-of-cycle releases, I'm considering an +alternate standard library version scheme that would use date independent +numbers like "33.1" and "33.2". See `Why a date-based versioning scheme?`_). User Scenarios ============== @@ -187,12 +249,12 @@ Novice user, downloading Python from python.org in March 2013 **Status quo:** must choose between 3.3 and 2.7 -**This PEP:** must choose between 3.3 (13.02), 3.3 (12.08) and 2.7. +**This PEP:** must choose between 3.3 (13.02), 3.3 and 2.7. **PEP 407:** must choose between 3.4, 3.3 (LTS) and 2.7. **Verdict:** explaining the meaning of a Long Term Support release is about as -complicated as explaining the meaning of the proposed standard library +complicated as explaining the meaning of the proposed standard library release version numbers. I call this a tie. @@ -555,12 +617,16 @@ and "3.4" directories to group entries for each language version and the associated maintenance and standard library releases). -Option: Slowing down the language release cycle -=============================================== +Other benefits of reduced version coupling +========================================== + +Slowing down the language release cycle +--------------------------------------- The current release cycle is a compromise between the desire for stability in the core language definition and C extension ABI, and the desire to get -new feature (most notably standard library updates) into users hands quickly. +new features (most notably standard library updates) into user's hands more +quickly. With the standard library release cycle decoupled (to some degree) from that of the core language definition, it provides an opportunity to actually @@ -574,34 +640,35 @@ allow us the flexibility to slow down language releases at the same time as standard library releases become more frequent. As simple example, if a full two years was allowed between 3.3 and 3.4, -the 3.3 release cycle would be up looking like:: +the 3.3 release cycle would end up looking like:: - 3.2.4 # Maintenance release - 3.3.0 + 12.08.0 # Language release + 3.2.4 # Maintenance release + 3.3.0 # Language release - 3.3.1 + 12.08.1 # Maintenance release - 3.3.1 + 13.02.0 # Standard library release + 3.3.1 # Maintenance release + 3.3 (13.02) # Standard library release - 3.3.2 + 12.08.2 # Maintenance release - 3.3.2 + 13.08.1 # Standard library release + 3.3.2 # Maintenance release + 3.3 (13.08) # Standard library release - 3.3.3 + 12.08.3 # Maintenance release - 3.3.3 + 14.02.1 # Standard library release + 3.3.3 # Maintenance release + 3.3 (14.02) # Standard library release - 3.3.4 + 12.08.4 # Maintenance release - 3.4.0 + 14.08.0 # Language release + 3.3.4 # Maintenance release + 3.4.0 # Language release The elegance of the proposed NEWS entry layout is that this decision wouldn't need to be made until after the 13.08 standard library release. At that point, the ``3.3-compat`` branch could be kept open (thus adding another standard library release to the cycle), or else it could be closed, -committing to the next release being a full language release. The choice -between another standard library release or a full language release would -then be available every 6 months after that. +committing to the next release being a full language release. If the +compatibility branch was kept open, the choice between another standard +library release or a full language release would then be available every +6 months after that. -Future: Further increasing the pace of standard library development -=================================================================== +Further increasing the pace of standard library development +----------------------------------------------------------- A further benefit of the scheme proposed in this PEP is that it almost *fully* decouples the language release cycle from the standard library @@ -610,19 +677,55 @@ even once a month, without having any flow on effects on the language version numbering or the perceived stability of the core language. While that pace of development isn't practical as long as the binary -installer creation for Windows and Mac OS X involves several manual steps and -for as long as we don't have separate "-release" trees that only -receive versions that have been marked as good by the stable buildbots, -it's a useful criterion to keep in mind: what if we want to make standard -library releases even *faster* than every 6 months? +installer creation for Windows and Mac OS X involves several manual steps +(including manual testing) and for as long as we don't have separate +"-release" trees that only receive versions that have been marked as +good by the stable buildbots, it's a useful criterion to keep in mind when +considering proposed new versioning schemes: what if we eventually want +to make standard library releases even *faster* than every 6 months? If the practical issues were ever resolved, then the separate date-based versioning scheme in this PEP could handle it. The approach proposed in -PEP 407 could not. +PEP 407 could not (at least, not without a lot of user confusion). + + +Other Questions +=============== + +Why a date-based versioning scheme? +----------------------------------- + +While a date-based scheme does mean additional work in the case of an +out-of-cycle release, it's an easy way to come up with a numbering system +that is clearly orthogonal to the existing versioning scheme used for +language releases. + +Another possibility would just be ``major.minor`` versioning where: + +* major: xy from the corresponding language release (that is, "33" for 3.3) +* minor: release sequence number for the standard library releases + +Then the Python 3.3 release series would look like:: + + 3.3.0 # Language release + + 3.3.1 # Maintenance release + 3.3 (33.1) # Standard library release + + 3.3.2 # Maintenance release + 3.3 (33.2) # Standard library release + + 3.3.3 # Maintenance release + 3.4.0 # Language release + +Such a scheme would have the advantage of tolerating out-of-cycle releases +without needing to adjust the version numbering for the release, but would +lose the minor benefit of providing clearer age information within the +versioning scheme itself. Why isn't PEP 384 enough? -========================= +------------------------- PEP 384 introduced the notion of a "Stable ABI" for CPython, a limited subset of the full C ABI that is guaranteed to remain stable. Extensions @@ -639,14 +742,14 @@ benefits to end users. Why not separate out the standard library entirely? -=================================================== +--------------------------------------------------- Because it's a lot of work for next to no pay-off. CPython without the standard library is useless (the build chain won't even run, let alone the -test suite). You can't create a standalone pure Python standard library, -because too many "modules" are actually tightly linked in to the internal -details of their respective interpreters (e.g. ``weakref``, ``gc``, ``sys``, -``inspect``, ``ast``). +test suite). You can't create a standalone pure Python standard library +either, because too many "modules" are actually tightly linked in to the +internal details of their respective interpreters (e.g. ``weakref``, ``gc``, +``sys``, ``inspect``, ``ast``). Creating a separate development branch that is kept compatible with the previous feature release, and making releases from that branch that are From 344b1d0c24eb8fb4db0471c3d251ccd7d408815c Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Sun, 26 Feb 2012 14:53:07 +1000 Subject: [PATCH 2/8] Propose a simpler sequential version numbering scheme for standard library releases --- pep-0413.txt | 465 +++++++++++++++++++++++++++++---------------------- 1 file changed, 264 insertions(+), 201 deletions(-) diff --git a/pep-0413.txt b/pep-0413.txt index f4d84b396..02f7df06c 100644 --- a/pep-0413.txt +++ b/pep-0413.txt @@ -14,8 +14,8 @@ Resolution: TBD Abstract ======== -This PEP proposes the adoption of a new date-based versioning scheme for -the standard library (distinct from, but coupled to, the existing language +This PEP proposes the adoption of a separate versioning scheme for the +standard library (distinct from, but coupled to, the existing language versioning scheme) that allows accelerated releases of the Python standard library, while maintaining (or even slowing down) the current rate of change in the core language definition. @@ -87,11 +87,11 @@ such a course of action is actually *necessary* (aside from, perhaps, making the lives of the CPython core developers a little easier at the expense of everyone else). -But, if we go back to the primary rationale for increasing the pace of change -(i.e. more timely support for web protocols and related technologies), we can -note that those only require *standard library* changes. That means many -(perhaps even most) of the negative effects on the wider community can be -avoided by explicitly limiting which parts of CPython are affected by the +However, if we go back to the primary rationale for increasing the pace of +change (i.e. more timely support for web protocols and related technologies), +we can note that those only require *standard library* changes. That means +many (perhaps even most) of the negative effects on the wider community can +be avoided by explicitly limiting which parts of CPython are affected by the new release cycle, and allowing other parts to evolve at their current, more sedate, pace. @@ -105,12 +105,12 @@ of release: * Language release: "x.y.0" * Maintenance release: "x.y.z" (where z > 0) -* Standard library release: "x.y (YY.MM)" +* Standard library release: "x.y (xy.z)" (where z > 0) Under this scheme, an unqualified version reference (such as "3.3") would always refer to the most recent corresponding language or maintenance release. It will never be used without qualification to refer to a standard -library release (at least by python-dev - obviously, we can only set an +library release (at least, not by python-dev - obviously, we can only set an example, not force the rest of the Python ecosystem to go along with it). Language releases will continue as they are now, as new versions of the @@ -126,7 +126,8 @@ release may contain any and all of the following changes: * changes to the AST * any other significant changes to the compilation toolchain * changes to the core interpreter eval loop -* changes to the C ABI (although the PEP 384 stable ABI must be preserved) +* binary incompatible changes to the C ABI (although the PEP 384 stable ABI + must still be preserved) * bug fixes Maintenance releases will also continue as they do today, being strictly @@ -134,9 +135,9 @@ limited to bug fixes for the corresponding language release. No new features or radical internal changes are permitted. The new standard library releases will occur in parallel with each -maintenance release and will be qualified with a date-based version -identifier documenting their *actual* release date. Standard library -releases may include the following changes: +maintenance release and will be qualified with a new version identifier +documenting the standard library version. Standard library releases may +include the following changes: * new features in pure Python modules * new features in C extension modules (subject to PEP 399 compatibility @@ -144,6 +145,11 @@ releases may include the following changes: * new features in language builtins (provided the C ABI remains unaffected) * bug fixes from the corresponding maintenance release +Standard library version identifiers are constructed by combining the major +and minor version numbers for the Python language release into a single two +digit number and then appending a sequential standard library version +identifier. + Release Cycle ------------- @@ -152,53 +158,53 @@ When maintenance releases are created, *two* new versions of Python would actually be published on python.org (using the first 3.3 maintenance release, planned for February 2013 as an example):: - 3.3.1 # Maintenance release - 3.3 (13.02) # Standard library release + 3.3.1 # Maintenance release + 3.3 (33.1) # Standard library release A further 6 months later, the next 3.3 maintenance release would again be accompanied by a new standard library release:: - 3.3.2 # Maintenance release - 3.3 (13.08) # Standard library release + 3.3.2 # Maintenance release + 3.3 (33.2) # Standard library release Again, the standard library release would be binary compatible with the previous language release, merely offering additional features at the Python level. Finally, 18 months after the release of 3.3, a new language release would -be made around the same time as the final 3.3 maintenance release:: +be made around the same time as the final 3.3 maintenance and standard +library releases:: - 3.3.3 # Maintenance release - 3.4.0 # Language release - -Language releases would then contain all the features that are not -permitted in standard library releases: + 3.3.3 # Maintenance release + 3.3 (33.3) # Standard library release + 3.4.0 # Language release The 3.4 release cycle would then follow a similar pattern to that for 3.3:: - 3.4.1 # Maintenance release - 3.4 (14.08) # Standard library release + 3.4.1 # Maintenance release + 3.4 (34.1) # Standard library release - 3.4.2 # Maintenance release - 3.4 (15.02) # Standard library release + 3.4.2 # Maintenance release + 3.4 (34.2) # Standard library release - 3.4.3 # Maintenance release - 3.5 # Language release + 3.4.3 # Maintenance release + 3.4 (34.3) # Standard library release + 3.5.0 # Language release Programmatic Version Identification ----------------------------------- -To expose the new release details programmatically, this PEP proposes the -addition of a new ``sys.stdlib_info`` attribute that records the date based +To expose the new version details programmatically, this PEP proposes the +addition of a new ``sys.stdlib_info`` attribute that records the new standard library version above and beyond the underlying interpreter version. Using the initial Python 3.3 release as an example:: - sys.stdlib_info(year=2012, month=8, micro=0, releaselevel='final', serial=0) + sys.stdlib_info(python=33, version=0, releaselevel='final', serial=0) This information would also be included in the ``sys.version`` string:: - Python 3.3.0 (12.08.0, default, Feb 17 2012, 23:03:41) + Python 3.3.0 (33.0, default, Feb 17 2012, 23:03:41) [GCC 4.6.1] @@ -211,19 +217,11 @@ remains the same as it is now: the minor version number is incremented and a new release is made incorporating the required bug fixes, as well as any other bug fixes that have been committed since the previous release. -For standard library releases, things are slightly more complicated, as the -corresponding source control branch may contain new features in addition to -bug fixes. If such features exist, then, to provide a comprehensible release, -it would be necessary to: +For standard library releases, the process is essentially the same, but the +corresponding "What's New?" document may require some tidying up for the +release (as the standard library release may incorporate new features, +not just bug fixes). -1. Change the release version number to the date of the current month. -2. Update the What's New, NEWS and documentation to refer to the new release - number. -3. Make the new release. - -(Note: due to this problem with out-of-cycle releases, I'm considering an -alternate standard library version scheme that would use date independent -numbers like "33.1" and "33.2". See `Why a date-based versioning scheme?`_). User Scenarios ============== @@ -249,7 +247,7 @@ Novice user, downloading Python from python.org in March 2013 **Status quo:** must choose between 3.3 and 2.7 -**This PEP:** must choose between 3.3 (13.02), 3.3 and 2.7. +**This PEP:** must choose between 3.3 (33.1), 3.3 and 2.7. **PEP 407:** must choose between 3.4, 3.3 (LTS) and 2.7. @@ -264,16 +262,15 @@ Novice user, attempting to judge currency of third party documentation **Status quo:** minor version differences indicate 18-24 months of language evolution -**This PEP:** same as status quo for language core, or just compare the -standard library version numbers to get a rough time in months. +**This PEP:** same as status quo for language core, standard library version +numbers indicate 6 months of standard library evolution. **PEP 407:** minor version differences indicate 18-24 months of language evolution up to 3.3, then 6 months of language evolution thereafter. -**Verdict:** date based numbering schemes with regular release cycles are -a *much* better way to give novices a rough handle on the currency of -information. Since keeping the minor version implications the same is a gain -for *current* Python users, I'm calling this a win twice over for the schemes +**Verdict:** Since language changes and deprecations can have a much bigger +effect on the accuracy of third party documentation than the addition of new +features to the standard library, I'm calling this a win for the scheme in this PEP. @@ -359,7 +356,7 @@ frequent occurrence. doesn't currently spell out a specific development strategy. Assuming a 3.3 compatibility branch is adopted (as proposed in this PEP), then the outcome would be much the same, but the version number signalling would be -slightly less clear (since you would have to look up to see if a particular +slightly less clear (since you would have to check to see if a particular release was an LTS release or not). **Verdict:** while not as clear cut as some previous scenarios, I'm still @@ -380,7 +377,7 @@ documentation, check against ``sys.version_info`` **This PEP:** look for "version added" or "version changed" markers in the documentation. If written as a bare Python version, such as "3.3", check against ``sys.version_info``. If qualified with a standard library version, -such as "3.3 (13.02)", check against ``sys.stdlib_info``. +such as "3.3 (33.1)", check against ``sys.stdlib_info``. **PEP 407:** same as status quo @@ -415,6 +412,26 @@ on our own tracker). However, by including it in ``sys.version``, many fault reports will already include it, and it is easy to request if needed. +CPython release managers, handling a security fix +------------------------------------------------- + +**Status quo:** create a new maintenance release incorporating the security +fix and any other bug fixes under source control. Also create source releases +for any branches open solely for security fixes. + +**This PEP:** same as the status quo for maintenance branches. Also create a +new standard library release (potentially incorporating new features along +with the security fix). For security branches, create source releases for +both the former maintenance branch and the standard library update branch. + +**PEP 407:** same as the status quo for maintenance and security branches, +but handling security fixes for non-LTS releases is currently an open +question. + +**Verdict:** until PEP 407 is updated to actually address this scenario, a +clear win for this PEP. + + Effects ======= @@ -427,10 +444,6 @@ in a language release, each language release will be limited to 6 months worth of standard library changes, as well as any changes associated with new syntax. -If a release date slips by a month or two, the current proposal is to keep -the planned standard library version number rather than updating it to -reflect the actual release date. - Effect on workflow ------------------ @@ -451,14 +464,15 @@ Otherwise it should be checked in directly to ``default``. The "version added" and "version changed" markers for any changes made on the ``3.3-compat`` branch would need to be flagged with both the language -version and the standard library version. For example: "3.3 (13.02)". +version and the standard library version. For example: "3.3 (33.1)". Any changes made directly on the ``default`` branch would just be flagged with "3.4" as usual. -The ``3.3-compat`` branch would be closed after the 3.3+13.08 release, as -the next release at that time will be a full language release and changes -(including standard library changes) should be marked accordingly. +The ``3.3-compat`` branch would be closed to normal development at the +same time as the ``3.3`` maintenance branch. The ``3.3-compat`` branch would +remain open for security fixes for the same period of time as the ``3.3`` +maintenance branch. Effect on bugfix cycle @@ -469,14 +483,14 @@ workflow for new features - there is one additional branch to pass through before the change reaches the ``default`` branch. If critical bugs are found in a maintenance release, then new maintenance and -standard library releases will be created to resolve the problem. The micro -release number will be incremented for both the language version and the -standard library version. +standard library releases will be created to resolve the problem. The final +part of the version number will be incremented for both the language version +and the standard library version. If critical bugs are found in a standard library release that do not affect the associated maintenance release, then only a new standard library release -will be created and only the standard library version's micro release number -will be incremented. +will be created and only the standard library's version number will be +incremented. Note that in these circumstances, the standard library release *may* include additional features, rather than just containing the bug fix. It is @@ -505,8 +519,8 @@ past, I believe perceptions of language stability would also fall (whether such opinions were justified or not). I believe isolating the increased pace of change to the standard library, -and clearly delineating it with a separate date-based version number will -greatly reassure the rest of the community that no, we're not suddenly +and clearly delineating it with a separate version number will greatly +reassure the rest of the community that no, we're not suddenly asking them to triple their own rate of development. Instead, we're merely going to ship standard library updates for the next language release in 6-monthly installments rather than delaying them all until the next language @@ -541,8 +555,9 @@ standard library releases and language releases. So, during the 3.3 release cycle, we would see: * What's New in Python 3.3? -* What's New in Python 3.3 (13.02)? -* What's New in Python 3.3 (13.08)? +* What's New in the Python Standard Library 33.1? +* What's New in the Python Standard Library 33.2? +* What's New in the Python Standard Library 33.3? And then finally, we would see the next language release: @@ -564,58 +579,59 @@ help to some degree, it would be good to eliminate the problem entirely. One suggestion from Barry Warsaw is to adopt a non-conflicting separate-files-per-change approach, similar to that used by Twisted [2_]. -For this PEP, one possible layout for such an approach (to be adopted -following the initial release of 3.3.0+12.8.0 that will use the current NEWS -process) might look like:: +Given that the current manually updated NEWS file will be used for the 3.3.0 +release, one possible layout for such an approach might look like:: Misc/ - news_entries/ - 3.3.1/ # Maintenance branch changes - builtins/ - - extensions/ - - library/ - - documentation/ - - tests/ - - 3.4.0/ # default branch changes - language/ # Only exists for "x.y.0" - - builtins/ - - extensions/ - - library/ - - documentation/ - - tests/ - - 13.02.0/ # 3.3 compatibility branch changes - builtins/ - - extensions/ - - library/ - - documentation/ - - tests/ - NEWS # Now autogenerated from news_entries + news_entries/ + 3.3/ + NEWS # Original 3.3 NEWS file + maint.1/ # Maintenance branch changes + core/ + + builtins/ + + extensions/ + + library/ + + documentation/ + + tests/ + + compat.1/ # Compatibility branch changes + builtins/ + + extensions/ + + library/ + + documentation/ + + tests/ + + # Add maint.2, compat.2 etc as releases are made + 3.4/ + core/ + + builtins/ + + extensions/ + + library/ + + documentation/ + + tests/ + + # Add maint.1, compat.1 etc as releases are made Putting the version information in the directory heirarchy isn't strictly necessary (since the NEWS file generator could figure out from the version history), but does make it easier for *humans* to keep the different versions in order. -Other layouts are obviously also possible (for example, having separate "3.3" -and "3.4" directories to group entries for each language version and the -associated maintenance and standard library releases). - Other benefits of reduced version coupling ========================================== @@ -633,81 +649,17 @@ of the core language definition, it provides an opportunity to actually *slow down* the rate of change in the language definition. The language moratorium for Python 3.2 effectively slowed that cycle down to *more than 3 years* (3.1: June 2009, 3.3: August 2012) without causing any major -complaints. +problems or complaints. The NEWS file management scheme described above is actually designed to allow us the flexibility to slow down language releases at the same time as standard library releases become more frequent. -As simple example, if a full two years was allowed between 3.3 and 3.4, +As a simple example, if a full two years was allowed between 3.3 and 3.4, the 3.3 release cycle would end up looking like:: - 3.2.4 # Maintenance release - 3.3.0 # Language release - - 3.3.1 # Maintenance release - 3.3 (13.02) # Standard library release - - 3.3.2 # Maintenance release - 3.3 (13.08) # Standard library release - - 3.3.3 # Maintenance release - 3.3 (14.02) # Standard library release - - 3.3.4 # Maintenance release - 3.4.0 # Language release - -The elegance of the proposed NEWS entry layout is that this decision -wouldn't need to be made until after the 13.08 standard library release. At -that point, the ``3.3-compat`` branch could be kept open (thus adding -another standard library release to the cycle), or else it could be closed, -committing to the next release being a full language release. If the -compatibility branch was kept open, the choice between another standard -library release or a full language release would then be available every -6 months after that. - - -Further increasing the pace of standard library development ------------------------------------------------------------ - -A further benefit of the scheme proposed in this PEP is that it almost -*fully* decouples the language release cycle from the standard library -release cycle. The standard library could be updated every 3 months, or -even once a month, without having any flow on effects on the language -version numbering or the perceived stability of the core language. - -While that pace of development isn't practical as long as the binary -installer creation for Windows and Mac OS X involves several manual steps -(including manual testing) and for as long as we don't have separate -"-release" trees that only receive versions that have been marked as -good by the stable buildbots, it's a useful criterion to keep in mind when -considering proposed new versioning schemes: what if we eventually want -to make standard library releases even *faster* than every 6 months? - -If the practical issues were ever resolved, then the separate date-based -versioning scheme in this PEP could handle it. The approach proposed in -PEP 407 could not (at least, not without a lot of user confusion). - - -Other Questions -=============== - -Why a date-based versioning scheme? ------------------------------------ - -While a date-based scheme does mean additional work in the case of an -out-of-cycle release, it's an easy way to come up with a numbering system -that is clearly orthogonal to the existing versioning scheme used for -language releases. - -Another possibility would just be ``major.minor`` versioning where: - -* major: xy from the corresponding language release (that is, "33" for 3.3) -* minor: release sequence number for the standard library releases - -Then the Python 3.3 release series would look like:: - - 3.3.0 # Language release + 3.2.4 # Maintenance release + 3.3.0 # Language release 3.3.1 # Maintenance release 3.3 (33.1) # Standard library release @@ -715,13 +667,64 @@ Then the Python 3.3 release series would look like:: 3.3.2 # Maintenance release 3.3 (33.2) # Standard library release - 3.3.3 # Maintenance release - 3.4.0 # Language release + 3.3.3 # Maintenance release + 3.3 (33.3) # Standard library release -Such a scheme would have the advantage of tolerating out-of-cycle releases -without needing to adjust the version numbering for the release, but would -lose the minor benefit of providing clearer age information within the -versioning scheme itself. + 3.3.4 # Maintenance release + 3.3 (33.4) # Standard library release + 3.4.0 # Language release + +The elegance of the proposed branch structure and NEWS entry layout is that +this decision wouldn't really need to be made until shortly before the planned +3.4 release date. At that point, the decision could be made to postpone the +3.4 release and keep the ``3.3`` and ``3.3-compat`` branches open after the +3.3.3 maintenance release and the 3.3 (33.3) standard library release, thus +adding another standard library release to the cycle. The choice between +another standard library release or a full language release would then be +available every 6 months after that. + + +Further increasing the pace of standard library development +----------------------------------------------------------- + +As noted in the previous section, one benefit of the scheme proposed in this +PEP is that it largely decouples the language release cycle from the +standard library release cycle. The standard library could be updated every +3 months, or even once a month, without having any flow on effects on the +language version numbering or the perceived stability of the core language. + +While that pace of development isn't practical as long as the binary +installer creation for Windows and Mac OS X involves several manual steps +(including manual testing) and for as long as we don't have separate +"-release" trees that only receive versions that have been marked as +good by the stable buildbots, it's still a useful criterion to keep in mind +when considering proposed new versioning schemes: what if we eventually want +to make standard library releases even *faster* than every 6 months? + +If the practical issues were ever resolved, then the separate standard +library versioning scheme in this PEP could handle it. The tagged version +number approach proposed in PEP 407 could not (at least, not without a lot +of user confusion and uncertainty). + + +Other Questions +=============== + +Why not a date-based versioning scheme? +--------------------------------------- + +Earlier versions of this PEP proposed a date-based versioning scheme for +the standard library. However, such a scheme made it very difficult to +handle out-of-cycle releases to fix security issues and other critical +bugs in standard library releases, as it required the following steps: + +1. Change the release version number to the date of the current month. +2. Update the What's New, NEWS and documentation to refer to the new release + number. +3. Make the new release. + +With the sequential scheme now proposed, such releases should at most require +a little tidying up of the What's New document before making the release. Why isn't PEP 384 enough? @@ -737,25 +740,85 @@ closely to a specific version of CPython. For existing modules, however, migrating to the stable ABI can involve quite a lot of work (especially for extension modules that define a lot of classes). With limited development resources available, any time spent on such a change is time that could -otherwise have been spent working on features that are offer more direct -benefits to end users. +otherwise have been spent working on features that offer more direct benefits +to end users. + +There are also other benefits to separate versioning (as described above) +that are not directly related to the question of binary compatibility with +third party C extensions. + + +Why no binary compatible additions to the C ABI in standard library releases? +----------------------------------------------------------------------------- + +There's a case to be made that *additions* to the CPython C ABI could +reasonably be permitted in standard library releases. This would give C +extension authors the same freedom as any other package or module author +to depend either on a particular language version or on a standard library +version. + +The PEP currently associates the interpreter version with the language +version, and therefore limits major interpreter changes (including C ABI +additions) to the language releases. + +An alternative, internally consistent, approach would be to link the +interpreter version with the standard library version, with only changes that +may affect backwards compatibility limited to language releases. + +Under such a scheme, the following changes would be acceptable in standard +library releases: + +* Standard library updates + + * new features in pure Python modules + * new features in C extension modules (subject to PEP 399 compatibility + requirements) + * new features in language builtins + +* Interpreter implementation updates + + * binary compatible additions to the C ABI + * changes to the compilation toolchain that do not affect the AST or alter + the bytecode magic number + * changes to the core interpreter eval loop + +* bug fixes from the corresponding maintenance release + +And the following changes would be acceptable in language releases: + +* new language syntax +* any updates acceptable in a standard library release +* new deprecation warnings +* removal of previously deprecated features +* changes to the AST +* changes to the emitted bytecode that require altering the magic number +* binary incompatible changes to the C ABI (although the PEP 384 stable ABI + must still be preserved) + +While such an approach could probably be made to work, there does not appear +to be a compelling justification for it, and the approach currently described +in the PEP is simpler and easier to explain. Why not separate out the standard library entirely? --------------------------------------------------- -Because it's a lot of work for next to no pay-off. CPython without the -standard library is useless (the build chain won't even run, let alone the -test suite). You can't create a standalone pure Python standard library -either, because too many "modules" are actually tightly linked in to the -internal details of their respective interpreters (e.g. ``weakref``, ``gc``, -``sys``, ``inspect``, ``ast``). +A concept that is occasionally discussed is the idea of making the standard +library truly independent from the CPython reference implementation. -Creating a separate development branch that is kept compatible with the -previous feature release, and making releases from that branch that are -flagged with a separate date-based version number should provide most of -the benefits of a separate standard library repository with only a fraction -of the pain. +My personal opinion is that actually making such a change would involve a +lot of work for next to no pay-off. CPython without the standard library is +useless (the build chain won't even run, let alone the test suite). You also +can't create a standalone pure Python standard library either, because too +many "standard library modules" are actually tightly linked in to the +internal details of their respective interpreters (for example, the builtins, +``weakref``, ``gc``, ``sys``, ``inspect``, ``ast``). + +Creating a separate CPython development branch that is kept compatible with +the previous language release, and making releases from that branch that are +identified with a separate standard library version number should provide +most of the benefits of a separate standard library repository with only a +fraction of the pain. Acknowledgements From ca5ec0e933e9acf503b8dd6272c9a2177b5e62e3 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Sun, 26 Feb 2012 15:33:31 +1000 Subject: [PATCH 3/8] Describe two more alternate numbering schemes --- pep-0413.txt | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/pep-0413.txt b/pep-0413.txt index 02f7df06c..d0487e73a 100644 --- a/pep-0413.txt +++ b/pep-0413.txt @@ -710,9 +710,75 @@ of user confusion and uncertainty). Other Questions =============== -Why not a date-based versioning scheme? +Why not use the major version number? +------------------------------------- + +The simplest and most logical solution would actually be to map the +major.minor.micro version numbers to the language version, stdlib version +and maintenance release version respectively. + +Instead of releasing Python 3.3.0, we would instead release Python 4.0.0 +and the release cycle would look like:: + + 4.0.0 # Language release + + 4.0.1 # Maintenance release + 4.1.0 # Standard library release + + 4.0.2 # Maintenance release + 4.2.0 # Standard library release + + 4.0.3 # Maintenance release + 4.3.0 # Standard library release + 5.0.0 # Language release + +However, the ongoing pain of the Python 2 -> Python 3 transition (and +associated workarounds like the ``python3`` and ``python2`` symlinks to +refer directly to the desired release series) means that this simple option +isn't viable for historical reasons. + +One way that this simple approach *could* be made to work is to merge the +current major and minor version numbers directly into a 2-digit major +version number:: + + 33.0.0 # Language release + + 33.0.1 # Maintenance release + 33.1.0 # Standard library release + + 33.0.2 # Maintenance release + 33.2.0 # Standard library release + + 33.0.3 # Maintenance release + 33.3.0 # Standard library release + 34.0.0 # Language release + + +Why not use a four part version number? --------------------------------------- +Another simple versioning scheme would just add a "standard library" version +into the existing versioning scheme:: + + 3.3.0.0 # Language release + + 3.3.0.1 # Maintenance release + 3.3.1.0 # Standard library release + + 3.3.0.2 # Maintenance release + 3.3.2.0 # Standard library release + + 3.3.0.3 # Maintenance release + 3.3.3.0 # Standard library release + 3.4.0.0 # Language release + +However, this scheme isn't viable due to backwards compatibility constraints +on the ``sys.version_info`` structure. + + +Why not use a date-based versioning scheme? +------------------------------------------- + Earlier versions of this PEP proposed a date-based versioning scheme for the standard library. However, such a scheme made it very difficult to handle out-of-cycle releases to fix security issues and other critical From 574066ff1104e0018b816c76708657264179b815 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Sun, 26 Feb 2012 15:37:24 -0800 Subject: [PATCH 4/8] Reject PEP 410 --- pep-0410.txt | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/pep-0410.txt b/pep-0410.txt index bccfc3630..deb480df2 100644 --- a/pep-0410.txt +++ b/pep-0410.txt @@ -3,13 +3,20 @@ Title: Use decimal.Decimal type for timestamps Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/x-rst Created: 01-February-2012 Python-Version: 3.3 +Rejection Notice +================ + +This PEP is rejected. +See http://mail.python.org/pipermail/python-dev/2012-February/116837.html. + + Abstract ======== From 91cc261ef62e84c3416fc8debf8b70a9bf00e378 Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Sun, 26 Feb 2012 20:29:14 -0500 Subject: [PATCH 5/8] pep for changing pep 409 implementation --- pep-0415.txt | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 pep-0415.txt diff --git a/pep-0415.txt b/pep-0415.txt new file mode 100644 index 000000000..049fff028 --- /dev/null +++ b/pep-0415.txt @@ -0,0 +1,78 @@ +PEP: 415 +Title: Implementing PEP 409 differently +Version: $Revision$ +Last-Modified: $Date$ +Author: Benjamin Peterson +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 26-Feb-2012 +Post-History: 26-Feb-2012 + + +Abstract +======== + +PEP 409 allows PEP 3134 exception contexts and causes to be suppressed when the +exception is printed. This is done using the ``raise exc from None`` +syntax. This PEP proposes to implement context and cause suppression +differently. + +Rationale +========= + +PEP 409 changes ``__cause__`` to be ``Ellipsis`` by default. Then if +``__cause__`` is set to ``None`` by ``raise exc from None``, no context or cause +will be printed should the exception be uncaught. + +The main problem with this scheme is it complicates the role of +``__cause__``. ``__cause__`` should indicate the cause of the exception not +whether ``__context__`` should be printed or not. This use of ``__cause__`` is +also not easily extended in the future. For example, we may someday want to +allow the programmer to select which of ``__context__`` and ``__cause__`` will +be printed. The PEP 409 implementation is not amendable to this. + +The use of ``Ellipsis`` is a hack. Before PEP 409, ``Ellipsis`` was used +exclusively in extended slicing. Extended slicing has nothing to do with +exceptions, so it's not clear to someone inspecting an exception object why +``__cause__`` should be set to ``Ellipsis``. Using ``Ellipsis`` by default for +``__cause__`` makes it asymmetrical with ``__context__``. + +Proposal +======== + +A new attribute on ``BaseException``, ``__suppress_context__``, will be +introduced. The ``raise exc from None`` syntax will cause +``exc.__suppress_context__`` to be set to ``True``. Exception printing code will +check for the attribute to determine whether context and cause will be +printed. ``__cause__`` will return to its original purpose and values. + +There is precedence for ``__suppress_context__`` with the +``print_line_and_file`` exception attribute. + +Patches +======= + +There is a patch on `Issue 14133`_. + + +References +========== + +.. _issue 14133: + http://bugs.python.org/issue6210 + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From d3966c5fcde468b981d2baef95a623157d5a4d23 Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Sun, 26 Feb 2012 20:39:52 -0500 Subject: [PATCH 6/8] kill extra letter --- pep-0415.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0415.txt b/pep-0415.txt index 049fff028..019e877a1 100644 --- a/pep-0415.txt +++ b/pep-0415.txt @@ -30,7 +30,7 @@ The main problem with this scheme is it complicates the role of whether ``__context__`` should be printed or not. This use of ``__cause__`` is also not easily extended in the future. For example, we may someday want to allow the programmer to select which of ``__context__`` and ``__cause__`` will -be printed. The PEP 409 implementation is not amendable to this. +be printed. The PEP 409 implementation is not amenable to this. The use of ``Ellipsis`` is a hack. Before PEP 409, ``Ellipsis`` was used exclusively in extended slicing. Extended slicing has nothing to do with From 43650eddca09e3ea04e52b7c0bb097ccb36cc27f Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Mon, 27 Feb 2012 10:51:11 -0500 Subject: [PATCH 7/8] give equivalent code for raise from --- pep-0415.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/pep-0415.txt b/pep-0415.txt index 019e877a1..653ed78aa 100644 --- a/pep-0415.txt +++ b/pep-0415.txt @@ -50,6 +50,12 @@ printed. ``__cause__`` will return to its original purpose and values. There is precedence for ``__suppress_context__`` with the ``print_line_and_file`` exception attribute. +To summarize, ``raise exc from cause`` will be equivalent to:: + + exc.__cause__ = cause + exc.__suppress_context__ = cause is None + raise exc + Patches ======= From 8a61da282ae8f946e468da7635ef0898d1e762ba Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Mon, 27 Feb 2012 18:59:27 -0500 Subject: [PATCH 8/8] I understand what Nick meant now --- pep-0415.txt | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/pep-0415.txt b/pep-0415.txt index 653ed78aa..aa6e1233f 100644 --- a/pep-0415.txt +++ b/pep-0415.txt @@ -41,11 +41,12 @@ exceptions, so it's not clear to someone inspecting an exception object why Proposal ======== -A new attribute on ``BaseException``, ``__suppress_context__``, will be -introduced. The ``raise exc from None`` syntax will cause -``exc.__suppress_context__`` to be set to ``True``. Exception printing code will -check for the attribute to determine whether context and cause will be -printed. ``__cause__`` will return to its original purpose and values. +A new attribute on ``BaseException``, ``__suppress_context__``, will +be introduced. The ``raise exc from cause`` syntax will set +``exc.__suppress_context__`` to ``True``. Exception printing code will +check for that attribute to determine whether context and cause will +be printed. ``__cause__`` will return to its original purpose and +values. There is precedence for ``__suppress_context__`` with the ``print_line_and_file`` exception attribute. @@ -53,7 +54,7 @@ There is precedence for ``__suppress_context__`` with the To summarize, ``raise exc from cause`` will be equivalent to:: exc.__cause__ = cause - exc.__suppress_context__ = cause is None + exc.__suppress_context__ = True raise exc Patches