PEP 598: Fix terminology, reference support policy NEP (#1134)
- several Scientific Python projects are working on a common version support policy for their projects, which we need to take into account when considering changes to our release cadence - reviewing that draft NEP highlighted several problems with the way this PEP was using the "major" and "minor" terms, so this takes a pass through that, replacing inappropriate uses of "major" with "feature", and and inapproriate uses of "minor" with "micro" - I also went through and collected statistics on how many changes we've been making in micro releases that warranted version added/changed notes in the docs
This commit is contained in:
Nick Coghlan
GitHub
parent
03b80ccc83
commit
2c960ef584
201
pep-0598.rst
201
pep-0598.rst
|
|
@ -16,31 +16,31 @@ Abstract
|
|||
|
||||
PEP 596 has proposed reducing the feature delivery latency for the Python
|
||||
standard library and CPython reference interpreter by increasing the frequency
|
||||
of major CPython releases from every 18-24 months to instead occur every 9-12
|
||||
of CPython feature releases from every 18-24 months to instead occur every 9-12
|
||||
months.
|
||||
|
||||
Adopting such an approach has several significant practical downsides, as a
|
||||
CPython major release carries certain expectations (most notably, a 5 year
|
||||
CPython feature release carries certain expectations (most notably, a 5 year
|
||||
maintenance lifecycle, support for parallel installation with the previous
|
||||
major release, and the possibility of breaking chnages to the CPython-specific
|
||||
ABI, requiring recompilation of all extension modules) that mean faster major
|
||||
feature release, and the possibility of breaking changes to the CPython-specific
|
||||
ABI, requiring recompilation of all extension modules) that mean faster feature
|
||||
releases would significantly increase the burden of maintaining 3rd party
|
||||
Python libraries and applications across all actively supported CPython
|
||||
releases.
|
||||
|
||||
It's also arguable whether such an approach would actually noticeably reduce
|
||||
the typical feature delivery latency, as the adoption cycle for new major
|
||||
the typical feature delivery latency, as the adoption cycle for new feature
|
||||
releases is typically measured in months or years, so more frequent releases
|
||||
may just lead to end users updating to every 3rd or 4th major release, rather
|
||||
than every 2nd or 3rd major release (as already happens today).
|
||||
may just lead to end users updating to every 3rd or 4th feature release, rather
|
||||
than every 2nd or 3rd feature release (as already happens today).
|
||||
|
||||
This PEP presents a competing proposal to instead *slow down* the frequency of
|
||||
parallel installable major feature releases that change the filesystem layout
|
||||
parallel installable feature releases that change the filesystem layout
|
||||
and CPython ABI to a consistent 24 month cycle, but to compensate for this by
|
||||
introducing the notion of build compatible incremental feature releases, and
|
||||
then deferring the full feature freeze of a given major release series from the
|
||||
initial baseline X.Y.0 release to a subsequent X.Y.Z feature complete release
|
||||
that occurs ~12 months after the initial baseline feature release.
|
||||
then deferring the full feature freeze of a given feature release series from
|
||||
the initial baseline X.Y.0 release to a subsequent X.Y.Z feature complete
|
||||
release that occurs ~12 months after the initial baseline feature release.
|
||||
|
||||
A new ``feature_complete`` attribute on the ``sys.version_info`` structure will
|
||||
provide a programmatic indicator as to whether or not a release series remains
|
||||
|
|
@ -58,12 +58,12 @@ Example Future Release Schedules
|
|||
================================
|
||||
|
||||
Under this proposal, Python 3.9.0a1 would be released in November 2019, shortly
|
||||
after the Python 3.8.0 feature freeze release in October 2019.
|
||||
after the Python 3.8.0 feature complete release in October 2019.
|
||||
|
||||
The 3.9.0b1 release would then follow 6 months later in May 2020, with 3.9.0
|
||||
itself being released in October 2020.
|
||||
|
||||
Assuming minor releases of 3.9.x were to occur quarterly, then the overall
|
||||
Assuming micro releases of 3.9.x were to occur quarterly, then the overall
|
||||
release timeline would look like:
|
||||
|
||||
* 2019-11: 3.9.0a1
|
||||
|
|
@ -134,12 +134,12 @@ TODO: this really needs a diagram to help explain it, so I'll add a picture
|
|||
once I have one to add.
|
||||
|
||||
This is quite similar to the status quo, but with a more consistent cadence,
|
||||
alternating between major feature release years (2020, 2022, etc) that focus
|
||||
alternating between baseline feature release years (2020, 2022, etc) that focus
|
||||
on the alpha and beta cycle for a new baseline feature release (while continuing
|
||||
to publish maintenance releases for the previous major release series), and
|
||||
incremental feature release years (2021, 2023, etc), that focus on making
|
||||
smaller improvements to the current major release series (while making plans
|
||||
for the next major release series the following year).
|
||||
to publish maintenance releases for the previous feature release series), and
|
||||
feature complete release years (2021, 2023, etc), that focus on making
|
||||
smaller improvements to the current feature release series (while making plans
|
||||
for the next feature release series the following year).
|
||||
|
||||
|
||||
Proposal
|
||||
|
|
@ -193,15 +193,15 @@ would qualify as a "feature release" in the PEP 13 sense (incremental feature
|
|||
releases wouldn't count).
|
||||
|
||||
|
||||
Baseline feature releases and major release series
|
||||
--------------------------------------------------
|
||||
Baseline feature releases and feature release series
|
||||
----------------------------------------------------
|
||||
|
||||
Baseline feature releases are essentially just the existing feature releases,
|
||||
given a new name to help distinguish them from the new incremental feature
|
||||
releases, and also to help indicate that unlike their predecessors, they are
|
||||
no longer considered feature complete at release.
|
||||
|
||||
Baseline feature releases would continue to define a new major release series,
|
||||
Baseline feature releases would continue to define a new feature release series,
|
||||
locking in the following language, build, and installation compatibility
|
||||
constraints for the remainder of that series:
|
||||
|
||||
|
|
@ -222,18 +222,18 @@ Baseline feature releases would also continue to be the only releases where:
|
|||
- other changes requiring "Porting to Python X.Y" entries in the What's New
|
||||
document can be introduced
|
||||
|
||||
Key characteristics of a major release series:
|
||||
Key characteristics of a feature release series:
|
||||
|
||||
- an installation within one major release series does not conflict with
|
||||
installations of other major release series (i.e. they can be installed in parallel)
|
||||
- an installation within a major release series can be updated to a later minor
|
||||
release within the same series without requiring reinstallation or any other
|
||||
changes to previously installed components
|
||||
- an installation within one feature release series does not conflict with
|
||||
installations of other feature release series (i.e. they can be installed in parallel)
|
||||
- an installation within a feature release series can be updated to a later
|
||||
micro release within the same series without requiring reinstallation
|
||||
or any other changes to previously installed components
|
||||
|
||||
Key characteristics of a baseline feature release:
|
||||
|
||||
- in a baseline feature release, ``sys.version_info.feature_complete == False``
|
||||
- in a baseline feature release, ``sys.version_info.minor == 0``
|
||||
- in a baseline feature release, ``sys.version_info.micro == 0``
|
||||
- baseline feature releases may contain higher risk changes to the language and
|
||||
interpreter, such as grammar modifications, major refactoring of interpreter
|
||||
and standard library internals, or potentially invasive feature additions that
|
||||
|
|
@ -242,16 +242,16 @@ Key characteristics of a baseline feature release:
|
|||
permitted to rely on ``sys.version_info`` as their sole runtime indicator
|
||||
of the feature's availability
|
||||
|
||||
Key expectations around major release series and baseline feature releases:
|
||||
Key expectations around feature release series and baseline feature releases:
|
||||
|
||||
- most public projects will only actively test against the *most recent*
|
||||
minor release within a release series
|
||||
micro release within a release series
|
||||
- many (most?) public projects will only add a new release series to their test
|
||||
matrix *after* the initial baseline feature release has already been published,
|
||||
which can make it difficult to resolve issues that require providing new flags
|
||||
or APIs to explicitly opt-in to old behaviour after a default behaviour changed
|
||||
- private projects with known target environments will test against whichever
|
||||
minor release version they're actually using
|
||||
micro release version they're actually using
|
||||
- most private projects will also only consider migrating to a new release
|
||||
series *after* the initial baseline feature release has already been published,
|
||||
again posing a problem if the resolution of their problems requires an API
|
||||
|
|
@ -288,12 +288,12 @@ relaxed policies around API additions and enhancements:
|
|||
The intent of this change in policy is to allow usability improvements for new
|
||||
(and existing!) language features to be delivered in a more timely fashion,
|
||||
rather than requiring users to incur the inherent delay and costs of waiting for
|
||||
and then upgrading to the next major release series.
|
||||
and then upgrading to the next feature release series.
|
||||
|
||||
Key characteristics of an incremental feature release:
|
||||
|
||||
- in an incremental feature release, ``sys.version_info.feature_complete == False``
|
||||
- in an incremental feature release, ``sys.version_info.minor != 0``
|
||||
- in an incremental feature release, ``sys.version_info.micro != 0``
|
||||
- all API additions made in an incremental feature release must support
|
||||
efficient runtime feature detection that doesn't rely on either
|
||||
``sys.version_info`` or runtime code object introspection. In most cases, a
|
||||
|
|
@ -307,7 +307,7 @@ Key characteristics of an incremental feature release:
|
|||
Key expectations around incremental feature releases:
|
||||
|
||||
- "don't break existing installations on upgrade" remains a key requirement
|
||||
for all minor releases, even with the more permissive change inclusion policy
|
||||
for all micro releases, even with the more permissive change inclusion policy
|
||||
- more intrusive changes should still be deferred to the next baseline feature
|
||||
release
|
||||
- public Python projects that start relying on features added in an incremental
|
||||
|
|
@ -326,7 +326,7 @@ permitted to add algorithms to ``hashlib.algorithms_available``)
|
|||
Feature complete release and subsequent maintenance releases
|
||||
------------------------------------------------------------
|
||||
|
||||
The feature complete release for a given major release series would be
|
||||
The feature complete release for a given feature release series would be
|
||||
developed under the normal policy for an incremental feature release, but
|
||||
would have one distinguishing feature:
|
||||
|
||||
|
|
@ -350,6 +350,28 @@ long as an API remains in the provisional state, regular backwards compatibility
|
|||
requirements would not apply to that API in baseline and incremental feature
|
||||
releases.
|
||||
|
||||
This policy is expected to provide increased clarity to end users (as even
|
||||
provisional APIs will become stable for that release series in the feature
|
||||
complete release), with minimal practical downsides for standard library
|
||||
maintainers, based on the following analysis of documented API additions and
|
||||
changes in micro releases of CPython since 3.0.0:
|
||||
|
||||
* 21 3.x.1 version added/changed notes
|
||||
* 30 3.x.2 version added/changed notes
|
||||
* 18 3.x.3 version added/changed notes
|
||||
* 11 3.x.4 version added/changed notes
|
||||
* 1 3.x.5 version added/changed notes
|
||||
* 0 3.x.6+ version added/changed notes
|
||||
|
||||
When post-baseline-release changes need to be made, the majority of them occur
|
||||
within the first two maintenance releases, which have always occurred within 12
|
||||
months of the baseline release.
|
||||
|
||||
(Note: these counts are not solely for provisional APIs - they cover all APIs
|
||||
where semantic changes were made after the baseline release that were considered
|
||||
necessary to cover in the documentation. To avoid double counting changes, the
|
||||
numbers exclude any change markers from the What's New section)
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
|
@ -367,7 +389,7 @@ ABI, and just generally having a high chance of inflicting a relatively high
|
|||
level of additional cost across the entire Python ecosystem.
|
||||
|
||||
The view taken in this PEP is that there's an alternative approach that provides
|
||||
most of the benefits of a faster major release without actually incurring the
|
||||
most of the benefits of a faster feature release without actually incurring the
|
||||
associated costs: we can split the current X.Y.0 "feature freeze" into two
|
||||
parts, such that the baseline X.Y.0 release only imposes a
|
||||
"runtime compatibility freeze", and the full standard library feature freeze
|
||||
|
|
@ -387,16 +409,16 @@ sprints). However, part of the goal of this proposal is to provide a consistent
|
|||
annual cadence for both contributors and end users, so adjustments ideally would
|
||||
be rare.
|
||||
|
||||
This PEP does not dictate a specific cadence for minor releases within a release
|
||||
series - it just specifies the rouch timelines for transitions between the
|
||||
This PEP does not dictate a specific cadence for micro releases within a release
|
||||
series - it just specifies the rough timelines for transitions between the
|
||||
release series lifecycle phases (pre-alpha, alpha, beta, feature releases,
|
||||
bug fixes, security fixes). The number of minor releases within each phase is
|
||||
bug fixes, security fixes). The number of micro releases within each phase is
|
||||
determined by the release manager for that series based on how frequently they
|
||||
and the rest of the release team for that series are prepared to undertake the
|
||||
associated work.
|
||||
|
||||
However, for the sake of the example timelines, the PEP assumes quarterly
|
||||
minor releases (the cadence used for Python 3.6 and 3.7, splitting the
|
||||
micro releases (the cadence used for Python 3.6 and 3.7, splitting the
|
||||
difference between the twice yearly cadence used for some historical release
|
||||
series, and the monthly cadence planned for Python 3.8 and 3.9).
|
||||
|
||||
|
|
@ -404,11 +426,11 @@ series, and the monthly cadence planned for Python 3.8 and 3.9).
|
|||
Design Discussion
|
||||
=================
|
||||
|
||||
Why this proposal over simply doing more frequent major releases?
|
||||
-----------------------------------------------------------------
|
||||
Why this proposal over simply doing more frequent baseline feature releases?
|
||||
----------------------------------------------------------------------------
|
||||
|
||||
The filesystem layout changes and other inherently incompatible changes involved
|
||||
in a major version update create additional work for large sections of the
|
||||
in a baseline feature release create additional work for large sections of the
|
||||
wider Python community.
|
||||
|
||||
Decoupling those layout changes from the Python version numbering scheme is also
|
||||
|
|
@ -418,7 +440,7 @@ install over the top of each other, and which can be installed in parallel on
|
|||
a single system.
|
||||
|
||||
We also don't have a straightforward means to communicate to the community
|
||||
variations in support periods like "Only support major version X.Y until
|
||||
variations in support periods like "Only support Python version X.Y until
|
||||
X.Y+1 is out, but support X.Z until X.Z+2 is out".
|
||||
|
||||
So this PEP takes as its starting assumption that the vast majority of Python
|
||||
|
|
@ -433,7 +455,7 @@ Implications for Python library development
|
|||
-------------------------------------------
|
||||
|
||||
Many Python libraries (both open source and proprietary) currently adopt the
|
||||
practice of testing solely against the latest minor release within each major
|
||||
practice of testing solely against the latest micro release within each feature
|
||||
release series that the project still supports.
|
||||
|
||||
The design assumption in this PEP is that this practice will continue to be
|
||||
|
|
@ -441,21 +463,59 @@ followed during the feature release phase of a release series, with the
|
|||
expectation being that anyone choosing to adopt a new release series before it
|
||||
is feature complete will closely track the incremental feature releases.
|
||||
|
||||
Libraries that support a previous major release series are unlikely to adopt
|
||||
Libraries that support a previous feature release series are unlikely to adopt
|
||||
features added in an incremental feature release, and if they do adopt such
|
||||
a feature, then any associated fallback compatibility strategies should be
|
||||
implemented in such a way that they're also effective on the earlier releases
|
||||
in that release series.
|
||||
|
||||
|
||||
Implications for the proposed Scientific Python ecosystem support period
|
||||
------------------------------------------------------------------------
|
||||
|
||||
Based on discussions at SciPy 2019, a NEP is currently being drafted [2_] to
|
||||
define a common convention across the Scientific Python ecosystem for dropping
|
||||
support for older Python versions.
|
||||
|
||||
While the exact formulation of that policy is still being discussed, the initial
|
||||
proposal was very simple: support any Python feature release published within
|
||||
the last 42 months.
|
||||
|
||||
For an 18 month feature release cadence, that works out to always supporting at
|
||||
least the two most recent feature releases, and then dropping support for all
|
||||
X.Y.z releases around 6 months after X.(Y+2).0 is released. This means there is
|
||||
a 6 month period roughly every other year where the three most recent feature
|
||||
releases are supported.
|
||||
|
||||
For a 12 month release cadence, it would work out to always supporting at
|
||||
least the three most recent feature releases, and then dropping support for all
|
||||
X.Y.z releases around 6 months after X.(Y+3).0 is released. This means that
|
||||
for half of each year, the four most recent feature releases would be supported.
|
||||
|
||||
For a 24 month release cadence, a 42 month support cycle works out to always
|
||||
supporting at least the most recent feature release, and then dropping support
|
||||
for all X.Y.z feature releases around 18 months after X.(Y+1).0 is released.
|
||||
This means there is a 6 month period every other year where only one feature
|
||||
release is supported (and that period overlaps with the pre-release testing
|
||||
period for the X.(Y+2).0 baseline feature release).
|
||||
|
||||
Importantly for the proposal in this PEP, that support period would abide by
|
||||
the recommendation that library developers maintain support for the previous
|
||||
release series until the latest release series has attained feature complete
|
||||
status: dropping support 18 months after the baseline feature release will be
|
||||
roughly equivalent to dropping support 6 months after the feature complete
|
||||
release, without needing to track exactly *which* release marked the series as
|
||||
feature complete.
|
||||
|
||||
|
||||
Implications for simple deployment environments
|
||||
-----------------------------------------------
|
||||
|
||||
For the purposes of this PEP, a "simple" deployment environment is any use case
|
||||
where it is straightforward to ensure that all target environments are updated
|
||||
to a new Python minor version at the same time (or at least in advance of the
|
||||
to a new Python micro version at the same time (or at least in advance of the
|
||||
rollout of new higher level application versions), such that any pre-release
|
||||
testing that occurs need only target a single Python minor version.
|
||||
testing that occurs need only target a single Python micro version.
|
||||
|
||||
The simplest such case would be scripting for personal use, where the testing
|
||||
and target environments are the exact same environment.
|
||||
|
|
@ -466,7 +526,7 @@ any application that bundles its own Python runtime, rather than relying on a
|
|||
pre-existing Python deployment on the target system.
|
||||
|
||||
For these use cases, this PEP shouldn't have any significant implications - only
|
||||
a single minor Python version needs to be tested, independently of whether that
|
||||
a single micro version needs to be tested, independently of whether that
|
||||
version is feature complete or not.
|
||||
|
||||
|
||||
|
|
@ -475,9 +535,9 @@ Implications for complex deployment environments
|
|||
|
||||
For the purposes of this PEP, "complex" deployment environments are use cases
|
||||
which don't meet the "simple deployment" criterion above: new application
|
||||
versions are combined with two or more distinct minor Python versions within
|
||||
versions are combined with two or more distinct micro versions within
|
||||
the same release series as part of the deployment process, rather than always
|
||||
targeting exactly one minor version at a time.
|
||||
targeting exactly one micro version at a time.
|
||||
|
||||
If the proposal in this PEP has the desired effect of reducing feature delivery
|
||||
latency, then it can be expected that developers using a release series that is
|
||||
|
|
@ -489,9 +549,9 @@ newer maintenance release is for older maintenance releases.
|
|||
|
||||
One option for handling such cases is to simply prohibit the use of new Python
|
||||
versions until the series has reached "feature complete" status. Such a policy
|
||||
is effectively already adopted by many organisations when it comes to new major
|
||||
feature releases, with acceptance into operational environments occurring months
|
||||
or years after the original release. If this policy is adopted, then such
|
||||
is effectively already adopted by many organisations when it comes to new
|
||||
feature release series, with acceptance into operational environments occurring
|
||||
months or years after the original release. If this policy is adopted, then such
|
||||
organisations could potentially still adopt a new Python version every other
|
||||
year - it would just be based on the availability of the feature complete
|
||||
releases, rather than the baseline feature releases.
|
||||
|
|
@ -505,7 +565,7 @@ However, a third variant, which allows selective adoption of new language
|
|||
features where appropriate, while also degrading gracefully enough to be
|
||||
suitable for mission critical applications, would be to institute a policy that
|
||||
applications wishing to target a release series that is not yet feature complete
|
||||
must also support the previous major release series for compatibility testing
|
||||
must also support the previous feature release series for compatibility testing
|
||||
purposes.
|
||||
|
||||
If this last policy is adopted, then testing against the previous release series
|
||||
|
|
@ -525,16 +585,16 @@ any additional safeguards in their testing process.
|
|||
Depending on demand and interest, there are also further enhancements that could
|
||||
be made to continuous integration pipelines to help ensure compatibility with
|
||||
a chosen minimum version within a release series, without needing to run tests
|
||||
against multiple minor releases. For example, applications could potentially be
|
||||
tested against the latest minor feature release, but typechecked against the
|
||||
oldest still deployed minor feature release.
|
||||
against multiple micro releases. For example, applications could potentially be
|
||||
tested against the latest incremental feature release, but typechecked against
|
||||
the oldest still deployed incremental feature release.
|
||||
|
||||
|
||||
Duration of the feature additions period
|
||||
----------------------------------------
|
||||
|
||||
This PEP proposes that feature additions be limited to 12 months after the
|
||||
initial major feature release.
|
||||
initial baseline feature release.
|
||||
|
||||
The primary motivation for that is specifically to sync up with the Ubuntu LTS
|
||||
timing, such that the feature complete release for the Python 3.9.x series gets
|
||||
|
|
@ -545,13 +605,13 @@ align with stable versions of their inputs. Canonical deliberately haven't
|
|||
given themselves that flexibility with their own release cycle).
|
||||
|
||||
The 12 month feature addition period then arises from splitting the time
|
||||
from the 2019-10 release of Python 3.8.0 and a final Python 3.9.x minor feature
|
||||
release in 2021-10 evenly between pre-release development and subsequent
|
||||
minor feature releases.
|
||||
from the 2019-10 release of Python 3.8.0 and a final Python 3.9.x incremental
|
||||
feature release in 2021-10 evenly between pre-release development and subsequent
|
||||
incremental feature releases.
|
||||
|
||||
This is an area where this PEP could adopt part of the proposal in PEP 596,
|
||||
by instead making that split ~9 months of pre-release development, and ~15
|
||||
months of minor feature releases:
|
||||
months of incremental feature releases:
|
||||
|
||||
* 2019-11: 3.9.0a1
|
||||
* ... additional alpha releases as determined by the release manager
|
||||
|
|
@ -609,11 +669,11 @@ few months later.
|
|||
While alternating the annual cadence between "big foundational enhancements"
|
||||
and "targeted low risk API usability improvements" is a deliberate feature of
|
||||
this proposal, it still seems strange to wait that long for feedback in the
|
||||
event that changes *are* made shortly after the previous major version is
|
||||
event that changes *are* made shortly after the previous release series is
|
||||
branched.
|
||||
|
||||
An alternative way of handling this would be to start publishing alpha releases
|
||||
for the next major feature release during the feature addition period (similar
|
||||
for the next baseline feature release during the feature addition period (similar
|
||||
to the way that PEP 596 proposes to starting publishing Python 3.9.0 alpha
|
||||
releases during the Python 3.8.0 release candidate period).
|
||||
|
||||
|
|
@ -644,11 +704,11 @@ release types.
|
|||
|
||||
If the proposal in this PEP were to be accepted by the Steering Council for
|
||||
Python 3.9, then a better time to tackle that technical question would be for
|
||||
the subsequent October 2022 major feature release, as there are already inherent
|
||||
the subsequent October 2022 baseline feature release, as there are already inherent
|
||||
compatibility risks associated with the choice of either "Python 4.0" (erroneous
|
||||
checks for the major version being exactly 3 rather than 3 or greater), or
|
||||
"Python 3.10" (code incorrectly assuming that the minor version will always
|
||||
contain exactly one decimal digit) [1].
|
||||
contain exactly one decimal digit) [1_].
|
||||
|
||||
While the text of this PEP assumes that the release published in 2022 will be
|
||||
3.10 (as the PEP author personally considers that the more reasonable and most
|
||||
|
|
@ -697,8 +757,11 @@ And the 5 year schedule forecast would look like:
|
|||
References
|
||||
==========
|
||||
|
||||
[1] Anthony Sottile created a pseudo "Python 3.10" to find and fix such issues
|
||||
(https://github.com/asottile/python3.10)
|
||||
.. [1] Anthony Sottile created a pseudo "Python 3.10" to find and fix such issues
|
||||
(https://github.com/asottile/python3.10)
|
||||
|
||||
.. [2] NEP proposing a standard policy for dropping support of old Python versions
|
||||
(https://github.com/numpy/numpy/pull/14086)
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
|
|
|||
Loading…
Reference in New Issue