iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 605: Change ABI management to encourage pre-built wheels (#1190) 

* Change ABI management to encourage pre-built wheels
* Actually dedicate a subsection to the 2 year cadence
* Update naming discussion for proposal changes
* Add discussion section for the pre-freeze flag
* Make the abstract much shorter (as suggested by Antoine Pitrou)
* Move example time line up and add pictures (based on PEP 602 images)
* Clarify 4 month window for X.Y.0a1 preparation

Co-Authored-By: Steve Dower <steve.dower@microsoft.com>
This commit is contained in:
Nick Coghlan 2019-10-06 00:04:21 +10:00 committed by GitHub
parent 16da804bc6
commit 6a05714aef
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 460 additions and 231 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
pep-0605-example-release-calendar.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 96 KiB

BIN
pep-0605-overlapping-support-matrix.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 76 KiB

691
pep-0605.rst
View File

@ -18,43 +18,110 @@ Abstract
Rather than proposing more frequent full CPython releases (as PEP 602 does),
or a policy change to allow backwards compatible feature additions later in a
release series (as PEP 598 does), this PEP instead proposes that we create a
rolling stream of production-ready beta releases by amending CPython's
pre-release management process as follows:
rolling stream of production-ready beta releases, together with alpha releases
that are specifically to be suitable as platforms for building extension modules
and wheel archives that are compatible with the subsequent beta releases.
* Feature freeze, ABI freeze, pyc file format freeze, and maintenance branch
creation all correspond with the first release candidate for an X.Y.0 release
* The X.Y.0 release candidate period is extended from 3 weeks to 2 months
* Beta releases from the master git branch would occur every 2 months
whenever there is no X.Y.0 release in the release candidate phase
* Routine alpha releases are removed from the process
* Release managers may still choose to mark particular releases as alpha
releases if they feel such releases are appropriate. If any such alpha
releases are made, it is expected that they would be in place of one of the
scheduled beta releases
* The deprecation policy for interfaces that have been included in an X.Y.0
release or release candidate without being explicitly declared provisional
remains unchanged (refer to PEP 387 for details)
* Each new alpha or beta release may introduce new deprecation warnings for
interfaces that appeared in the previous X.Y release series
* Each new alpha or beta release may change or remove interfaces that either did
not appear in the previous X.Y release series, or emitted a relevant
deprecation warning in that series
* Additions to the `abi3` stable C ABI are permanent once they have been
declared to be part of that ABI in at least one beta release
* All other newly added interfaces that have appeared only in alpha or beta
releases are considered provisional, and may be changed in incompatible ways
without requiring a preceding deprecation warning
The key desired outcome of this proposal is that the usage guidance given for
beta releases would become "suitable for production use only in environments
with sufficiently robust compatibility testing and operational monitoring
capabilities", rather than current unqualified "not for production use".
As part of this change, the current "not for production use" guidance given for
beta releases would be amended to state "suitable for production use only in
environments with sufficiently robust compatibility testing and operational
monitoring capabilities".
Similarly, the guidance given for alpha releases would be amended to state
"intended for library compatibility testing and the creation of ABI compatible
binary artifacts", rather than simply saying "not for production use".
The PEP authors believe these outcomes can be achieved by amending CPython's
pre-release management process as described in the Proposal section below.
This PEP also proposes that the frequency of X.Y.0 releases be adjusted to
begin each new release series in August every two years (starting in 2021,
around two years after the release of Python 3.8.0).
Example Future Release Schedules
================================
Under this proposal, Python 3.9.0a1 would be released in December 2019, two
months after the Python 3.8.0 baseline feature release in October 2019.
Assuming no further breaking changes were made to the full CPython ABI, the
3.9.0b2 release would then follow 2 months later in February 2020, continuing
through to 3.9.0b9 in April 2021.
Any time a breaking change to the full CPython ABI was introduced, the first
pre-release that included it would be marked as an alpha release.
3.9.0rc1 would be published in June 2021, 3.9.0rc2 in July 2021, and then
the full release published as 3.9.0 in August 2021.
The cycle would start over again in October 2021, with the publication
of 3.10.0a1 (4 months after the creation of the 3.9.x maintenance branch).
The exact schedule of maintenance releases would be up to the release team,
but assuming maintenance releases of 3.9.x were also to occur every other month
(offset from the 3.10.0 beta releases), the overall release timeline
would look like:
* 2019-12: 3.9.0a1
* 2020-02: 3.9.0b2
* ... beta (or alpha) releases every other month
* 2021-04: 3.9.0b9
* 2021-06: 3.9.0rc1 (feature freeze, ABI freeze, pyc format freeze)
* 2021-07: 3.9.0rc2
* 2021-08: 3.9.0
* 2021-09: 3.9.1, 3.8.x (final 3.8.x binary maintenance release)
* 2021-10: 3.10.0a1
* 2021-11: 3.9.2
* 2021-12: 3.10.0b2
* ... beta (or alpha) and maintenance releases continue in alternate months
* 2023-04: 3.10.0b10
* 2023-05: 3.9.11
* 2023-06: 3.10.0rc1 (feature freeze, ABI freeze, pyc format freeze)
* 2023-07: 3.10.0rc2, 3.9.12
* 2023-08: 3.10.0
* 2023-09: 3.10.1, 3.9.13 (final 3.9.x binary maintenance release)
* 2023-10: 3.11.0a1
* 2023-12: 3.11.0b2
* ... etc
If we assume two additional pre-releases were made that introduced breaking
changes to the full CPython ABI in the 3.9.0a5 and 3.9.0a7 releases, then the
overall calendar would look like:
.. figure:: pep-0605-example-release-calendar.png
:align: center
:width: 100%
Figure 1. Impact of the pre-release process changes on the calendar.
There are always two or three active maintenance branches in this model,
which preserves the status quo in that respect. The major difference is that
we would start encouraging publishers to provide pre-built binaries for the
pre-freeze rolling releases in addition to providing them for the stable
maintenance branches.
.. figure:: pep-0605-overlapping-support-matrix.png
:align: center
:width: 50%
Figure 2. Testing matrix in the 18-month cadence vs. the 24-month
Package publishers targeting the full CPython ABI that choose to provide
pre-built binaries for the rolling pre-freeze releases would at least need
to build new wheel archives following the 3.9.0a1 release. Whether they needed
to publish updated binaries after subsequent alpha releases (e.g. 3.9.0a5 or
3.9.0a7 releases in the example timeline) would depend on whether or not they
were actually affected by the ABI breaks in those later releases.
As with the status quo, all package publishers wishing to provide pre-built
binaries for the final release will need to build new wheel archives following
the ABI freeze date. Unlike the status quo, this date will be clearly marked
by the publication of the first release candidate, and it will occur early
enough to give publishers a couple of months to get ready for the final release.
Motivation
==========
@ -124,40 +191,41 @@ feature delivery latency.
Aims of this Proposal
=====================
The core of the proposal in this PEP is changing the beta phase of the CPython
pre-release process to produce a rolling stream of releases at a regular
cadence, and to ensure those builds offer a sufficient level of stability as
to be suitable for use in appropriately managed production systems.
The core of the proposal in this PEP is changing the CPython pre-release process
to produce a rolling stream of incremental feature releases at a regular
cadence, and to ensure that most of those builds offer a sufficient level of
stability as to be suitable for use in appropriately managed production systems.
By adopting this approach, the proposal aims to provide an improved outcome
for almost all Python users and contributors:
* for users of the new release stream, targeting the beta phase allows for even
lower feature delivery latency than the annual cadence proposed in PEP 602;
* for users of the new incremental feature release stream, targeting the
pre-release phase allows for even lower feature delivery latency than the
annual cadence proposed in PEP 602;
* for core developers working on new features, increased frequency and adoption
of the beta releases should improve pre-release feedback cycles;
of pre-releases should improve pre-release feedback cycles;
* for users of the established release stream, the increased adoption and
improved feedback cycles during the pre-release period should result in
increased feature maturity at the time of its first X.Y.0 release, as well
as higher levels of ecosystem readiness;
* for Python library maintainers, the rolling stream of beta releases will
* for Python library maintainers, the rolling stream of pre-releases will
hopefully provide more opportunities to identify and resolve design issues
before they make it into a full stable release than is offered by the current
pre-release management process; and
* for developers of alternative Python implementations, the proposed rolling
beta releases provide an additional incentive for extension module authors
to migrate from the full CPython ABI to the Python stable ABI, which may
over time make more of the ecosystem compatible with implementations that
don't fully emulate the CPython C API.
* for developers of alternative Python implementations, the rolling stream of
pre-releases may provide an additional incentive for extension module authors
to migrate from the full CPython ABI to the Python stable ABI, which would
also serve to make more of the ecosystem compatible with implementations that
don't emulate the full CPython C API.
That said, it is acknowledged that not all the outcomes of this proposal will be
beneficial for all members of the wider Python ecosystem:
* for Python library maintainers, both this PEP and PEP 602 would likely
result in user pressure to support the faster release cadence. While this PEP
attempts to mitigate that by marking the faster releases with the "beta" label
and removing the expectation to publish pre-built wheel archives, and PEP 602
attempts to mitigate it by keeping the minimum time between full releases at
attempts to mitigate that by clearly marking which pre-releases include
potentially breaking changes to the full CPython C ABI, and PEP 602 attempts
to mitigate it by keeping the minimum time between full releases at
12 months, it isn't possible to eliminate this downside completely;
* for third party extension module maintainers, both this PEP and PEP 602 would
likely result in user pressure to start supporting the stable ABI in order to
@ -165,16 +233,17 @@ beneficial for all members of the wider Python ecosystem:
available. Whether that's a net negative or not will depend on how the request
is presented to them (it could be a positive if the request comes in the form
of a courteous contribution to their project from a developer interested in
supporting the rolling beta releases);
supporting the rolling pre-freeze releases);
* for some users of the established release stream that rely on the
availability of pre-built wheel archives, switching to adopting a new release
every 12 months may be an acceptable rate increase, while moving consistently
to the 24 month end of the historical 18-24 month cadence would be an
undesirable rate reduction relative to the 18 month cycle used for recent
releases. Whether this proposal would be a net negative for these users in the
long run would depend on how many projects migrated over to targeting the
Python stable ABI and hence produced wheel archives that would still be usable
on the rolling beta releases.
releases. Whether this proposal would be a net negative for these users will
depend on whether or not we're able to persuade library maintainers that
it's worth their while to support the upcoming stable release throughout its
pre-freeze period, rather than waiting until its API and ABI have been
frozen.
Proposal
@ -184,11 +253,80 @@ The majority of the proposed changes in this PEP only affect the handling of
pre-release versions. The one change affecting full release versions is a
suggested change to their cadence.
Changes to beta release policy, phase duration, and cadence
-----------------------------------------------------------
This PEP proposes that the policy for beta releases be adjusted to be a
combination of the policies for alpha and beta releases:
Two year cadence for stable releases
------------------------------------
With the rolling pre-freeze releases available to users that are looking to
use leading edge versions of the reference interpreter and standard library,
this PEP proposes that the frequency of X.Y.0 releases be adjusted to publish
a new stable release in August every two years (starting in 2021,
around two years after the release of Python 3.8.0).
This change is arguably orthogonal to the proposed changes to the handling of
the pre-freeze release period, but the connection is that without those
pre-release management changes, the downsides of a two year full release cadence
would probably outweigh the upsides, whereas the opposite is true for a 12
month release cadence (i.e. with the pre-release management changes proposed
in this PEP in place, the downsides of a 12 month full release cadence would
outweigh the upsides).
Merging of the alpha and beta phases into a "pre-freeze" phase
--------------------------------------------------------------
Rather than continuing the status quo where the pre-release alpha and beta
phases are distinct and sequential, this PEP proposes that they instead be
combined into a single "pre-freeze" phase with a monotonically increasing serial
number on the releases.
Rather than denoting distinct phases, the "alpha" and "beta" names would
instead indicate whether or not the release contains breaking changes to the
full CPython C ABI:
* "alpha" releases would be "ABI breaking" releases where extension modules
built against the full CPython ABI in the preceding pre-release are not
necessarily going to load correctly
* "beta" releases would be "binary compatible" releases, where extension modules
built against the full CPython ABI in the preceding pre-release are expected
to load correctly, as long as those modules abide by the following additional
criteria:
* the module must not be using any provisional or private C APIs (either from
the previous stable release series, or the in development pre-release series)
that were removed in this beta release, or were changed in an ABI incompatible
way
* the module must not be using any C APIs that were deprecated in the previous
stable release series, and removed in this beta release
Pre-freeze phase duration and cadence
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Rather than being released monthly for a period of a few months while preparing
a new X.Y.0 release, pre-freeze releases would instead be consistently published
every two months.
The only time this would not be the case is during the two month release
candidate period for an upcoming X.Y.0 release (see the release candidate
section below for more details). This means two otherwise scheduled releases
would be skipped (one corresponding with the first release candidate date, one
with the final release date).
The pre-freeze phase would typically be expected to start 2 months after the
preceding stable X.Y.0 release.
The first pre-freeze release for any new release series will always be X.Y.0a1
(as there is no preceding release with the same ABI version markers to judge
binary compatibility against).
Pre-freeze releases would gain an additional flag in their C ABI compatibility
markers to avoid binary compatibility issues with the eventual stable release.
Release policy for beta releases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This PEP proposes that the policy for beta releases be set as follows:
* as with current beta releases, the stable BuildBot fleet is expected to be
green prior to preparation and publication of the beta release
@ -199,28 +337,50 @@ combination of the policies for alpha and beta releases:
be expected to become a permanent part of that ABI unless and until that
stable ABI version is retired completely (Note: there are no current plans
to increment the stable ABI version)
* as with current alpha releases, beta releases would *not* be considered
feature complete for the next X.Y.0 release
* as with current alpha releases, all APIs added since the last CPython feature
* unlike current beta releases, beta releases under this PEP would *not* be
considered feature complete for the next X.Y.0 release
* unlike current beta releases, all APIs added since the last CPython feature
release (other than additions to the stable C ABI) would be considered
provisional
* as with current alpha releases, beta releases would be prepared and published
from the master development branch
Rather than being released monthly for a period of a few months while preparing
a new X.Y.0 release, beta releases would instead be consistently published every
two months.
The only time this would not be the case is during the two month release
candidate period for an upcoming X.Y.0 release, as described in the following
section.
* unlike current beta releases, beta releases under this PEP would be prepared
and published from the master development branch
* unlike current alpha or beta releases, beta releases under this PEP would be
required to be fully ABI compatible with the immediately preceding pre-release
in the series (excluding any changes to provisional APIs, or the removal of
APIs that were deprecated in the previous release series)
Changes to release candidate policy, phase duration, and cadence
----------------------------------------------------------------
Release policy for alpha releases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Given the proposed changes to the beta release phase, the release candidate
phase would see the following related adjustments:
This PEP proposes that the policy for alpha releases be set as follows:
* as with current alpha releases, the stable BuildBot fleet is expected to be
green prior to preparation and publication of the alpha release
* as with current alpha releases, the release manager is expected to review
open release blocker issues prior to preparation and publication of the beta
release
* unlike current alpha release, the release manager would be expected to
target a similar level of stability to the current beta releases, even
for the alpha releases
Under this PEP, an alpha release would be published whenever it isn't possible
to publish a release that satisfies the criteria for a beta release, and
allowing some additional time before making the release won't resolve the issue.
It is expected that the full CPython API changing in a way that breaks ABI
compatibility (for example, a field may have been added to or removed from a
public struct definition) will be the most likely reason for publishing
additional alpha releases beyond the initial compatibility tag defining
X.Y.0a1 release, but the decision for any particular release rests with the
release manager.
Release candidate policy, phase duration, and cadence
-----------------------------------------------------
Given the proposed changes to the alpha and beta release phases, the release
candidate phase would see the following related adjustments:
* Feature freeze, ABI freeze, pyc file format freeze, and maintenance branch
creation would all correspond with the creation of X.Y.0rc1 (currently these
@ -241,35 +401,6 @@ of Python projects to build and publish pre-built wheel archives for the new
stable release series, significantly improving the initial user experience of
the X.Y.0 release.
Removal of routine alpha releases
---------------------------------
Due to the proposed policy changes for beta releases and release candidates,
the traditional CPython alpha release period would no longer serve a clear
purpose, and as such would be skipped, with the first pre-release in a new
series being X.Y.0b1.
However, when it seems appropriate to do so, release managers may choose to
declare that a scheduled beta release will be marked as an alpha release
instead. For example, a release manager may choose to do this when major changes
have been made to the release management tooling, when a particularly
large feature has landed that the release manager views as having a high chance
of introducing unintended side effects, or when a new interface has been added
to the stable C ABI that is only useful when it appears as part of that ABI
(e.g. it may be enabling a new capability in the stable ABI that the full ABI
already supports in a different way), and requires further testing and design
feedback before being locked down in a beta release.
In such cases, the beta serial number would be retained, but the release would
be marked as an alpha release instead, and hence the "not for production use"
caveat would apply.
(The serial number would still be reset to 1 when entering the release
candidate phase - only alpha and beta release would share a serial number
sequence)
Changes to management of the CPython stable C ABI
-------------------------------------------------
@ -278,11 +409,11 @@ built against any particular CPython release will continue to work on future
CPython releases that support the same stable ABI version (this version is
currently ``abi3``).
Under the proposed rolling beta release model, this commitment would be extended
to also apply to the beta releases: once an intentional addition to the ``abi3``
stable ABI for the upcoming Python version has been shipped in a beta release,
then it will not be removed from future releases for as long as the ``abi3``
stable ABI remains supported.
Under the proposed rolling pre-freeze release model, this commitment would be
extended to also apply to the beta releases: once an intentional addition to the
``abi3`` stable ABI for the upcoming Python version has been shipped in a beta
release, then it will not be removed from future releases for as long as the
``abi3`` stable ABI remains supported.
Two main mechanisms will be available for obtaining community feedback on
additions to the stable ABI:
@ -293,8 +424,8 @@ additions to the stable ABI:
* for APIs where that approach is unavailable for some reason (e.g. some API
additions may serve no useful purpose when the full CPython API is available),
then developers may request that the release manager mark the next release
as an alpha release, and attempt to obtain further feedback that way
as an alpha release (even in the absence of an ABI break in the full CPython
API), and attempt to obtain further feedback that way
As a slight readability and usability improvement, this PEP also proposes the
introduction of aliases for each major stable ABI version::
@ -320,19 +451,44 @@ be made available::
// A Python 3.9+ addition to the stable ABI would appear here
#endif
The documentation for the rolling beta releases and the stable C ABI would make
it clear that extension modules built against the stable ABI in a later beta
release may not load correctly on earlier alpha or beta releases.
The documentation for the rolling pre-freeze releases and the stable C ABI would
make it clear that extension modules built against the stable ABI in a later
pre-freeze release may not load correctly on an earlier pre-freeze release.
The documentation for alpha releases and the stable C ABI would make it clear
that even extension modules built against the stable ABI in an alpha release
release may not load correctly on any other release.
release may not load correctly on the next release if two alpha releases are
published in a row (this situation would ideally be rare).
Changes to management of the CPython version-specific ABI
---------------------------------------------------------
Changes to management of the full CPython ABI
---------------------------------------------
The CPython version-specific ABI has long operated under a policy where binary
This PEP proposes two changes to the management of the full CPython ABI.
An explicit NEWS file convention to mark ABI breaking changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The proposal in this PEP requires that release managers be able to appropriately
mark a pre-freeze release as either an alpha or a beta release based on whether
or not it includes an ABI breaking change.
To assist in that process, core developers would be requested to include a
"(CPython ABI break)" marker at the beginning of all NEWS file snippets for
changes that introduce a breaking change in the full CPython C ABI.
The "CPython" marker is included to make it clear that these annotations relate
to the full CPython ABI, not the stable ABI.
In addition to being useful for release managers, these markers should also be
useful for developers investigating unexpected segfaults when testing against
the affected release.
Explicitly marking builds against the pre-freeze ABI
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The full CPython ABI has long operated under a policy where binary
compatibility only applies within a release series after the ABI has been
declared frozen, and only source compatibility applies between different
release series.
@ -373,51 +529,31 @@ A proposed reference implementation for this change is available at [4_] (Note:
at time of writing, that implementation had not yet been tested on Windows).
Example Future Release Schedules
================================
Updating Python-Requires for projects affected by full C ABI changes
--------------------------------------------------------------------
Under this proposal, Python 3.9.0b1 would be released in December 2019, two
months after the Python 3.8.0 baseline feature release in October 2019.
When a project first opts in to providing pre-built binary wheels for the
rolling pre-freeze release series, they don't need to do anything special: they
would add the rolling release series to their build and test matrices and
publish binary archives that are flagged as being compatible with that release
series, just as they would if providing pre-built binary wheels after the
full CPython ABI freeze for that release series.
The 3.9.0b2 release would then follow 2 months later in February 2020,
continuing through to 3.9.0b9 in April 2021.
However, if the project is affected by a CPython ABI compatibility break in the
rolling release stream, then they will need to issue a version update that
includes both the new binary build, and a new environment constrained
``Python-Requires`` marker.
3.9.0rc1 would be published in June 2021, 3.9.0rc2 in July 2021, and then
the full release published as 3.9.0 in August 2021.
For example, if a project supporting the rolling release stream was affected by
a CPython ABI compatibility break in the 3.9.0a5 release, then they would add
the following metadata entry on the version that published the updated binary
build::
The cycle would start over again in October 2021, with the publication
of 3.10.0b1.
Python-Requires: >= "3.9.0a5"; python_version == "3.9"
Assuming maintenance releases of 3.9.x were also to occur every other month
(offset from the 3.10.0 beta releases), the overall release timeline
would look like:
* 2019-12: 3.9.0b1
* ... beta releases every other month
* 2021-04: 3.9.0b9
* 2021-06: 3.9.0rc1 (feature freeze, ABI freeze, pyc format freeze)
* 2021-07: 3.9.0rc2
* 2021-08: 3.9.0
* 2021-09: 3.9.1, 3.8.x (final 3.8.x binary maintenance release)
* 2021-10: 3.10.0b1
* 2021-11: 3.9.2
* 2021-12: 3.10.0b2
* ... beta and maintenance releases every other month
* 2023-04: 3.10.0b10
* 2023-05: 3.9.11
* 2023-06: 3.10.0rc1 (feature freeze, ABI freeze, pyc format freeze)
* 2023-07: 3.10.0rc2, 3.9.12
* 2023-08: 3.10.0
* 2023-09: 3.10.1, 3.9.13 (final 3.9.x binary maintenance release)
* 2023-10: 3.11.0b1
* ... etc
(The exact schedule of maintenance releases would be up to the release team -
alternating months with the rolling beta releases is the proposed target)
There are always two or three active maintenance branches in this model,
which preserves the status quo in that respect.
What this does is add an additional compatibility constraint as part of the
published packages, so Python 3.9.x versions older than 3.9.0a5 won't consider
the updated package as a candidate for installation.
Caveats and Limitations
@ -435,14 +571,14 @@ only proposes an expected cadence for pre-releases and X.Y.0 releases.
However, for the sake of the example timelines, the PEP assumes maintenance
releases every other month, allowing them to alternate months with the rolling
beta releases.
pre-freeze releases.
Design Discussion
=================
Why rolling beta releases over simply doing more frequent X.Y.0 releases?
-------------------------------------------------------------------------
Why rolling pre-freeze releases over simply doing more frequent X.Y.0 releases?
-------------------------------------------------------------------------------
For large parts of Python's user base, *availability* of new CPython feature
releases isn't the limiting factor on their adoption of those new releases
@ -460,48 +596,48 @@ interests of operating environments that are able to consume new releases as
fast as the CPython core team is prepared to produce them.
Why rolling beta releases rather than rolling alpha releases?
----------------------------------------------------------------
Is it necessary to keep the "alpha" and "beta" naming scheme?
-------------------------------------------------------------
The code quality standards upheld by the CPython code review process and
BuildBot fleet make the "beta" label more suitable than the "alpha" label.
Using the "a" and "b" initials for the proposed rolling releases is a design
constraint imposed by some of the pragmatic aspects of the way CPython version
numbers are published.
The "production ready for some environments, but subject to change with limited
notice" caveat also aligns well with at least some uses of the "beta" term,
whereas "alpha" almost universally indicates "not yet ready for production".
Specifically, alpha releases, beta releases, and release candidates are reported
in some places using the strings "a", "b", and "c" respectively, while in others
they're reported using the hex digits ``0xA``, ``0xB``, and ``0xC``. We want to
preserve that, while also ensuring that any ``Python-Requires`` constraints
are expressed against the beta releases rather than the alpha releases (since
the latter may not enforce the ``abi3`` stability requirements if two alpha
releases occur in succession).
Why rolling beta releases rather than something like "brisk release cadence"?
-----------------------------------------------------------------------------
Using the "b" initial for the proposed rolling releases is a design constraint
imposed by some of the pragmatic aspects of the way CPython version numbers are
published (specifically, alpha releases, beta releases, and release candidates
are reported in some places using the strings "a", "b", and "c" respectively,
while in others they're reported using the hex digits ``0xA``, ``0xB``, and
``0xC``, and we want to preserve that, while also ensuring that the rolling
releases are all sorted after any alpha releases, and before the first release
candidate).
However, there isn't anything forcing us to say that the "b" stands for "beta".
However, there isn't anything forcing us to say that the "a" stands for "alpha"
or the "b" stands for "beta".
That means that if we wanted to increase adoption amongst folks that were
only being put off by the "beta" label, then it could make sense to rebrand the
rolling beta releases as "brisk" releases, emphasising the rate of release over
the fact that the release includes APIs and ABIs that aren't necessarily stable
yet.
only being put off by the "beta" label, then it may make sense to emphasise
the "\*A\*BI breaking" and "\*B\*inary compatible" names over the "alpha"
and "beta" names, giving:
In the near term, limiting adoption to folks that are comfortable with the
"beta" label would be a good thing, as initial adopters are likely to
encounter unexpected consequences at the level of the wider Python ecosystem,
and would need to take an active part in getting those issues resolved.
* 3.9.0a1: ABI breaking pre-freeze release
* 3.9.0b2: binary compatible pre-freeze release
* 3.9.0rc1: release candidate
* 3.9.0: final release
This iteration of the PEP doesn't go that far, as limiting initial adoption
of the rolling pre-freeze releases to folks that are comfortable with the
"beta" label is likely to be a good thing, as it is the early adopters of these
releases that are going to encounter any unexpected consequences that occur
at the level of the wider Python ecosystem, and we're going to need them to
be willing to take an active part in getting those issues resolved.
Moving away from the "beta" naming would then become an option to keep in mind
for the future (with "brisk releases" as a potential alternative name).
for the future, assuming the resulting experience is sufficiently positive that
we decide the approach is worth continuing.
Why rolling beta releases rather than alternating between stable and unstable release series?
---------------------------------------------------------------------------------------------
Why rolling pre-freeze releases rather than alternating between stable and unstable release series?
---------------------------------------------------------------------------------------------------
Rather than using the beta period for rolling releases, another option would be
to alternate between traditional stable releases (for 3.8.x, 3.10.x, etc), and
@ -518,7 +654,7 @@ be assigned to the *value* of any of the numbers in a release version. These
community members instead prefer that all the semantic significance be
associated with the *position* within the release number that is changing.
By contrast, the rolling beta release proposal aims to address that concern by
By contrast, the rolling pre-freeze release proposal aims to address that concern by
ensuring that the proposed changes in policy all revolve around whether a
particular release is an alpha release, beta release, release candidate, or
final release.
@ -548,7 +684,7 @@ all the new features released in Python 3.9.0?". With calendar versioning on
the rolling releases, that's impossible to answer without consulting a release
calendar to see when 3.9.0rc1 was branched off from the rolling release series.
By contrast, the equivalent question for rolling beta releases is
By contrast, the equivalent question for rolling pre-freeze releases is
straightforward to answer: "Does Python 3.10.0b2 include all the new features
released in Python 3.9.0?". Just from formulating the question, the answer is
clearly "Yes, unless they were provisional features that got removed".
@ -559,8 +695,8 @@ versioning concept, such as how ``sys.version_info``, ``PY_VERSION_HEX``,
module naming would work.
How would users of the rolling beta releases detect API changes?
----------------------------------------------------------------
How would users of the rolling pre-freeze releases detect API changes?
----------------------------------------------------------------------
When adding new features, core developers would be strongly encouraged to
support feature detection and graceful fallback to alternative approaches via
@ -575,13 +711,13 @@ and the various ``os.supports_*`` sets that the ``os`` module already offers for
platform dependent capability detection.
It would also be possible to add features that need to be explicitly enabled
via a ``__future__`` import when first included in the rolling beta releases,
via a ``__future__`` import when first included in the rolling pre-freeze releases,
even if that feature flag was subsequently enabled by default before its first
appearance in an X.Y.0 release candidate.
The rationale behind these approaches is that explicit detection/enabling like
this would make it straightforward for users of the rolling beta release stream
to notice when we remove or change provisional features
this would make it straightforward for users of the rolling pre-freeze release
stream to notice when we remove or change provisional features
(e.g. ``from __future__`` imports break on compile if the feature flag no
longer exists), or to safely fall back on previous functionality.
@ -591,6 +727,72 @@ have any practical way to add for checks against the value of
``sys.version_info``.
Why add a new pre-freeze ABI flag to force rebuilds after X.Y.0rc1?
-------------------------------------------------------------------
The core development team currently actively *discourage* the creation of
public pre-built binaries for an X.Y series prior to the ABI freeze date.
The reason we do that is to avoid the risk of painful debugging sessions
on the stable X.Y.0 release that get traced back to "Oh, our dependency
'superfast-binary-operation' was affected by a CPython ABI break in
X.Y.0a3, but the project hasn't published a new build since then".
With the proposed pre-freeze ABI flag in place, this aspect of the
release adoption process continues on essentially unchanged from the
status quo: a new CPython X.Y release series hits ABI freeze -> package
maintainers publish new binary extension modules for that release
series -> end users only get segfaults due to actual bugs, not just
builds against an incompatible ABI.
The primary goal of the new pre-freeze ABI flag is then to improve
the user experience of the rolling pre-freeze releases themselves, by
allowing pre-built binary archives to be published for those releases
without risking the problems that currently cause us to actively
discourage the publication of binary artifacts prior to ABI freeze.
In the ideal case, package maintainers will only need to publish
one pre-freeze binary build at X.Y.0a1, and then a post-freeze
build after X.Y.0rc1. The only situations that should *require*
a rebuild in the meantime are those where the project was
actually affected by a CPython ABI break in an intervening alpha
release.
As a concrete example, consider the scenario where we end up having three
releases that include ABI breaks: X.Y.0a1, X.Y.0a5, X.Y.0a7. The X.Y.0a7 ABI is
then the ABI that carries through all the subsequent beta releases and into
X.Y.0rc1. (This is the scenario illustrated in figure 1)
Forcing everyone to rebuild the world every time there’s an alpha release in
the rolling release stream would almost certainly lead to publishers deciding
supporting the rolling releases was more trouble than it was worth, so we want
to allow modules built against X.Y.0a1 to be loaded against X.Y.0a7, as they’re
*probably* going to be compatible (there are very few projects that use every
C API that CPython publishes, and most ABI breaks affect a single specific API).
Once we publish X.Y.0rc1 though, we want to ensure that any binaries that were
built against X.Y.0a1 and X.Y.0a4 are completely removed from the end user
experience. It would be nice to be able to keep the builds against X.Y.0a7 and
any subsequent beta releases (since it turned out those actually were built
against the post-freeze ABI, even if we didn’t know that at the time), but
losing them isn’t any *worse* than the status quo.
This means that the pre-freeze flag is “the simplest thing that could possibly
work” to solve this problem - it’s just a new ABI flag, and we already have
the tools available to deal with ABI flags (both in the interpreter and in
package publication and installation tools).
Since the ABI flags have changed relative to the pre-releases, projects don't
even need to publish a new release: they can upload new wheel archives to their
existing releases, just as they can today.
A cleverer scheme that was able to retroactively accept everything built
against the last alpha or subsequent beta releases would likely be possible,
but it isn't considered *necessary* for adoption of this PEP, as even if we
initially start out with a simple pre-release ABI flag, it would still be
possible to devise a more sophisticated approach in the future.
Implications for CPython core development
-----------------------------------------
@ -599,15 +801,20 @@ branch more consistently release ready.
While the main requirement for that would be to keep the stable BuildBot fleet
green, there would also be encouragement to keep the development version of
the documentation up to date for the benefit of users of the rolling beta
the documentation up to date for the benefit of users of the rolling pre-freeze
releases. This will include providing draft What's New entries for changes as
they are implemented, although the initial versions may be relatively sparse,
and then expanded based on feedback from beta release users.
For core developers working on the CPython C API, there would also be a new
requirement to consistently mark ABI breaking changes in their NEWS file
snippets.
On the specific topic of the stable ABI, most API designs will be able to go
through a process where they're first introduced as part of the full CPython
API (allowing changes between beta releases), and only promoted to the stable
ABI once developers are confident that the interface is genuinely stable.
through a process where they're first introduced as a provisional part of the
full CPython API (allowing changes between pre-freeze releases), and only
promoted to the stable ABI once developers are confident that the interface
is genuinely stable.
It's only in rare cases where an API serves no useful purpose outside the
stable ABI that it may make sense to publish an alpha release containing a
@ -618,20 +825,31 @@ provisional CPython API instead.
Implications for Python library development
-------------------------------------------
By using the "beta" labelling rather than the incremental feature release
numbering proposed in PEP 598, the hope would be that consumers of these
new rolling releases would realise that they're likely going to need to build
their own wheel archives from source, and will generally be more prone to
encountering library compatibility issues when updating to a new release.
If this PEP is successful in its aims, then supporting the rolling pre-freeze
release stream shouldn't be subtantially more painful for library authors than
supporting the stable releases.
Library authors who actually want to support the beta stream would have the
option of testing against the latest beta release in their pre-merge test
matrices (just as they test against the latest maintenance release of previously
published versions), with the CPython nightly builds offered by some CI
providers used solely in an advisory capacity for early detection of potential
compatibility problems.
For publishers of pure Python packages, this would be a matter of publishing
"py3" tagged wheel archives, and potentially adding the rolling pre-freeze
release stream to their test matrix if that option is available to them.
Having a rolling beta release stream available may also make it more feasible
For publishers of binary extension modules, the preferred option would be to
target the stable C ABI (if feasible), and thus enjoy an experience similar to
that of pure Python packages, where a single pre-built wheel archive is able to
cover multiple versions of Python, including the rolling pre-freeze release
stream.
This option isn't going to be viable for all libraries, and the desired outcome
for those authors is that they be able to support the rolling releases by
building and publishing one additional wheel archive, built against the initial
X.Y.0a1 release. The subsequent build against X.Y.0rc1 or later is then the same
build that would have been needed if only supporting the final stable release.
Additional wheel builds beyond those two should then only be needed if that
particular library is directly affected by an ABI break in any other alpha
release that occurs between those two points.
Having a rolling pre-freeze release stream available may also make it more feasible
for more CI providers to offer a "CPython beta release" testing option. At the
moment, this feature is only available from CI providers that are willing and
able to put the necessary time and effort into creating, testing, and publishing
@ -665,7 +883,7 @@ supporting at least the most recent feature release, and then dropping support
for all X.Y.Z releases around 18 months after X.(Y+1).0 is released.
This means there is a 6 month period every other year where only one feature
release is supported. Under the proposal in this PEP, that period would
correspond to the final few rolling beta releases and the release candidate
correspond to the final few rolling pre-freeze releases and the release candidate
phase for the upcoming stable feature release.
@ -681,27 +899,36 @@ work on bug fixes and smaller features before the first release candidate goes
out, while the Language Summit and core developer discussions can focus on
plans for the next release series.
The post-release core development sprint in release years will provide an
opportunity to incorporate feedback received on the release, either as part of
the next maintenance release (for bug fixes and feedback on provisional APIs),
or as part of the next release series. These sprints would also likely
correspond with the Steering Council elections for the next release cycle.
The pre-alpha core development sprint in release years will provide an
opportunity to incorporate feedback received on the previous release, either
as part of the next maintenance release (for bug fixes and feedback on
provisional APIs), or as part of the first alpha release of the next release
series (for feedback received on stable APIs).
Those initial alpha releases would also be the preferred target for ABI breaking
changes to the full CPython ABI (while changes later in the release cycle
would still be permitted as described in this PEP, landing them in the X.Y.0a1
release means that they won't trigger any additional work for publishers of
pre-built binary packages).
The Steering Council elections for the next release cycle are also likely to
occur around the same time as the pre-alpha development sprints.
In non-release years, the focus for both events would just be on the upcoming
maintenance and beta releases. These less intense years would hopefully provide
an opportunity to tackle various process changes and infrastructure upgrades
without impacting the release candidate preparation process.
maintenance and pre-freeze releases. These less intense years would hopefully
provide an opportunity to tackle various process changes and infrastructure
upgrades without impacting the release candidate preparation process.
Release cycle alignment for prominent Linux distributions
---------------------------------------------------------
Some rolling release Linux distributions (e.g. Arch, Gentoo) may be in a
position to consume the new rolling beta releases proposed in this PEP, but it
is expected that most distributions would continue to use the established
position to consume the new rolling pre-freeze releases proposed in this PEP,
but it is expected that most distributions would continue to use the established
releases.
The specific dates for those releases proposed in this PEP are chosen to align
The specific dates for final releases proposed in this PEP are chosen to align
with the feature freeze schedules for the annual October releases of the Ubuntu
and Fedora Linux distributions.
@ -760,24 +987,26 @@ pre-existing Python deployment on the target system.
For these use cases, there is a straightforward mechanism to minimise the
impact of this PEP: continue using the stable releases, and ignore the rolling
beta releases.
pre-freeze releases.
To actually adopt the rolling beta releases in these environments, the main
challenge will be handling the potential for extension module segfaults if the
CPython ABI changes in an incompatible way between beta releases.
To actually adopt the rolling pre-freeze releases in these environments, the
main challenge will be handling the potential for extension module segfaults
when the next pre-freeze release is an alpha release rather than a beta
release, indicating that the CPython ABI may have changed in an incompatible
way.
If all extension modules in use target the stable ABI, then there's no problem,
and everything will work just as smoothly as it does on the stable releases.
Alternatively, "rebuild and recache all extension modules" could become a
standard activity undertaken as part of updating to each new beta release.
standard activity undertaken as part of updating to an alpha release.
Finally, it would also be reasonable to just not worry about it until something
actually breaks, and then handle it like any other library compatibility issue
found in a new beta release.
found in a new alpha or beta release.
Aside from extension module ABI compatibilty, the other main point of additional
complexity when using the rolling beta releases would be "roll-back"
complexity when using the rolling pre-freeze releases would be "roll-back"
compatibility for independently versioned features, such as pickle and SQLite,
where use of new or provisional features in the beta stream may create files
that are not readable by the stable release. Applications that use these

BIN
pep-0605/example-release-calendar.odp Normal file
View File

Binary file not shown.

BIN
pep-0605/overlapping-support-matrix.odp Normal file
View File

Binary file not shown.