python-peps/pep-0607.rst

PEP: 607
Title: Reducing CPython's Feature Delivery Latency
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa <lukasz@python.org>,
        Steve Dower <steve.dower@python.org>,
        Nick Coghlan <ncoghlan@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-607-shared-background-for-the-release-cadence-peps/2528
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 11-Oct-2019
Python-Version: 3.9
Post-History: 20-Oct-2019

Abstract
========

PEP 602 and PEP 605 describe two alternative approaches to delivering smaller
collections of features to Python's users more frequently (as compared to the
current approach of offering new feature releases every 18-24 months, with
the first binary alpha release taking place 6-8 months before the final release).

Both PEPs also propose moving to a release cadence that results in full releases
occuring at a consistent time of year (every year for PEP 602, every other
year for PEP 605).

This PEP (from the authors of both competing proposals) provides common
background on *why* a change in the release cadence is considered desirable,
as well as the perceived risks that both PEPs attempt to mitigate.


Rationale for change
====================

Reducing the size of feature delivery batches
---------------------------------------------

When multiple large changes are delivered together, a complex investigation
may be required to determine the root cause of any new issues that arise.
Large batch sizes also make it more likely that problems *will* be encountered,
given that they include larger pieces of relatively untested code.

The easiest way to simplify those investigations and reduce the likelihood of
users encountering problems is to reduce the size of the batches being shipped.

PEP 602 proposes to address this problem via the straightforward approach of 
reducing CPython's typical batch size by 50%, shipping 12 months of changes
each time, rather than accumulating 18+ months of changes.

PEP 605 proposes to address it by regularly delivering 2 months worth of changes
to a subset of Python's user base that opts in to running a rolling stream of
beta releases (similar to running Windows Insider builds instead of the Windows
retail release, or running Debian testing instead of Debian stable).


Reducing the latency of feature delivery
----------------------------------------

When only stable releases are seeing significant user adoption, and there's a
long period of time between stable releases, it creates an incredibly strong
temptation for developers to push changes into stable releases before they're
really ready for general use.

PEP 602 proposes to address this problem by reducing the period of time
between stable releases to 12 months rather than 18 months.

PEP 605 proposes to address it by actively creating a community of
Python users that regularly install and use CPython beta releases, providing an
incentive for core developers to start shipping changes earlier in the
pre-release cycle, in order to obtain feedback before the feature gets locked
down in a stable release.


Aligning the release cadence with the calendar year
---------------------------------------------------

While the current release cadence is nominally 18-24 months, in practice it has
consistently been towards the 18 month end of that range. This means that the
target dates for pre-releases and final releases move around from release to
release, and the only way to remember them is to either look at the release PEP,
or else to add those dates to your calendar. This is annoying for both
individual volunteers and for corporate contributors, and also complicates
alignment with events like PyCon US (typically April/May) and the now-annual
core development sprints (typically in September).

PEP 602 proposes to address this problem by publishing a new release in October
every year, and basing the pre-release calendar for each year off that.

PEP 605 proposes to address this problem by alternating between release years
(where a new stable release is published in August), and non-release years
(where only maintenance releases and new rolling beta releases are published).


Improving the pre-release design feedback cycle
-----------------------------------------------

One of the challenges of designing changes to the core interpreter and standard
library APIs is that the user base in a position to provide feedback on
nightly builds and the current pre-releases is relatively limited. This means
that much user feedback isn't received until after an API design has already
shipped in a full X.Y.0 release.

If the API is a regular API, then deprecation cycles mean that it may take
literally years to correct any design mistakes identified at that point.
Marking APIs as provisional nominally offers a way to avoid that constraint,
but actually taking advantage of that freedom causes other problems.

PEP 602 proposes to address this problem by starting the alpha period
immediately after the previous stable release.

PEP 605 proposes to address this problem by actively promoting adoption of
CPython pre-releases for running production workloads (not just for library and
application compatibility testing), and adjusting the pre-release management
process as necessary to make that a reasonable thing to do.

(Note: some standard library APIs are amenable to initially being shipped as
part of separately versioned packages via PyPI, and only later incorporated
into the standard library. This section is more about the lower level APIs
and non-library features where that approach to obtaining early design
feedback doesn't apply)


Risks to be mitigated
=====================

While the status quo could stand to be improved in some respects, Python's
popularity indicates that a lot of users and other participants in the wider
Python ecosystem are happy enough with the current release management process.

Python's user base is too large and
`too varied <https://www.curiousefficiency.org/posts/2017/10/considering-pythons-target-audience.html>`__
to cover all the potential downsides of changing our release cadence here, so
instead this section just covers some of the points that have been specifically
taken into account in the design of the PEPs.


Impact on users and redistributors that already skip some releases
------------------------------------------------------------------

It is already the case that not all users and redistributors update to every
published CPython release series (for example, Debian stable and Ubuntu LTS
sometimes skip releases due to the mismatch between their 24 month release
cycles and CPython's typically 18-month cycle).

The faster 12-month full release cadence in PEP 602 means that users in this
category may end up skipping two releases where they would previously have only
skipped one. However, the extended notice period for deprecations means that
skipping a single release should no longer result in missed deprecation warnings.

The slower 24-month full release cadence in PEP 605 may move some of the users
that have historically been in this category into the "update to every stable
release" category.


Impact on users and redistributors that update to every release
---------------------------------------------------------------

Many of Python's users never install a pre-release, but do update to every
stable release series at some point after it is published.

PEP 602 aims to mitigate the potential negative impact on members of this group
by keeping the minimum gap between releases to 12 months, and retaining the
18 month full support period for each release.

Keeping the 18-month full support period for each release branch means that the
branches will spend roughly the same amount of time in full support and
security-fix-only mode as they do now (~18 months and ~42 months, respectively).

PEP 605 aims to mitigate the potential negative impact on members of this group
by increasing use during the pre-release period to achieve more stable final
releases with wider ecosystem support at launch.

With a 24-month release cadence each release branch will spend proportionally
more time in full support mode and less time in security-fix-only mode
(~24 months and ~36 months, respectively).

Full discussion of the impact on this group is left to the individual PEPs.


Impact on users and redistributors of CPython nightly builds
------------------------------------------------------------

Despite the difficulties of doing so, there are already some users and
redistributors that take on the challenge of using or publishing the CPython
master branch directly.

Neither PEP 602 nor PEP 605 should directly affect this group, but the rolling
release stream proposal in PEP 605 aims to lower the barriers to more users
adopting this style of usage, by allowing them to adopt the tested rolling
beta stream, rather than needing to use the master branch directly.


Impact on maintainers of third party libraries
----------------------------------------------

For maintainers of third party libraries, the key source of support complexity
is the *number* of different Python versions in widespread use.

PEP 602 aims to mitigate the potential negative impact on members of this group
by keeping the minimum gap between full releases to 12 months.

PEP 605 aims to mitigate the potential negative impact on members of this group
by increasing the gap between full releases to 24 months, retaining the current
policy of moving each release branch to security-fix-only mode not long after
its successor is released, and retaining the "beta" naming scheme for the new
rolling release stream (at least for the Python 3.9 release cycle).

Full discussion of the impact on this group is left to the individual PEPs.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


..
  Local Variables:
  mode: indented-text
  indent-tabs-mode: nil
  sentence-end-double-space: t
  fill-column: 72
  coding: utf-8
  End: