python-peps/peps/pep-0101.rst

937 lines
37 KiB
ReStructuredText
Raw Normal View History

PEP: 101
Title: Doing Python Releases 101
2017-09-05 16:17:42 -04:00
Author: Barry Warsaw <barry@python.org>, Guido van Rossum <guido@python.org>
Status: Active
Type: Informational
Created: 22-Aug-2001
Post-History:
Replaces: 102
.. highlight:: shell
Abstract
2017-09-04 17:38:47 -04:00
========
2017-09-04 17:38:47 -04:00
Making a Python release is a thrilling and crazy process. You've heard
the expression "herding cats"? Imagine trying to also saddle those
purring little creatures up, and ride them into town, with some of their
buddies firmly attached to your bare back, anchored by newly sharpened
claws. At least they're cute, you remind yourself.
2023-10-09 07:01:36 -04:00
Actually, no, that's a slight exaggeration 😉 The Python release
2017-09-04 17:38:47 -04:00
process has steadily improved over the years and now, with the help of our
amazing community, is really not too difficult. This PEP attempts to
2023-10-09 07:01:36 -04:00
collect, in one place, all the steps needed to make a Python release.
Most of the steps are now automated or guided by automation, so manually
following this list is no longer necessary.
Things You'll Need
2017-09-04 17:38:47 -04:00
==================
2017-09-04 17:38:47 -04:00
As a release manager there are a lot of resources you'll need to access.
Here's a hopefully-complete list.
2017-09-04 17:38:47 -04:00
* A GPG key.
2017-09-04 17:38:47 -04:00
Python releases are digitally signed with GPG; you'll need a key,
which hopefully will be on the "web of trust" with at least one of
the other release managers.
2017-09-04 17:38:47 -04:00
* A bunch of software:
* A checkout of the `python/release-tools`_ repo.
It contains a `requirements.txt
<https://github.com/python/release-tools/blob/master/requirements.txt>`_
file that you need to install
2023-10-09 07:01:36 -04:00
dependencies from first. Afterwards, you can fire up scripts in the
repo, covered later in this PEP.
* `blurb <https://github.com/python/blurb>`__, the
2023-10-09 07:01:36 -04:00
`Misc/NEWS <https://github.com/python/cpython/tree/main/Misc/NEWS.d>`_
management tool. You can pip install it.
2017-09-04 17:38:47 -04:00
* A fairly complete installation of a recent TeX distribution,
such as texlive. You need that for building the PDF docs.
2023-10-09 07:01:36 -04:00
* Access to servers where you will upload files:
* ``downloads.nyc1.psf.io``, the server that hosts download files; and
* ``docs.nyc1.psf.io``, the server that hosts the documentation.
* Administrator access to `python/cpython`_.
* An administrator account on `www.python.org`_, including an "API key".
* Write access to the `python/peps`_ repository.
2017-09-04 17:38:47 -04:00
If you're reading this, you probably already have this--the first
task of any release manager is to draft the release schedule. But
in case you just signed up... sucker! I mean, uh, congratulations!
* Posting access to `blog.python.org`_, a Blogger-hosted weblog.
2017-09-04 17:38:47 -04:00
The RSS feed from this blog is used for the 'Python News' section
on `www.python.org`_.
2014-08-01 16:39:37 -04:00
2017-09-04 17:38:47 -04:00
* A subscription to the super secret release manager mailing list, which may
or may not be called ``python-cabal``. Bug Barry about this.
2015-06-05 21:55:54 -04:00
* A ``@python.org`` email address that you will use to sign your releases
with. Ask ``postmaster@`` for an address; you can either get a full
account, or a redirecting alias + SMTP credentials to send email from
this address that looks legit to major email providers.
Types of Releases
=================
There are several types of releases you will need to make. These include:
* ``alpha``
2023-10-09 07:01:36 -04:00
* ``begin beta``, also known as ``beta 1``, also known as ``new branch``
* ``beta 2+``
* ``release candidate 1``
* ``release candidate 2+``
* ``final``
* ``new branch``
* ``begin bugfix mode``
* ``begin security-only mode``
* ``end-of-life``
Some of these release types actually involve more than
one release branch. In particular, a **new branch** is that point in the
release cycle when a new feature release cycle begins. Under the current
organization of the CPython Git repository, the *main* branch is always
the target for new features. At some point in the release cycle of the
next feature release, a **new branch** release is made which creates a
new separate branch for stabilization and later maintenance of the
2023-10-09 07:01:36 -04:00
current in-progress feature release (``3.n.0``) and the *main* branch is modified
to build a new version (which will eventually be released as ``3.n+1.0``).
While the **new branch** release step could occur at one of several points
in the release cycle, current practice is for it to occur at feature code
cutoff for the release which is scheduled for the first beta release.
In the descriptions that follow, steps specific to release types are
labeled accordingly, for now, **new branch** and **final**.
2023-10-09 07:01:36 -04:00
How To Make A Release
2017-09-04 17:38:47 -04:00
=====================
2017-09-04 17:38:47 -04:00
Here are the steps taken to make a Python release. Some steps are more
fuzzy than others because there's little that can be automated (e.g.
writing the NEWS entries). Where a step is usually performed by An
Expert, the role of that expert is given. Otherwise, assume the step is
done by the Release Manager (RM), the designated person performing the
release. The roles and their current experts are:
2017-09-05 18:12:11 -04:00
* RM = Release Manager
- Thomas Wouters <thomas@python.org> (NL)
- Pablo Galindo Salgado <pablogsal@python.org> (UK)
- Łukasz Langa <lukasz@python.org> (PL)
2018-10-29 21:59:38 -04:00
* WE = Windows - Steve Dower <steve.dower@python.org>
2017-09-05 18:12:11 -04:00
* ME = Mac - Ned Deily <nad@python.org> (US)
2018-10-29 21:59:38 -04:00
* DE = Docs - Julien Palard <julien@python.org> (Central Europe)
2008-10-01 23:10:35 -04:00
.. note:: It is highly recommended that the RM contact the Experts the day
before the release. Because the world is round and everyone lives
in different timezones, the RM must ensure that the release tag is
created in enough time for the Experts to cut binary releases.
2008-10-01 23:10:35 -04:00
You should not make the release public (by updating the website and
sending announcements) before all experts have updated their bits.
In rare cases where the expert for Windows or Mac is MIA, you may add
a message "(Platform) binaries will be provided shortly" and proceed.
2008-12-05 18:33:03 -05:00
2017-09-04 17:38:47 -04:00
As much as possible, the release steps are automated and guided by the
release script, which is available in a separate repository:
`python/release-tools`_.
2017-09-04 17:38:47 -04:00
We use the following conventions in the examples below. Where a release
2023-10-09 07:01:36 -04:00
number is given, it is of the form ``3.X.YaN``, e.g. 3.13.0a3 for Python 3.13.0
2017-09-04 17:38:47 -04:00
alpha 3, where "a" == alpha, "b" == beta, "rc" == release candidate.
2023-10-09 07:01:36 -04:00
Release tags are named ``v3.X.YaN``. The branch name for minor release
maintenance branches is ``3.X``.
2017-09-04 17:38:47 -04:00
This helps by performing several automatic editing steps, and guides you
to perform some manual editing steps.
2008-02-21 07:00:38 -05:00
2023-10-09 07:01:36 -04:00
- Log into Discord and join the Python Core Devs server. Ask Thomas
or Łukasz for an invite.
2008-08-20 23:04:15 -04:00
2017-09-04 17:38:47 -04:00
You probably need to coordinate with other people around the world.
2023-10-09 07:01:36 -04:00
This communication channel is where we've arranged to meet.
2008-09-12 20:22:56 -04:00
2017-09-04 17:38:47 -04:00
- Check to see if there are any showstopper bugs.
2003-07-24 22:34:02 -04:00
2023-10-09 07:01:36 -04:00
Go to https://github.com/python/cpython/issues and look for any open
bugs that can block this release. You're looking at two relevant labels:
2003-07-24 22:34:02 -04:00
`release-blocker`_
2017-09-04 17:38:47 -04:00
Stops the release dead in its tracks. You may not
make any release with any open release blocker bugs.
2003-07-24 22:34:02 -04:00
`deferred-blocker`_
2017-09-04 17:38:47 -04:00
Doesn't block this release, but it will block a
future release. You may not make a final or
candidate release with any open deferred blocker
bugs.
2008-09-12 20:22:56 -04:00
2017-09-04 17:38:47 -04:00
Review the release blockers and either resolve them, bump them down to
deferred, or stop the release and ask for community assistance. If
you're making a final or candidate release, do the same with any open
deferred.
2017-09-04 17:38:47 -04:00
- Check the stable buildbots.
2003-07-24 22:34:02 -04:00
Go to https://buildbot.python.org/all/#/release_status
2003-07-24 22:34:02 -04:00
2017-09-04 17:38:47 -04:00
Look at the buildbots for the release
you're making. Ignore any that are offline (or inform the community so
they can be restarted). If what remains are (mostly) green buildbots,
you're good to go. If you have non-offline red buildbots, you may want
to hold up the release until they are fixed. Review the problems and
use your judgement, taking into account whether you are making an alpha,
beta, or final release.
2003-07-24 22:34:02 -04:00
2017-09-04 17:38:47 -04:00
- Make a release clone.
On a fork of the CPython repository on GitHub, create a release branch
2017-09-04 17:38:47 -04:00
within it (called the "release clone" from now on). You can use the same
GitHub fork you use for CPython development. Using the standard setup
recommended in the `Python Developer's Guide <https://devguide.python.org/>`__,
your fork would be referred
to as ``origin`` and the standard CPython repo as ``upstream``. You will
2017-09-04 17:38:47 -04:00
use the branch on your fork to do the release engineering work, including
tagging the release, and you will use it to share with the other experts
for making the binaries.
For a **final** or **release candidate 2+** release, if you are going
to cherry-pick a subset of changes for the next rc or final from all those
merged since the last rc, you should create a release
engineering branch starting from the most recent release candidate tag,
i.e. ``v3.8.0rc1``. You will then cherry-pick changes from the standard
release branch as necessary into the release engineering branch and
then proceed as usual. If you are going to take all of the changes
since the previous rc, you can proceed as normal.
2017-09-04 17:38:47 -04:00
- Make sure the current branch of your release clone is the branch you
want to release from (``git status``).
2017-09-04 17:38:47 -04:00
- Run ``blurb release <version>`` specifying the version number
(e.g. ``blurb release 3.4.7rc1``). This merges all the recent news
blurbs into a single file marked with this release's version number.
- Regenerate ``Lib/pydoc-topics.py``.
While still in the ``Doc`` directory, run::
2010-10-06 06:26:59 -04:00
make pydoc-topics
cp build/pydoc-topics/topics.py ../Lib/pydoc_data/topics.py
- Commit your changes to ``pydoc_topics.py``
2017-09-04 17:38:47 -04:00
(and any fixes you made in the docs).
- Consider running ``autoconf`` using the currently accepted standard version
in case ``configure`` or other Autoconf-generated files were last
2017-09-04 17:38:47 -04:00
committed with a newer or older version and may contain spurious or
harmful differences. Currently, Autoconf 2.71 is our de facto standard.
2017-09-04 17:38:47 -04:00
if there are differences, commit them.
- Make sure the ``SOURCE_URI`` in ``Doc/tools/extensions/pyspecific.py``
points to the right branch in the Git repository (``main`` or ``3.X``).
For a **new branch** release, change the branch in the file from ``main``
2023-10-09 07:01:36 -04:00
to the new release branch you are about to create (``3.X``).
- Bump version numbers via the release script::
.../release-tools/release.py --bump 3.X.YaN
2003-07-24 22:34:02 -04:00
Reminder: ``X``, ``Y``, and ``N`` should be integers.
``a`` should be one of ``a``, ``b``, or ``rc`` (e.g. ``3.4.3rc1``).
For **final** releases omit the ``aN`` (``3.4.3``). For the first
release of a new version ``Y`` should be ``0`` (``3.6.0``).
2017-09-04 17:38:47 -04:00
This automates updating various release numbers, but you will have to
modify a few files manually. If your ``$EDITOR`` environment variable is
set up correctly, ``release.py`` will pop up editor windows with the files
2017-09-04 17:38:47 -04:00
you need to edit.
2003-07-24 22:34:02 -04:00
Review the blurb-generated ``Misc/NEWS`` file and edit as necessary.
2003-07-24 22:34:02 -04:00
2017-09-04 17:38:47 -04:00
- Make sure all changes have been committed. (``release.py --bump``
doesn't check in its changes for you.)
2017-09-04 17:38:47 -04:00
- Check the years on the copyright notice. If the last release
was some time last year, add the current year to the copyright
notice in several places:
2003-09-21 21:11:35 -04:00
- ``README``
- ``LICENSE`` (make sure to change on ``main`` and the branch)
- ``Python/getcopyright.c``
- ``Doc/copyright.rst``
- ``Doc/license.rst``
- ``PC/python_ver_rc.h`` sets up the DLL version resource for Windows
2017-09-04 17:38:47 -04:00
(displayed when you right-click on the DLL and select
Properties). This isn't a C include file, it's a Windows
"resource file" include file.
2003-09-21 21:11:35 -04:00
- For a **final** major release, edit the first paragraph of
``Doc/whatsnew/3.X.rst`` to include the actual release date; e.g. "Python
2017-09-04 17:38:47 -04:00
2.5 was released on August 1, 2003." There's no need to edit this for
alpha or beta releases.
- Do a ``git status`` in this directory.
You should not see any files, i.e., you better not have any uncommitted
2017-09-04 17:38:47 -04:00
changes in your working directory.
- Tag the release for ``3.X.YaN``::
2009-06-26 08:54:46 -04:00
.../release-tools/release.py --tag 3.X.YaN
2009-06-26 08:54:46 -04:00
This executes a ``git tag`` command with the ``-s`` option so that the
release tag in the repo is signed with your GPG key. When prompted
2017-09-04 17:38:47 -04:00
choose the private key you use for signing release tarballs etc.
- For **begin security-only mode** and **end-of-life** releases, review the
two files and update the versions accordingly in all active branches.
2017-09-04 17:38:47 -04:00
- Time to build the source tarball. Use the release script to create
the source gzip and xz tarballs,
documentation tar and zip files, and GPG signature files::
.../release-tools/release.py --export 3.X.YaN
This can take a while for **final** releases, and it will leave all the
2023-10-09 07:01:36 -04:00
tarballs and signatures in a subdirectory called ``3.X.YaN/src``, and the
built docs in ``3.X.YaN/docs`` (for **final** releases).
Note that the script will sign your release with Sigstore. Use
your **@python.org** email address for this. See here for more information:
https://www.python.org/download/sigstore/.
2017-09-04 17:38:47 -04:00
- Now you want to perform the very important step of checking the
tarball you just created, to make sure a completely clean,
virgin build passes the regression test. Here are the best
steps to take::
cd /tmp
tar xvf /path/to/your/release/clone/<version>//Python-3.2rc2.tgz
cd Python-3.2rc2
ls
# (Do things look reasonable?)
ls Lib
# (Are there stray .pyc files?)
./configure
# (Loads of configure output)
make test
# (Do all the expected tests pass?)
2017-09-04 17:38:47 -04:00
If you're feeling lucky and have some time to kill, or if you are making
a release candidate or **final** release, run the full test suite::
make testall
2017-09-04 17:38:47 -04:00
If the tests pass, then you can feel good that the tarball is
fine. If some of the tests fail, or anything else about the
freshly unpacked directory looks weird, you better stop now and
figure out what the problem is.
- Push your commits to the remote release branch in your GitHub fork::
2009-06-26 08:54:46 -04:00
2017-09-04 17:38:47 -04:00
# Do a dry run first.
git push --dry-run --tags origin
# Make sure you are pushing to your GitHub fork,
# *not* to the main python/cpython repo!
git push --tags origin
2017-09-04 17:38:47 -04:00
- Notify the experts that they can start building binaries.
2023-10-09 07:01:36 -04:00
.. warning::
2008-12-05 18:33:03 -05:00
2023-10-09 07:01:36 -04:00
**STOP**: at this point you must receive the "green light" from other experts
in order to create the release. There are things you can do while you wait
2017-09-04 17:38:47 -04:00
though, so keep reading until you hit the next STOP.
2008-10-01 23:10:35 -04:00
- The WE generates and publishes the Windows files using the Azure
Pipelines build scripts in ``.azure-pipelines/windows-release/``,
currently set up at https://dev.azure.com/Python/cpython/_build?definitionId=21.
The build process runs in multiple stages, with each stage's output being
available as a downloadable artifact. The stages are:
2008-10-02 09:41:14 -04:00
- Compile all variants of binaries (32-bit, 64-bit, debug/release),
including running profile-guided optimization.
- Compile the HTML Help file containing the Python documentation.
- Codesign all the binaries with the PSF's certificate.
- Create packages for python.org, nuget.org, the embeddable distro and
the Windows Store.
- Perform basic verification of the installers.
- Upload packages to python.org and nuget.org, purge download caches and
run a test download.
After the uploads are complete, the WE copies the generated hashes from
the build logs and emails them to the RM. The Windows Store packages are
uploaded manually to https://partner.microsoft.com/dashboard/home by the
WE.
2017-09-04 17:38:47 -04:00
- The ME builds Mac installer packages and uploads them to
downloads.nyc1.psf.io together with GPG signature files.
- ``scp`` or ``rsync`` all the files built by ``release.py --export``
to your home directory on ``downloads.nyc1.psf.io``.
2008-10-01 23:10:35 -04:00
2017-09-04 17:38:47 -04:00
While you're waiting for the files to finish uploading, you can continue
on with the remaining tasks. You can also ask folks on #python-dev
and/or python-committers to download the files as they finish uploading
so that they can test them on their platforms as well.
2008-10-01 23:10:35 -04:00
- Now you need to go to ``downloads.nyc1.psf.io`` and move all the files in place
2017-09-04 17:38:47 -04:00
over there. Our policy is that every Python version gets its own
directory, but each directory contains all releases of that version.
- On ``downloads.nyc1.psf.io``, ``cd /srv/www.python.org/ftp/python/3.X.Y``
creating it if necessary. Make sure it is owned by group ``downloads``
2017-09-04 17:38:47 -04:00
and group-writable.
- Move the release ``.tgz``, and ``.tar.xz`` files into place, as well as the
``.asc`` GPG signature files. The Win/Mac binaries are usually put there
2017-09-04 17:38:47 -04:00
by the experts themselves.
2008-10-01 23:10:35 -04:00
2017-09-04 17:38:47 -04:00
Make sure they are world readable. They should also be group
writable, and group-owned by ``downloads``.
2017-09-04 17:38:47 -04:00
- Use ``gpg --verify`` to make sure they got uploaded intact.
- If this is a **final** or rc release: Move the doc zips and tarballs to
2023-10-09 07:01:36 -04:00
``/srv/www.python.org/ftp/python/doc/3.X.Y[rcA]``, creating the directory
if necessary, and adapt the "current" symlink in ``.../doc`` to point to
2017-09-04 17:38:47 -04:00
that directory. Note though that if you're releasing a maintenance
release for an older version, don't change the current link.
2008-10-02 09:41:14 -04:00
- If this is a **final** or rc release (even a maintenance release), also
2023-10-09 07:01:36 -04:00
unpack the HTML docs to ``/srv/docs.python.org/release/3.X.Y[rcA]`` on
``docs.nyc1.psf.io``. Make sure the files are in group ``docs`` and are
group-writeable.
2008-12-05 03:19:56 -05:00
2017-09-04 17:38:47 -04:00
- Let the DE check if the docs are built and work all right.
2008-12-05 03:19:56 -05:00
2017-09-04 17:38:47 -04:00
- Note both the documentation and downloads are behind a caching CDN. If
you change archives after downloading them through the website, you'll
need to purge the stale data in the CDN like this::
2014-09-09 11:47:10 -04:00
curl -X PURGE https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tar.xz
2014-09-09 11:47:10 -04:00
2017-09-04 17:38:47 -04:00
You should always purge the cache of the directory listing as people
use that to browse the release files::
curl -X PURGE https://www.python.org/ftp/python/3.12.0/
2017-09-04 17:38:47 -04:00
- For the extra paranoid, do a completely clean test of the release.
This includes downloading the tarball from `www.python.org`_.
2017-09-04 17:38:47 -04:00
Make sure the md5 checksums match. Then unpack the tarball,
and do a clean make test::
make distclean
./configure
make test
2017-09-04 17:38:47 -04:00
To ensure that the regression test suite passes. If not, you
screwed up somewhere!
2023-10-09 07:01:36 -04:00
.. warning::
**STOP** and confirm:
2023-10-09 07:01:36 -04:00
- Have you gotten the green light from the WE?
2023-10-09 07:01:36 -04:00
- Have you gotten the green light from the ME?
2023-10-09 07:01:36 -04:00
- Have you gotten the green light from the DE?
If green, it's time to merge the release engineering branch back into
the main repo.
2023-10-09 07:01:36 -04:00
- In order to push your changes to GitHub, you'll have to temporarily
disable branch protection for administrators. Go to the
``Settings | Branches`` page:
https://github.com/python/cpython/settings/branches
"Edit" the settings for the branch you're releasing on.
This will load the settings page for that branch.
Uncheck the "Include administrators" box and press the
"Save changes" button at the bottom.
- Merge your release clone into the main development repo::
# Pristine copy of the upstream repo branch
git clone git@github.com:python/cpython.git merge
cd merge
# Checkout the correct branch:
# 1. For feature pre-releases up to and including a
# **new branch** release, i.e. alphas and first beta
# do a checkout of the main branch
git checkout main
2019-07-03 14:20:45 -04:00
# 2. Else, for all other releases, checkout the
# appropriate release branch.
git checkout 3.X
2019-07-03 14:20:45 -04:00
# Fetch the newly created and signed tag from your clone repo
git fetch --tags git@github.com:your-github-id/cpython.git v3.X.YaN
2019-07-03 14:20:45 -04:00
# Merge the temporary release engineering branch back into
git merge --no-squash v3.X.YaN
git commit -m 'Merge release engineering branch'
- If this is a **new branch** release, i.e. first beta,
now create the new release branch::
2019-07-03 14:20:45 -04:00
git checkout -b 3.X
Do any steps needed to setup the new release branch, including:
* In ``README.rst``, change all references from ``main`` to
the new branch, in particular, GitHub repo URLs.
- For *all* releases, do the guided post-release steps with the
release script::
.../release-tools/release.py --done 3.X.YaN
- For a **final** or **release candidate 2+** release, you may need to
do some post-merge cleanup. Check the top-level ``README.rst``
and ``include/patchlevel.h`` files to ensure they now reflect
the desired post-release values for on-going development.
The patchlevel should be the release tag with a ``+``.
Also, if you cherry-picked changes from the standard release
branch into the release engineering branch for this release,
you will now need to manually remove each blurb entry from
the ``Misc/NEWS.d/next`` directory that was cherry-picked
into the release you are working on since that blurb entry
is now captured in the merged ``x.y.z.rst`` file for the new
release. Otherwise, the blurb entry will appear twice in
the ``changelog.html`` file, once under ``Python next`` and again
under ``x.y.z``.
- Review and commit these changes::
2019-07-03 14:20:45 -04:00
git commit -m 'Post release updates'
- If this is a **new branch** release (e.g. the first beta),
update the ``main`` branch to start development for the
following feature release. When finished, the ``main``
branch will now build Python ``X.Y+1``.
- First, set ``main`` up to be the next release, i.e. X.Y+1.a0::
git checkout main
.../release-tools/release.py --bump 3.9.0a0
- Edit all version references in ``README.rst``
- Move any historical "what's new" entries from ``Misc/NEWS`` to
``Misc/HISTORY``.
- Edit ``Doc/tutorial/interpreter.rst`` (two references to '[Pp]ython3x',
one to 'Python 3.x', also make the date in the banner consistent).
- Edit ``Doc/tutorial/stdlib.rst`` and ``Doc/tutorial/stdlib2.rst``, which
have each one reference to '[Pp]ython3x'.
- Add a new ``whatsnew/3.x.rst`` file (with the comment near the top
and the toplevel sections copied from the previous file) and
add it to the toctree in ``whatsnew/index.rst``. But beware that
the initial ``whatsnew/3.x.rst`` checkin from previous releases
may be incorrect due to the initial midstream change to ``blurb``
that propagates from release to release! Help break the cycle: if
necessary make the following change:
.. code-block:: diff
-For full details, see the :source:`Misc/NEWS` file.
+For full details, see the :ref:`changelog <changelog>`.
- Update the version number in ``configure.ac`` and re-run ``autoconf``.
- Make sure the ``SOURCE_URI`` in ``Doc/tools/extensions/pyspecific.py``
points to ``main``.
- Update the version numbers for the Windows builds
which have references to ``python38``::
ls PC/pyconfig.h.in PCbuild/rt.bat | xargs sed -i 's/python3\(\.\?\)[0-9]\+/python3\19/g'
- Commit these changes to the main branch::
git status
git add ...
git commit -m 'Bump to 3.9.0a0'
- Do another ``git status`` in this directory.
You should not see any files, i.e., you better not have any uncommitted
changes in your working directory.
- Commit and push to the main repo::
# Do a dry run first.
2019-07-03 14:20:45 -04:00
# For feature pre-releases prior to a **new branch** release,
# i.e. a feature alpha release:
git push --dry-run --tags git@github.com:python/cpython.git main
# If it looks OK, take the plunge. There's no going back!
git push --tags git@github.com:python/cpython.git main
2019-07-03 14:20:45 -04:00
# For a **new branch** release, i.e. first beta:
git push --dry-run --tags git@github.com:python/cpython.git 3.X
git push --dry-run --tags git@github.com:python/cpython.git main
# If it looks OK, take the plunge. There's no going back!
git push --tags git@github.com:python/cpython.git 3.X
git push --tags git@github.com:python/cpython.git main
# For all other releases:
git push --dry-run --tags git@github.com:python/cpython.git 3.X
# If it looks OK, take the plunge. There's no going back!
git push --tags git@github.com:python/cpython.git 3.X
- If this is a **new branch** release, add a ``Branch protection rule``
2023-10-09 07:01:36 -04:00
for the newly created branch (3.X). Look at the values for the previous
release branch (3.X-1) and use them as a template.
https://github.com/python/cpython/settings/branches
2019-07-03 14:20:45 -04:00
2023-10-09 07:01:36 -04:00
Also, add a ``needs backport to 3.X`` label to the GitHub repo.
https://github.com/python/cpython/labels
2019-07-03 14:20:45 -04:00
- You can now re-enable enforcement of branch settings against administrators
2023-10-09 07:01:36 -04:00
on GitHub. Go back to the ``Settings | Branch`` page:
https://github.com/python/cpython/settings/branches
"Edit" the settings for the branch you're releasing on.
Re-check the "Include administrators" box and press the
"Save changes" button at the bottom.
Now it's time to twiddle the website. Almost none of this is automated, sorry.
2017-09-04 17:38:47 -04:00
To do these steps, you must have the permission to edit the website. If you
don't have that, ask someone on pydotorg@python.org for the proper
2019-07-03 14:20:45 -04:00
permissions. (Or ask Ewa, who coordinated the effort for the new website
2017-09-04 17:38:47 -04:00
with RevSys.)
- Log in to https://www.python.org/admin
- Create a new "release" for the release. Currently "Releases" are
2017-09-04 17:38:47 -04:00
sorted under "Downloads".
The easiest thing is probably to copy fields from an existing
Python release "page", editing as you go.
2008-10-01 23:10:35 -04:00
You can use `Markdown <https://daringfireball.net/projects/markdown/syntax>`_ or
`reStructured Text <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
to describe your release. The former is less verbose, while the latter has nifty
integration for things like referencing PEPs.
2008-10-01 23:10:35 -04:00
Leave the "Release page" field on the form empty.
2008-10-01 23:10:35 -04:00
2017-09-04 17:38:47 -04:00
- "Save" the release.
2017-09-04 17:38:47 -04:00
- Populate the release with the downloadable files.
2017-09-04 17:38:47 -04:00
Your friend and mine, Georg Brandl, made a lovely tool
called ``add-to-pydotorg.py``. You can find it in the
`python/release-tools`_ repo (next to ``release.py``). You run the
tool on ``downloads.nyc1.psf.io``, like this::
AUTH_INFO=<username>:<python.org-api-key> python add-to-pydotorg.py <version>
This walks the correct download directory for ``<version>``,
looks for files marked with ``<version>``, and populates
2017-09-04 17:38:47 -04:00
the "Release Files" for the correct "release" on the web
site with these files. Note that clears the "Release Files"
for the relevant version each time it's run. You may run
it from any directory you like, and you can run it as
many times as you like if the files happen to change.
Keep a copy in your home directory on dl-files and
keep it fresh.
2023-10-09 07:01:36 -04:00
If new types of files are added to the release, someone will need to
update ``add-to-pydotorg.py`` so it recognizes these new files.
(It's best to update ``add-to-pydotorg.py`` when file types
2017-09-04 17:38:47 -04:00
are removed, too.)
The script will also sign any remaining files that were not
signed with Sigstore until this point. Again, if this happens,
do use your ``@python.org`` address for this process. More info:
https://www.python.org/download/sigstore/
- In case the CDN already cached a version of the Downloads page
without the files present, you can invalidate the cache using::
curl -X PURGE https://www.python.org/downloads/release/python-XXX/
- If this is a **final** release:
- Add the new version to the *Python Documentation by Version*
page ``https://www.python.org/doc/versions/`` and
2017-09-04 17:38:47 -04:00
remove the current version from any 'in development' section.
2023-10-09 07:01:36 -04:00
- For 3.X.Y, edit all the previous X.Y releases' page(s) to
2017-09-04 17:38:47 -04:00
point to the new release. This includes the content field of the
``Downloads -> Releases`` entry for the release::
2023-10-09 07:01:36 -04:00
Note: Python 3.x.(y-1) has been superseded by
`Python 3.x.y </downloads/release/python-3xy/>`_.
2017-09-04 17:38:47 -04:00
And, for those releases having separate release page entries
(phasing these out?), update those pages as well,
2023-10-09 07:01:36 -04:00
e.g. ``download/releases/3.x.y``::
2023-10-09 07:01:36 -04:00
Note: Python 3.x.(y-1) has been superseded by
`Python 3.x.y </download/releases/3.x.y/>`_.
- Update the "Current Pre-release Testing Versions web page".
There's a page that lists all the currently-in-testing versions
of Python:
* https://www.python.org/download/pre-releases/
Every time you make a release, one way or another you'll
have to update this page:
2023-10-09 07:01:36 -04:00
- If you're releasing a version before *3.x.0*,
you should add it to this page, removing the previous pre-release
2023-10-09 07:01:36 -04:00
of version *3.x* as needed.
2023-10-09 07:01:36 -04:00
- If you're releasing *3.x.0 final*, you need to remove the pre-release
version from this page.
This is in the "Pages" category on the Django-based website, and finding
it through that UI is kind of a chore. However! If you're already logged
in to the admin interface (which, at this point, you should be), Django
will helpfully add a convenient "Edit this page" link to the top of the
page itself. So you can simply follow the link above, click on the
"Edit this page" link, and make your changes as needed. How convenient!
- If appropriate, update the "Python Documentation by Version" page:
* https://www.python.org/doc/versions/
This lists all releases of Python by version number and links to their
static (not built daily) online documentation. There's a list at the
bottom of in-development versions, which is where all alphas/betas/RCs
should go. And yes you should be able to click on the link above then
press the shiny, exciting "Edit this page" button.
- Write the announcement on `discuss.python.org`_. This is the
fuzzy bit because not much can be automated. You can use an earlier
announcement as a template, but edit it for content!
2008-12-05 18:33:03 -05:00
2023-10-09 07:01:36 -04:00
- Once the announcement is up on Discourse, send an equivalent to the
following mailing lists:
* python-list@python.org
* python-announce@python.org
* python-dev@python.org
- Also post the announcement to the
`Python Insider blog <http://blog.python.org>`_.
2017-09-04 17:38:47 -04:00
To add a new entry, go to
`your Blogger home page, here <https://www.blogger.com/home>`_.
2014-08-01 16:39:37 -04:00
2023-10-09 07:01:36 -04:00
- Update any release PEPs (e.g. 719) with the release dates.
2008-07-17 23:44:59 -04:00
2023-10-09 07:01:36 -04:00
- Update the labels on https://github.com/python/cpython/issues:
2009-05-17 18:07:13 -04:00
- Flip all the `deferred-blocker`_ issues back to `release-blocker`_
2017-09-04 17:38:47 -04:00
for the next release.
2009-05-17 18:07:13 -04:00
- Add version ``3.X+1`` as when version ``3.X`` enters alpha.
2009-05-17 18:07:13 -04:00
- Change non-doc feature requests to version ``3.X+1`` when version ``3.X``
2023-10-09 07:01:36 -04:00
enters beta.
2009-05-17 18:07:13 -04:00
2023-10-09 07:01:36 -04:00
- Update issues from versions that your release makes
2017-09-04 17:38:47 -04:00
unsupported to the next supported version.
2009-05-17 18:07:13 -04:00
2017-09-04 17:38:47 -04:00
- Review open issues, as this might find lurking showstopper bugs,
besides reminding people to fix the easy ones they forgot about.
2008-07-17 23:44:59 -04:00
- You can delete the remote release clone branch from your repo clone.
- If this is a **new branch** release, you will need to ensure various
pieces of the development infrastructure are updated for the new branch.
These include:
2019-07-03 14:20:45 -04:00
- Update the `issue tracker`_ for the new branch:
add the new version to the versions list.
- Update the `devguide
<https://github.com/python/devguide/blob/main/include/release-cycle.json>`__
to reflect the new branches and versions.
2020-04-20 12:26:54 -04:00
- Create a PR to update the supported releases table on the
`downloads page <https://www.python.org/downloads/>`__ (see
`python/pythondotorg#1302 <https://github.com/python/pythondotorg/issues/1302>`__).
2020-04-20 12:26:54 -04:00
2023-10-09 07:01:36 -04:00
- Ensure buildbots are defined for the new branch (contact Łukasz
or Zach Ware).
2023-10-09 07:01:36 -04:00
- Ensure the various GitHub bots are updated, as needed, for the
new branch, in particular, make sure backporting to the new
branch works (contact the `core-workflow team
<https://github.com/python/core-workflow/issues>`__).
- Review the most recent commit history for the ``main`` and new release
branches to identify and backport any merges that might have been made
to the ``main`` branch during the release engineering phase and that
should be in the release branch.
- Verify that CI is working for new PRs for the ``main`` and new release
branches and that the release branch is properly protected (no direct
pushes, etc).
- Verify that the `on-line docs <https://docs.python.org/>`__ are building
properly (this may take up to 24 hours for a complete build on the website).
What Next?
2017-09-04 17:38:47 -04:00
==========
* Verify! Pretend you're a user: download the files from `www.python.org`_, and
2017-09-04 17:38:47 -04:00
make Python from it. This step is too easy to overlook, and on several
occasions we've had useless release files. Once a general server problem
caused mysterious corruption of all files; once the source tarball got
built incorrectly; more than once the file upload process on SF truncated
files; and so on.
2010-07-31 09:26:15 -04:00
2017-09-04 17:38:47 -04:00
* Rejoice. Drink. Be Merry. Write a PEP like this one. Or be
like unto Guido and take A Vacation.
2017-09-04 17:38:47 -04:00
You've just made a Python release!
2019-03-15 17:28:49 -04:00
Moving to End-of-life
=====================
Under current policy, a release branch normally reaches end-of-life status
five years after its initial release. The policy is discussed in more detail
in the `Python Developer's Guide
<https://devguide.python.org/developer-workflow/development-cycle/index.html>`_.
2019-03-15 17:28:49 -04:00
When end-of-life is reached, there are a number of tasks that need to be
performed either directly by you as release manager or by ensuring someone
else does them. Some of those tasks include:
- Optionally making a final release to publish any remaining unreleased
changes.
- Freeze the state of the release branch by creating a tag of its current HEAD
and then deleting the branch from the CPython repo. The current HEAD should
2019-03-15 17:28:49 -04:00
be at or beyond the final security release for the branch::
git fetch upstream
git tag --sign -m 'Final head of the former 3.3 branch' 3.3 upstream/3.3
git push upstream refs/tags/3.3
- If all looks good, delete the branch. This may require the assistance of
someone with repo administrator privileges::
2023-10-09 07:01:36 -04:00
git push upstream --delete 3.3 # or perform from GitHub Settings page
2019-03-15 17:28:49 -04:00
- Remove the release from the list of "Active Python Releases" on the Downloads
page. To do this, `log in to the admin page <https://www.python.org/admin>`__
for python.org, navigate to Boxes,
and edit the ``downloads-active-releases`` entry. Strip out the relevant
paragraph of HTML for your release. (You'll probably have to do the ``curl -X PURGE``
trick to purge the cache if you want to confirm you made the change correctly.)
- Add a retired notice to each release page on python.org for the retired branch.
2019-03-15 17:28:49 -04:00
For example:
* https://www.python.org/downloads/release/python-337/
2019-03-15 17:28:49 -04:00
* https://www.python.org/downloads/release/python-336/
2019-03-15 17:28:49 -04:00
- In the `developer's guide
<https://github.com/python/devguide/blob/main/include/release-cycle.json>`__,
set the branch status to end-of-life
and update or remove references to the branch elsewhere in the devguide.
2019-03-15 17:28:49 -04:00
- Retire the release from the `issue tracker`_. Tasks include:
2019-03-15 17:28:49 -04:00
* remove version label from list of versions
2019-03-15 17:28:49 -04:00
* remove the ``needs backport to`` label for the retired version
2019-03-15 17:28:49 -04:00
* review and dispose of open issues marked for this branch
2019-03-15 17:28:49 -04:00
- Announce the branch retirement in the usual places:
* `discuss.python.org`_
2019-03-15 17:28:49 -04:00
* mailing lists (python-dev, python-list, python-announcements)
2023-10-09 07:01:36 -04:00
* Python Dev blog
2019-03-15 17:28:49 -04:00
- Enjoy your retirement and bask in the glow of a job well done!
Windows Notes
2017-09-04 17:38:47 -04:00
=============
Windows has a MSI installer, various flavors of Windows have
"special limitations", and the Windows installer also packs
2023-10-09 07:01:36 -04:00
precompiled "foreign" binaries (Tcl/Tk, expat, etc).
The installer is tested as part of the Azure Pipeline. In the past,
those steps were performed manually. We're keeping this for posterity.
2017-09-04 17:38:47 -04:00
Concurrent with uploading the installer, the WE installs Python
from it twice: once into the default directory suggested by the
installer, and later into a directory with embedded spaces in its
name. For each installation, the WE runs the full regression suite
2017-09-04 17:38:47 -04:00
from a DOS box, and both with and without -0. For maintenance
release, the WE also tests whether upgrade installations succeed.
2017-09-04 17:38:47 -04:00
The WE also tries *every* shortcut created under Start -> Menu -> the
2017-09-04 17:38:47 -04:00
Python group. When trying IDLE this way, you need to verify that
Help -> Python Documentation works. When trying pydoc this way
(the "Module Docs" Start menu entry), make sure the "Start
Browser" button works, and make sure you can search for a random
module (like "random" <wink>) and then that the "go to selected"
button works.
It's amazing how much can go wrong here -- and even more amazing
how often last-second checkins break one of these things. If
you're "the Windows geek", keep in mind that you're likely the
only person routinely testing on Windows, and that Windows is
simply a mess.
Repeat the testing for each target architecture. Try both an
Admin and a plain User (not Power User) account.
Copyright
2017-09-04 17:38:47 -04:00
=========
2017-09-04 17:38:47 -04:00
This document has been placed in the public domain.
.. _blog.python.org: https://blog.python.org
.. _deferred-blocker: https://github.com/python/cpython/labels/deferred-blocker
.. _discuss.python.org: https://discuss.python.org
.. _issue tracker: https://github.com/python/cpython/issues
.. _python/cpython: https://github.com/python/cpython
.. _python/peps: https://github.com/python/peps
.. _python/release-tools: https://github.com/python/release-tools
.. _release-blocker: https://github.com/python/cpython/labels/release-blocker
.. _www.python.org: https://www.python.org