diff --git a/peps/pep-0101.rst b/peps/pep-0101.rst index e877f8958..e84a68679 100644 --- a/peps/pep-0101.rst +++ b/peps/pep-0101.rst @@ -20,12 +20,12 @@ purring little creatures up, and ride them into town, with some of their buddies firmly attached to your bare back, anchored by newly sharpened claws. At least they're cute, you remind yourself. -Actually, no that's a slight exaggeration . The Python release +Actually, no, that's a slight exaggeration 😉 The Python release process has steadily improved over the years and now, with the help of our amazing community, is really not too difficult. This PEP attempts to -collect, in one place, all the steps needed to make a Python release. It -is organized as a recipe and you can actually print this out and check -items off as you complete them. +collect, in one place, all the steps needed to make a Python release. +Most of the steps are now automated or guided by automation, so manually +following this list is no longer necessary. Things You'll Need ================== @@ -41,25 +41,23 @@ Here's a hopefully-complete list. * A bunch of software: - * "release.py", the Python release manager's friend. It's in the - python/release-tools repo on GitHub. It doesn't pip install - or have any sort of install process--you'll have to put it on - your path yourself, or just run it with a relative path, or - whatever. + * A checkout of the `python/release-tools + `_ repo. + It contains a ``requirements.txt`` file that you need to install + dependencies from first. Afterwards, you can fire up scripts in the + repo, covered later in this PEP. - * "blurb", the Misc/NEWS management tool. The release process - currently uses three blurb subcommands: - release, merge, and export. Installable via pip3. - - * "virtualenv". The release script installs Sphinx in a virtualenv - when building the docs (for 2.7 and 3.5+). + * ``blurb``, the + `Misc/NEWS `_ + management tool. You can pip install it. * A fairly complete installation of a recent TeX distribution, such as texlive. You need that for building the PDF docs. -* Access to ``downloads.nyc1.psf.io``, the server that hosts download files, - and ``docs.nyc1.psf.io``, the server that hosts the documentation. - You'll be uploading files directly here. +* Access to servers where you will upload files: + + * ``downloads.nyc1.psf.io``, the server that hosts download files; and + * ``docs.nyc1.psf.io``, the server that hosts the documentation. * Administrator access to ``https://github.com/python/cpython``. @@ -89,7 +87,8 @@ Types of Releases There are several types of releases you will need to make. These include: * ``alpha`` -* ``beta`` +* ``begin beta``, also known as ``beta 1``, also known as ``new branch`` +* ``beta 2+`` * ``release candidate 1`` * ``release candidate 2+`` * ``final`` @@ -105,8 +104,8 @@ organization of the cpython git repository, the *main* branch is always the target for new features. At some point in the release cycle of the next feature release, a **new branch** release is made which creates a new separate branch for stabilization and later maintenance of the -current in-progress feature release (x.y.0) and the *main* branch is modified -to build a new version (which will eventually be released as x.y+1.0). +current in-progress feature release (``3.n.0``) and the *main* branch is modified +to build a new version (which will eventually be released as ``3.n+1.0``). While the **new branch** release step could occur at one of several points in the release cycle, current practice is for it to occur at feature code cutoff for the release which is scheduled for the first beta release. @@ -114,7 +113,7 @@ cutoff for the release which is scheduled for the first beta release. In the descriptions that follow, steps specific to release types are labeled accordingly, for now, **new branch** and **final**. -How to Make A Release +How To Make A Release ===================== Here are the steps taken to make a Python release. Some steps are more @@ -129,12 +128,10 @@ release. The roles and their current experts are: - Thomas Wouters (NL) - Pablo Galindo Salgado (UK) - Łukasz Langa (PL) - - Ned Deily (US) * WE = Windows - Steve Dower * ME = Mac - Ned Deily (US) * DE = Docs - Julien Palard (Central Europe) -* IE = Idle Expert - Terry Reedy (US) .. note:: It is highly recommended that the RM contact the Experts the day before the release. Because the world is round and everyone lives @@ -146,49 +143,42 @@ release. The roles and their current experts are: In rare cases where the expert for Windows or Mac is MIA, you may add a message "(Platform) binaries will be provided shortly" and proceed. -XXX: We should include a dependency graph to illustrate the steps that can -be taken in parallel, or those that depend on other steps. - As much as possible, the release steps are automated and guided by the release script, which is available in a separate repository: https://github.com/python/release-tools We use the following conventions in the examples below. Where a release -number is given, it is of the form ``X.Y.ZaN``, e.g. 3.3.0a3 for Python 3.3.0 +number is given, it is of the form ``3.X.YaN``, e.g. 3.13.0a3 for Python 3.13.0 alpha 3, where "a" == alpha, "b" == beta, "rc" == release candidate. -Release tags are named ``vX.Y.ZaN``. The branch name for minor release -maintenance branches is ``X.Y``. +Release tags are named ``v3.X.YaN``. The branch name for minor release +maintenance branches is ``3.X``. This helps by performing several automatic editing steps, and guides you to perform some manual editing steps. -- Log into irc.libera.chat and join the #python-dev channel. +- Log into Discord and join the Python Core Devs server. Ask Thomas + or Łukasz for an invite. You probably need to coordinate with other people around the world. - This IRC channel is where we've arranged to meet. + This communication channel is where we've arranged to meet. - Check to see if there are any showstopper bugs. - Go to https://bugs.python.org and look for any open bugs that can block - this release. You're looking at the Priority of the open bugs for the - release you're making; here are the relevant definitions: + Go to https://github.com/python/cpython/issues and look for any open + bugs that can block this release. You're looking at two relevant labels: - release blocker + release-blocker Stops the release dead in its tracks. You may not make any release with any open release blocker bugs. - deferred blocker + deferred-blocker Doesn't block this release, but it will block a future release. You may not make a final or candidate release with any open deferred blocker bugs. - critical - Important bugs that should be fixed, but which does not block - a release. - Review the release blockers and either resolve them, bump them down to deferred, or stop the release and ask for community assistance. If you're making a final or candidate release, do the same with any open @@ -244,22 +234,22 @@ to perform some manual editing steps. - Consider running ``autoconf`` using the currently accepted standard version in case ``configure`` or other autoconf-generated files were last committed with a newer or older version and may contain spurious or - harmful differences. Currently, autoconf 2.69 is our de facto standard. + harmful differences. Currently, autoconf 2.71 is our de facto standard. if there are differences, commit them. - Make sure the ``SOURCE_URI`` in ``Doc/tools/extensions/pyspecific.py`` - points to the right branch in the git repository (``main`` or ``X.Y``). + points to the right branch in the git repository (``main`` or ``3.X``). For a **new branch** release, change the branch in the file from *main* - to the new release branch you are about to create (``X.Y``). + to the new release branch you are about to create (``3.X``). - Bump version numbers via the release script:: - $ .../release-tools/release.py --bump X.Y.ZaN + $ .../release-tools/release.py --bump 3.X.YaN - Reminder: X, Y, Z, and N should be integers. + Reminder: X, Y, and N should be integers. a should be one of "a", "b", or "rc" (e.g. "3.4.3rc1"). For **final** releases omit the aN ("3.4.3"). For the first - release of a new version Z should be 0 ("3.6.0"). + release of a new version Y should be 0 ("3.6.0"). This automates updating various release numbers, but you will have to modify a few files manually. If your $EDITOR environment variable is @@ -285,11 +275,8 @@ to perform some manual editing steps. Properties). This isn't a C include file, it's a Windows "resource file" include file. -- Check with the IE (if there is one ) to be sure that - Lib/idlelib/NEWS.txt has been similarly updated. - - For a **final** major release, edit the first paragraph of - Doc/whatsnew/X.Y.rst to include the actual release date; e.g. "Python + Doc/whatsnew/3.X.rst to include the actual release date; e.g. "Python 2.5 was released on August 1, 2003." There's no need to edit this for alpha or beta releases. @@ -298,9 +285,9 @@ to perform some manual editing steps. You should not see any files. I.e. you better not have any uncommitted changes in your working directory. -- Tag the release for X.Y.ZaN:: +- Tag the release for 3.X.YaN:: - $ .../release-tools/release.py --tag X.Y.ZaN + $ .../release-tools/release.py --tag 3.X.YaN This executes a ``git tag`` command with the ``-s`` option so that the release tag in the repo is signed with your gpg key. When prompted @@ -321,11 +308,11 @@ to perform some manual editing steps. the source gzip and xz tarballs, documentation tar and zip files, and gpg signature files:: - $ .../release-tools/release.py --export X.Y.ZaN + $ .../release-tools/release.py --export 3.X.YaN This can take a while for **final** releases, and it will leave all the - tarballs and signatures in a subdirectory called ``X.Y.ZaN/src``, and the - built docs in ``X.Y.ZaN/docs`` (for **final** releases). + tarballs and signatures in a subdirectory called ``3.X.YaN/src``, and the + built docs in ``3.X.YaN/docs`` (for **final** releases). Note that the script will sign your release with Sigstore. Please use your **@python.org** email address for this. See here for more information: @@ -368,20 +355,16 @@ to perform some manual editing steps. - Notify the experts that they can start building binaries. -- STOP STOP STOP STOP STOP STOP STOP STOP +.. warning:: - At this point you must receive the "green light" from other experts in - order to create the release. There are things you can do while you wait + **STOP**: at this point you must receive the "green light" from other experts + in order to create the release. There are things you can do while you wait though, so keep reading until you hit the next STOP. - The WE generates and publishes the Windows files using the Azure Pipelines build scripts in ``.azure-pipelines/windows-release/``, currently set up at https://dev.azure.com/Python/cpython/_build?definitionId=21. - Note that this build requires a separate VM containing the code signing - certificate. This VM is managed by the WE to ensure only official releases - have access to the certificate. - The build process runs in multiple stages, with each stage's output being available as a downloadable artifact. The stages are: @@ -420,7 +403,7 @@ to perform some manual editing steps. over there. Our policy is that every Python version gets its own directory, but each directory contains all releases of that version. - - On downloads.nyc1.psf.io, cd /srv/www.python.org/ftp/python/X.Y.Z + - On downloads.nyc1.psf.io, ``cd /srv/www.python.org/ftp/python/3.X.Y`` creating it if necessary. Make sure it is owned by group 'downloads' and group-writable. @@ -434,13 +417,13 @@ to perform some manual editing steps. - Use ``gpg --verify`` to make sure they got uploaded intact. - If this is a **final** or rc release: Move the doc zips and tarballs to - ``/srv/www.python.org/ftp/python/doc/X.Y.Z[rcA]``, creating the directory + ``/srv/www.python.org/ftp/python/doc/3.X.Y[rcA]``, creating the directory if necessary, and adapt the "current" symlink in ``.../doc`` to point to that directory. Note though that if you're releasing a maintenance release for an older version, don't change the current link. - If this is a **final** or rc release (even a maintenance release), also - unpack the HTML docs to ``/srv/docs.python.org/release/X.Y.Z[rcA]`` on + unpack the HTML docs to ``/srv/docs.python.org/release/3.X.Y[rcA]`` on docs.nyc1.psf.io. Make sure the files are in group ``docs`` and are group-writeable. If it is a release of a security-fix-only version, tell the DE to start a build (``security-fixes`` and ``EOL`` version @@ -449,18 +432,18 @@ to perform some manual editing steps. - Let the DE check if the docs are built and work all right. - If this is a **final** major release: Tell the DE to adapt redirects for - docs.python.org/X.Y in the nginx config for docs.python.org. + docs.python.org/3.X in the nginx config for docs.python.org. - Note both the documentation and downloads are behind a caching CDN. If you change archives after downloading them through the website, you'll need to purge the stale data in the CDN like this:: - $ curl -X PURGE https://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.xz + $ curl -X PURGE https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tar.xz You should always purge the cache of the directory listing as people use that to browse the release files:: - $ curl -X PURGE https://www.python.org/ftp/python/2.7.5/ + $ curl -X PURGE https://www.python.org/ftp/python/3.12.0/ - For the extra paranoid, do a completely clean test of the release. This includes downloading the tarball from www.python.org. @@ -475,18 +458,20 @@ to perform some manual editing steps. To ensure that the regression test suite passes. If not, you screwed up somewhere! -- STOP STOP STOP STOP STOP STOP STOP STOP +.. warning:: - - Have you gotten the green light from the WE? + **STOP** and confirm: - - Have you gotten the green light from the ME? + - Have you gotten the green light from the WE? - - Have you gotten the green light from the DE? + - Have you gotten the green light from the ME? + + - Have you gotten the green light from the DE? If green, it's time to merge the release engineering branch back into the main repo. -- In order to push your changes to Github, you'll have to temporarily +- In order to push your changes to GitHub, you'll have to temporarily disable branch protection for administrators. Go to the ``Settings | Branches`` page: @@ -511,18 +496,18 @@ the main repo. # 2. Else, for all other releases, checkout the # appropriate release branch. - $ git checkout X.Y + $ git checkout 3.X # Fetch the newly created and signed tag from your clone repo - $ git fetch --tags git@github.com:your-github-id/cpython.git vX.Y.ZaN + $ git fetch --tags git@github.com:your-github-id/cpython.git v3.X.YaN # Merge the temporary release engineering branch back into - $ git merge --no-squash vX.Y.ZaN + $ git merge --no-squash v3.X.YaN $ git commit -m 'Merge release engineering branch' - If this is a **new branch** release, i.e. first beta, now create the new release branch:: - $ git checkout -b X.Y + $ git checkout -b 3.X Do any steps needed to setup the new release branch, including: @@ -532,7 +517,7 @@ the main repo. - For *all* releases, do the guided post-release steps with the release script.:: - $ .../release-tools/release.py --done X.Y.ZaN + $ .../release-tools/release.py --done 3.X.YaN - For a **final** or **release candidate 2+** release, you may need to do some post-merge cleanup. Check the top-level ``README.rst`` @@ -621,27 +606,27 @@ the main repo. $ git push --tags git@github.com:python/cpython.git main # For a **new branch** release, i.e. first beta: - $ git push --dry-run --tags git@github.com:python/cpython.git X.Y + $ git push --dry-run --tags git@github.com:python/cpython.git 3.X $ git push --dry-run --tags git@github.com:python/cpython.git main # If it looks OK, take the plunge. There's no going back! - $ git push --tags git@github.com:python/cpython.git X.Y + $ git push --tags git@github.com:python/cpython.git 3.X $ git push --tags git@github.com:python/cpython.git main # For all other releases: - $ git push --dry-run --tags git@github.com:python/cpython.git X.Y + $ git push --dry-run --tags git@github.com:python/cpython.git 3.X # If it looks OK, take the plunge. There's no going back! - $ git push --tags git@github.com:python/cpython.git X.Y + $ git push --tags git@github.com:python/cpython.git 3.X - If this is a **new branch** release, add a ``Branch protection rule`` - for the newly created branch (X.Y). Look at the values for the previous - release branch (X.Y-1) and use them as a template. + for the newly created branch (3.X). Look at the values for the previous + release branch (3.X-1) and use them as a template. https://github.com/python/cpython/settings/branches/ - Also, add a ``needs backport to X.Y`` label to the Github repo. + Also, add a ``needs backport to 3.X`` label to the GitHub repo. https://github.com/python/cpython/labels - You can now re-enable enforcement of branch settings against administrators - on Github. Go back to the ``Settings | Branch`` page: + on GitHub. Go back to the ``Settings | Branch`` page: https://github.com/python/cpython/settings/branches/ @@ -692,10 +677,8 @@ with RevSys.) Keep a copy in your home directory on dl-files and keep it fresh. - If new types of files are added to the release - (e.g. the web-based installers or redistributable zip - files added to Python 3.5) someone will need to update - add-to-pydotorg.py so it recognizes these new files. + If new types of files are added to the release, someone will need to + update add-to-pydotorg.py so it recognizes these new files. (It's best to update add-to-pydotorg.py when file types are removed, too.) @@ -715,19 +698,19 @@ with RevSys.) page ``https://www.python.org/doc/versions/`` and remove the current version from any 'in development' section. - - For X.Y.Z, edit all the previous X.Y releases' page(s) to + - For 3.X.Y, edit all the previous X.Y releases' page(s) to point to the new release. This includes the content field of the ``Downloads -> Releases`` entry for the release:: - Note: Python x.y.m has been superseded by - `Python x.y.n `_. + Note: Python 3.x.(y-1) has been superseded by + `Python 3.x.y `_. And, for those releases having separate release page entries (phasing these out?), update those pages as well, - e.g. ``download/releases/x.y.z``:: + e.g. ``download/releases/3.x.y``:: - Note: Python x.y.m has been superseded by - `Python x.y.n `_. + Note: Python 3.x.(y-1) has been superseded by + `Python 3.x.y `_. - Update the "Current Pre-release Testing Versions web page". @@ -739,12 +722,11 @@ with RevSys.) Every time you make a release, one way or another you'll have to update this page: - - If you're releasing a version before *x.y.0*, - or *x.y.z release candidate N,* + - If you're releasing a version before *3.x.0*, you should add it to this page, removing the previous pre-release - of version *x.y* as needed. + of version *3.x* as needed. - - If you're releasing *x.y.z final*, you need to remove the pre-release + - If you're releasing *3.x.0 final*, you need to remove the pre-release version from this page. This is in the "Pages" category on the Django-based website, and finding @@ -764,15 +746,12 @@ with RevSys.) should go. And yes you should be able to click on the link above then press the shiny, exciting "Edit this page" button. - - Other steps (other update for new web site)?? - -- Write the announcement for the mailing lists. This is the +- Write the announcement on https://discuss.python.org/. This is the fuzzy bit because not much can be automated. You can use an earlier announcement as a template, but edit it for content! - -- Once the announcement is ready, send it to the following - addresses: +- Once the announcement is up on Discourse, send an equivalent to the + following mailing lists: python-list@python.org python-announce@python.org @@ -783,29 +762,19 @@ with RevSys.) To add a new entry, go to `your Blogger home page, here. `_ -- Send email to python-committers informing them that the release has been - published and a reminder about any relevant changes in policy - based on the phase of the release cycle. In particular, - if this is a **new branch** release, remind everyone that the - new release branch exists and that they need to start - considering whether to backport to it when merging changes to - main. +- Update any release PEPs (e.g. 719) with the release dates. -- Update any release PEPs (e.g. 361) with the release dates. +- Update the labels on https://github.com/python/cpython/issues: -- Update the tracker at https://bugs.python.org: - - - Flip all the deferred blocker issues back to release blocker + - Flip all the deferred-blocker issues back to release-blocker for the next release. - - Add version X.Y+1 as when version X.Y enters alpha. + - Add version 3.X+1 as when version 3.X enters alpha. - - Change non-doc RFEs to version X.Y+1 when version X.Y enters beta. + - Change non-doc feature requests to version 3.X+1 when version 3.X + enters beta. - - Add ``X.Yregression`` keyword (https://bugs.python.org/keyword) - when version X.Y enters beta. - - - Update 'behavior' issues from versions that your release make + - Update issues from versions that your release makes unsupported to the next supported version. - Review open issues, as this might find lurking showstopper bugs, @@ -817,13 +786,8 @@ with RevSys.) pieces of the development infrastructure are updated for the new branch. These include: - - Update the issue tracker for the new branch. - - * Add the new version to the versions list (contact the tracker - admins?). - - * Add a `regressions keyword `_ - for the release + - Update the issue tracker for the new branch: add the new version to + the versions list. - Update the devguide to reflect the new branches and versions. @@ -831,12 +795,13 @@ with RevSys.) `downloads page `_. (See https://github.com/python/pythondotorg/issues/1302) - - Ensure buildbots are defined for the new branch (contact zware). + - Ensure buildbots are defined for the new branch (contact Łukasz + or Zach Ware). - Ensure the daily docs build scripts are updated to include the new branch (contact DE). - - Ensure the various Github bots are updated, as needed, for the + - Ensure the various GitHub bots are updated, as needed, for the new branch, in particular, make sure backporting to the new branch works (contact core-workflow team) https://github.com/python/core-workflow/issues @@ -904,7 +869,7 @@ else does them. Some of those tasks include: - If all looks good, delete the branch. This may require the assistance of someone with repo administrator privileges:: - git push upstream --delete 3.3 # or perform from Github Settings page + git push upstream --delete 3.3 # or perform from GitHub Settings page - Remove the release from the list of "Active Python Releases" on the Downloads page. To do this, log in to the admin page for python.org, navigate to Boxes, @@ -923,31 +888,20 @@ else does them. Some of those tasks include: list (https://devguide.python.org/devcycle/#end-of-life-branches) and update or remove references to the branch elsewhere in the devguide. -- Retire the release from the bugs.python.org issue tracker. Tasks include: +- Retire the release from the issue tracker. Tasks include: - * remove branch from tracker list of versions + * remove version label from list of versions - * remove any release-release keywords (3.3regressions) + * remove the "needs backport to" label for the retired version * review and dispose of open issues marked for this branch - Note, with the likely future migration of bug tracking from the current - Roundup bugs.python.org to Github issues and with the impending end-of-life - of Python 2.7, it probably makes sense to avoid unnecessary churn for - currently and about-to-be retired 3.x branches by deferring any major - wholesale changes to existing issues until the migration process is - clarified. - - In practice, you're probably not going to do this yourself, you're going - to ask one of the bpo maintainers to do it for you (e.g. Ezio Melotti, - Zachary Ware.) - - Announce the branch retirement in the usual places: - * mailing lists (python-committers, python-dev, python-list, python-announcements) - * discuss.python.org + * mailing lists (python-dev, python-list, python-announcements) + * Python Dev blog - Enjoy your retirement and bask in the glow of a job well done! @@ -956,12 +910,12 @@ else does them. Some of those tasks include: Windows Notes ============= -NOTE, have Steve Dower review; probably obsolete. - Windows has a MSI installer, various flavors of Windows have "special limitations", and the Windows installer also packs -precompiled "foreign" binaries (Tcl/Tk, expat, etc). So Windows -testing is tiresome but very necessary. +precompiled "foreign" binaries (Tcl/Tk, expat, etc). + +The installer is tested as part of the Azure Pipeline. In the past, +those steps were performed manually. We're keeping this for posterity. Concurrent with uploading the installer, the WE installs Python from it twice: once into the default directory suggested by the