python-peps/pep-0102.txt

PEP: 102
Title: Doing Python Micro Releases
Version: $Revision$
Last-Modified: $Date$
Author: anthony@interlink.com.au (Anthony Baxter)
    barry@zope.com (Barry A. Warsaw)
    guido@python.org (Guido van Rossum)
Status: Active
Type: Informational
Created: 22-Aug-2001 (edited down on 9-Jan-2002 to become PEP 102)
Post-History:


Abstract

    Making a Python release is an arduous processes that takes a
    minimum of half a day's work even for an experienced releaser.
    Until recently, most -- if not all -- of that burden was borne by
    Guido himself.  But several recent releases have been performed by
    other folks, so this PEP attempts to collect, in one place, all
    the steps needed to make a Python bugfix release.

    The major Python release process is covered in PEP 101 - this PEP
    is just PEP 101, trimmed down to only include the bits that are
    relevant for micro releases, a.k.a. patch, or bug fix releases.

    It is organized as a recipe and you can actually print this out and 
    check items off as you complete them.


How to Make A Release

    Here are the steps taken to make a Python release.  Some steps are
    more fuzzy than others because there's little that can be
    automated (e.g. writing the NEWS entries).  Where a step is
    usually performed by An Expert, the name of that expert is given.
    Otherwise, assume the step is done by the Release Manager (RM),
    the designated person performing the release.  Almost every place
    the RM is mentioned below, this step can also be done by the BDFL
    of course!

    XXX: We should include a dependency graph to illustrate the steps
    that can be taken in parallel, or those that depend on other
    steps.

    We use the following conventions in the examples below.  Where a
    release number is given, it is of the form X.Y.MaA, e.g. 2.1.2c1
    for Python 2.1.2 release candidate 1, where "a" == alpha, "b" ==
    beta, "c" == release candidate.  Final releases are tagged with
    "releaseXYZ" in CVS.  The micro releases are made from the
    maintenance branch of the major release, e.g. Python 2.1.2 is made
    from the release21-maint branch.

  ___ Send an email to python-dev@python.org indicating the release is 
      about to start.

  ___ Put a freeze on check ins into the maintenance branch.  At this 
      point, nobody except the RM should make any commits to the branch 
      (or his duly assigned agents, i.e. Guido the BDFL, Fred Drake for
      documentation, or Tim Peters for Windows).  If the RM screwed up
      and some desperate last minute change to the branch is
      necessary, it can mean extra work for Fred and Tim.  So try to
      avoid this!

  ___ On the branch, change Include/patchlevel.h in two places, to
      reflect the new version number you've just created.  You'll want
      to change the PY_VERSION macro, and one or several of the
      version subpart macros just above PY_VERSION, as appropriate.

  ___ If you're changing the version number for Python (e.g. from
      Python 2.1.1 to Python 2.1.2), you also need to update the
      README file, which has a big banner at the top proclaiming its
      identity.  Don't do this if you're just releasing a new alpha or
      beta release, but /do/ do this if you're release a new micro,
      minor or major release.

  ___ The LICENSE file also needs to be changed, due to several
      references to the release number.  As for the README file, changing
      these are necessary for a new micro, minor or major release.

  ___ When the year changes, copyright legends need to be updated in
      many places, including the README and LICENSE files.

  ___ After starting the process, the most important thing to do next
      is to update the Misc/NEWS file.  Tim will need this in order to
      do the Windows release and he likes to stay up late.  This step
      can be pretty tedious, so it's best to get to it immediately
      after making the branch, or even before you've made the branch.
      The sooner the better (but again, watch for new checkins up
      until the release is made!)

      Add high level items new to this release.  E.g. if we're
      releasing 2.2a3, there must be a section at the top of the file
      explaining "What's new in Python 2.2a3".  It will be followed by
      a section entitled "What's new in Python 2.2a2".

      Note that you /hope/ that as developers add new features to the
      trunk, they've updated the NEWS file accordingly.  You can't be
      positive, so double check.  If you're a Unix weenie, it helps to
      verify with Tim Peters about changes on Windows, and Jack Jansen
      about changes on the Mac.

      This command should help you (but substitute the correct -r tag!):

      % cvs log -rr22a1: | python Tools/scripts/logmerge.py > /tmp/news.txt

      IOW, you're printing out all the cvs log entries from the
      previous release until now.  You can then troll through the
      news.txt file looking for interesting things to add to NEWS.

  ___ Check your NEWS changes into the maintenance branch.  It's easy
      to forget to update the release date in this file!

  ___ Once the release is underway, Fred Drake needs to create the 
      HTML from the documentation.  He does this and uploads the file to
      www.python.org.  Then he tells Tim Peters where this file is.
      This may generate some last minute changes on the branch.  Once
      Fred is done, there can be no further checkins on the branch in
      the Doc/ directory -- not even by the RM.  For final releases,
      Fred also sends email to Milan Zamazal for conversion to the GNU
      Info format.

      Note that Fred is responsible both for merging doc changes from
      the trunk to the branch AND for merging any branch changes from
      the branch to the trunk during the cleaning up phase.
      Basically, if it's in Doc/ Fred will take care of it.

  ___ Tim Peters grabs the HTML and uses this to build the Windows
      installer.

  ___ Tim performs his Windows magic, generating an installer
      executable.  He uploads this file to python.org.  He then sends
      the RM a notice which includes the location and MD5 checksum of
      the Windows executable.

      Note that Tim's creation of the Windows executable may generate
      a few more commits on the branch.  Tim will be responsible for
      merging Windows-specific changes from trunk to branch, and from
      branch to trunk.

  ___ It's Build Time!

      Now, you're ready to build the source tarball.  First cd to your
      working directory for the branch.  E.g.
      % cd .../python-22a3

  ___ Do a "cvs update" in this directory.  Do NOT include the -A flag!

      You should not see any "M" files, but you may see several "P"
      files.  I.e. you better not have any uncommitted changes in your
      working directory, but you may pick up some of Fred's or Tim's
      last minute changes.

  ___ Now tag the branch using a symbolic name like "rXYMaZ",
      e.g. r212
      % cvs tag r212

  ___ Change to a neutral directory, i.e. one in which you can do a
      fresh, virgin, cvs export of the branch.  You will be creating a
      new directory at this location, to be named "Python-X.Y.M".  Do
      a CVS export of the tagged branch.

      % cd ~
      % cvs -d <cvsroot> export -rr212 -d Python-2.1.2 python/dist/src

  ___ Generate the tarball.  Note that we're not using the `z' option
      on the tar command because 1) that's only supported by GNU tar
      as far as we know, and 2) we're going to max out the compression
      level, which isn't a supported option.
      % tar -cf - Python-2.1.2 | gzip -9 > Python-2.1.2.tgz

  ___ Calculate the MD5 checksum of the tgz file you just created
      % md5sum Python-2.1.2.tgz

      Note that if you don't have the md5sum program, there is a
      Python replacement in the Tools/scripts/md5sum.py file.

  ___ Now you want to perform the very important step of checking the
      tarball you just created, to make sure a completely clean,
      virgin build passes the regression test.  Here are the best
      steps to take:

      % cd /tmp
      % tar zxvf ~/Python-2.1.2.tgz
      % cd Python-2.1.2
      % ls
      (Do things look reasonable?)
      % ./configure
      (Loads of configure output)
      % make test
      (Do all the expected tests pass?)

      If the tests pass, then you can feel good that the tarball is
      fine.  If some of the tests fail, or anything else about the
      freshly unpacked directory looks weird, you better stop now and
      figure out what the problem is.

  ___ You need to upload the tgz file to creosote.python.org.  Tim
      will have already uploaded the exe file to creosote, but if not,
      you'll need to do that too.  These steps can take a long time
      depending on your network bandwidth.  scp both files from your
      own machine to creosote.

  ___ While you're waiting, you can start twiddling the web pages to
      include the announcement.

    ___ In the python.org web site CVS tree, cd to the X.Y
        subdirectory, and copy index.ht to new-index.ht.  Be sure to
        do a "cvs update" first!

        % cd .../pydotorg
        % cvs -q up -P -d
        % cd 2.2
        % cp index.ht new-index.ht

    ___ Edit the file for content: usually you can globally replace
        X.Ya(Z-1) with X.YaZ.  However, you'll need to think about the
        "What's New?" section.

    ___ Copy the Misc/NEWS file to NEWS.txt in the X.Y directory for
        python.org; this contains the "full scoop" of changes to
        Python since the previous release for this version of Python.

        If this is the first release in a series (e.g. 2.2.1c1) then
        you'll need to create a new directory (e.g. 2.2.1).  It's
        probably best to copy the last directory across:

        % cp -r 2.2 2.2.1
        % rm -rf 2.2.1/CVS
        % cvs add 2.2.1

        and then make changes as needed.  Verbiage from prereleases of
        previous versions can be found in CVS if you're feeling uninspired.

    ___ Also, update the MD5 checksums.

    ___ Preview the web page by doing a "make" -- NOT a "make install".
        View the page via a file: url.

    ___ Similarly, edit the ../index.ht file, i.e. the python.org home
        page.  In the Big Blue Announcement Block, move the paragraph
        for the new version up to the top and boldify the phrase
        "Python X.YaZ is out".  Edit for content, and preview as
        above.  Do NOT do a "make install" yet!

  ___ Now we're waiting for the scp to creosote to finish.  Da de da,
      da de dum, hmm, hmm, dum de dum.

  ___ Once that's done you need to go to creosote.python.org and move
      all the files in place over there.  Our policy is that every
      Python version gets its own directory, but each directory may
      contain several releases.  We keep all old releases, moving them
      into a "prev" subdirectory when we have a new release.

      So, there's a directory called "2.2" which contains
      Python-2.2a2.exe and Python-2.2a2.tgz, along with a "prev"
      subdirectory containing Python-2.2a1.exe and Python-2.2a1.tgz.

      So...

    ___ On creosote, cd to ~ftp/pub/python/X.Y creating it if
        necessary.

    ___ Move the previous release files to a directory called "prev"
        creating the directory if necessary (make sure the directory
        has g+ws bits on).  If this is the first alpha release of a
        new Python version, skip this step.

    ___ Move the .tgz file and the .exe file to this directory.  Make
        sure they are world readable.  They should also be group
        writable, and group-owned by webmaster.

    ___ md5sum the files and make sure they got uploaded intact.


  ___ Update the X.Y/bugs.ht file if necessary.  It is best to get
      BDFL input for this step.

  ___ Now preview the new-index.ht file once more.  IMPORTANT: follow
      every link on the page to make sure it goes where you expect it
      to go, and that what you expect to be there is there.

  ___ If everything looks good, move new-index.ht to index.ht and do a
      "make install" in this directory.  Go up to the parent directory
      (i.e. the root of the web page hierarchy) and do a "make
      install" there too.  You're release is now live!

  ___ Now it's time to write the announcement for the mailing lists.
      This is the fuzzy bit because not much can be automated.  You
      can use one of Guido's earlier announcements as a template, but
      please edit it for content!

      Once the announcement is ready, send it to the following
      addresses:

      python-list@python.org
      python-announce@python.org
      python-dev@python.org

  ___ Send a SourceForge News Item about the release.  From the
      project's "menu bar", select the "News" link; once in News,
      select the "Submit" link.  Type a suitable subject (e.g. "Python
      2.2c1 released" :-) in the Subject box, add some text to the
      Details box (at the very least including the release URL at
      www.python.org and the fact that you're happy with the release)
      and click the SUBMIT button.

      Feel free to remove any old news items.

    Now it's time to do some cleanup.  These steps are very important!

  ___ Edit the file Include/patchlevel.h so that the PY_VERSION
      string says something like "X.YaZ+".  Note the trailing `+'
      indicating that the trunk is going to be moving forward with
      development.  E.g. the line should look like:

      #define PY_VERSION              "2.1.2+"

      Make sure that the other PY_ version macros contain the
      correct values.  Commit this change.

  ___ For the extra paranoid, do a completely clean test of the
      release.  This includes downloading the tarball from
      www.python.org.

  ___ Make sure the md5 checksums match.  Then unpack the tarball,
      and do a clean make test.

      % make distclean
      % ./configure
      % make test

      To ensure that the regression test suite passes.  If not, you
      screwed up somewhere!

    Step 5 ...

    Verify!  This can be interleaved with Step 4.  Pretend you're a
    user:  download the files from python.org, and make Python from
    it.  This step is too easy to overlook, and on several occasions
    we've had useless release files.  Once a general server problem
    caused mysterious corruption of all files; once the source tarball
    got built incorrectly; more than once the file upload process on
    SF truncated files; and so on.


What Next?

    Rejoice.  Drink.  Be Merry.  Write a PEP like this one.  Or be
    like unto Guido and take A Vacation.

    You've just made a Python release!

    Actually, there is one more step.  You should turn over ownership
    of the branch to Jack Jansen.  All this means is that now he will
    be responsible for making commits to the branch.  He's going to
    use this to build the MacOS versions.  He may send you information
    about the Mac release that should be merged into the informational
    pages on www.python.org.  When he's done, he'll tag the branch
    something like "rX.YaZ-mac".  He'll also be responsible for
    merging any Mac-related changes back into the trunk.


Final Release Notes

    The Final release of any major release, e.g. Python 2.2 final, has
    special requirements, specifically because it will be one of the
    longest lived releases (i.e. betas don't last more than a couple
    of weeks, but final releases can last for years!).

    For this reason we want to have a higher coordination between the
    three major releases: Windows, Mac, and source.  The Windows and
    source releases benefit from the close proximity of the respective
    release-bots.  But the Mac-bot, Jack Jansen, is 6 hours away.  So
    we add this extra step to the release process for a final
    release:

    ___ Hold up the final release until Jack approves, or until we
        lose patience <wink>.


Windows Notes

    Windows has a GUI installer, various flavors of Windows have
    "special limitations", and the Windows installer also packs
    precompiled "foreign" binaries (Tcl/Tk, expat, etc).  So Windows
    testing is tiresome but very necessary.

    Concurrent with uploading the installer, Tim installs Python from
    it twice: once into the default directory suggested by the
    installer, and later into a directory with embedded spaces in its
    name.  For each installation, he runs the full regression suite
    from a DOS box, and both with and without -0.

    He also tries *every* shortcut created under Start -> Menu -> the
    Python group.  When trying IDLE this way, you need to verify that
    Help -> Python Documentation works.  When trying pydoc this way
    (the "Module Docs" Start menu entry), make sure the "Start
    Browser" button works, and make sure you can search for a random
    module (Tim uses "random" <wink>) and then that the "go to
    selected" button works.

    It's amazing how much can go wrong here -- and even more amazing
    how often last-second checkins break one of these things.  If
    you're "the Windows geek", keep in mind that you're likely the
    only person routinely testing on Windows, and that Windows is
    simply a mess.

    Repeat all of the above on at least one flavor of Win9x, and one
    of NT/2000.  On NT/2000, try both an Admin and a plain User (not
    Power User) account.

    WRT Step 5 above (verify the release media), since by the time
    release files are ready to download Tim has generally run many
    Windows tests on the installer he uploaded, he usually doesn't do
    anything for Step 5 except a full byte-comparison ("fc /b" if
    using a Windows shell) of the downloaded file against the file he
    uploaded.


Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: