Add PEP 502: Migrating from hg.python.org to GitHub
This commit is contained in:
parent
796880beeb
commit
7f86b12f64
|
@ -0,0 +1,782 @@
|
|||
PEP: 512
|
||||
Title: Migrating from hg.python.org to GitHub
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Brett Cannon <brett@python.org>
|
||||
Status: Active
|
||||
Type: Process
|
||||
Content-Type: text/x-rst
|
||||
Created:
|
||||
Post-History: 17-Jan-2015
|
||||
|
||||
Abstract
|
||||
========
|
||||
This PEP outlines the steps required to migrate Python's development
|
||||
process from Mercurial [#hg]_ as hosted at
|
||||
hg.python.org [#h.p.o]_ to Git [#git]_ on GitHub [#GitHub]_. Meeting
|
||||
the minimum goals of this PEP should allow for the development
|
||||
process of Python to be as productive as it currently is, and meeting
|
||||
its extended goals should improve it.
|
||||
|
||||
Rationale
|
||||
=========
|
||||
In 2014, it became obvious that Python's custom development
|
||||
process was becoming a hindrance. As an example, for an external
|
||||
contributor to submit a fix for a bug that eventually was committed,
|
||||
the basic steps were:
|
||||
|
||||
1. Open an issue for the bug at bugs.python.org [#b.p.o]_.
|
||||
2. Checkout out the CPython source code from hg.python.org [#h.p.o]_.
|
||||
3. Make the fix.
|
||||
4. Upload a patch.
|
||||
5. Have a core developer review the patch using our fork of the
|
||||
Rietveld code review tool [#rietveld]_.
|
||||
6. Download the patch to make sure it still applies cleanly.
|
||||
7. Run the test suite manually.
|
||||
8. Commit the change manually.
|
||||
9. If the change was for a bugfix release, merge into the
|
||||
in-development branch.
|
||||
10. Run the test suite manually again.
|
||||
11. Commit the merge.
|
||||
12. Push the changes.
|
||||
|
||||
This is a very heavy, manual process for core developers. Even in the
|
||||
simple case, you could only possibly skip the code review step, as you
|
||||
would still need to build the documentation. This led to patches
|
||||
languishing on the issue tracker due to core developers not being
|
||||
able to work through the backlog fast enough to keep up with
|
||||
submissions. In turn, that led to a side-effect issue of discouraging
|
||||
outside contribution due to frustration from lack of attention, which
|
||||
is dangerous problem for an open source project as it runs counter to
|
||||
having a viable future for the project. Simply accepting patches
|
||||
uploaded to bugs.python.org [#b.p.o]_ is potentially simple for an
|
||||
external contributor, it is as slow and burdensome as it gets for
|
||||
a core developer to work with.
|
||||
|
||||
Hence the decision was made in late 2014 that a move to a new
|
||||
development process was needed. A request for PEPs
|
||||
proposing new workflows was made, in the end leading to two:
|
||||
PEP 481 and PEP 507 proposing GitHub [#github]_ and
|
||||
GitLab [#gitlab]_, respectively.
|
||||
|
||||
The year 2015 was spent off-and-on working on those proposals and
|
||||
trying to tease out details of what made them different from each
|
||||
other on the core-workflow mailing list [#core-workflow]_.
|
||||
PyCon US 2015 also showed that the community was a bit frustrated
|
||||
with our process due to both cognitive overhead for new contributors
|
||||
and how long it was taking for core developers to
|
||||
look at a patch (see the end of Guido van Rossum's
|
||||
keynote at PyCon US 2015 [#guido-keynote]_ as an example of the
|
||||
frustration).
|
||||
|
||||
On January 1, 2016, the decision was made by Brett Cannon to move the
|
||||
development process to GitHub. The key reasons for choosing GitHub
|
||||
were [#reasons]_:
|
||||
|
||||
* Maintaining custom infrastructure has been a burden on volunteers
|
||||
(e.g., a custom fork of Rietveld [#rietveld]_
|
||||
that is not being maintained is currently being used).
|
||||
* The custom workflow is very time-consuming for core developers
|
||||
(not enough automated tooling built to help support it).
|
||||
* The custom workflow is a hindrance to external contributors
|
||||
(acts as a barrier of entry due to time required to ramp up on
|
||||
development process).
|
||||
* There is no feature differentiating GitLab from GitHub beyond
|
||||
GitLab being open source.
|
||||
* Familiarity with GitHub is far higher amongst core developers and
|
||||
external contributors than with GitLab.
|
||||
* Our BDFL prefers GitHub (who would be the first person to tell
|
||||
you that his opinion shouldn't matter, but the person making the
|
||||
decision felt it was important that the BDFL feel comfortable with
|
||||
the workflow of his own programming language to encourage his
|
||||
continued participation).
|
||||
|
||||
There's even already an unofficial image to use to represent the
|
||||
migration to GitHub [#pythocat]_.
|
||||
|
||||
The overarching goal of this migration is to improve the development
|
||||
process to the extent that a core developer can go from external
|
||||
contribution submission through all the steps leading to committing
|
||||
said contribution all from within a browser on a tablet with WiFi.
|
||||
All of this will be done in such a way that if an external
|
||||
contributor chooses not to use GitHub then they will continue to have
|
||||
that option.
|
||||
|
||||
Repositories to Migrate
|
||||
=======================
|
||||
While hg.python.org [#h.p.o]_ hosts many repositories, there are only
|
||||
six key repositories that must move:
|
||||
|
||||
1. devinabox [#devinabox-repo]_
|
||||
2. benchmarks [#benchmarks-repo]_
|
||||
3. tracker [#tracker-repo]_
|
||||
4. peps [#peps-repo]_
|
||||
5. devguide [#devguide-repo]_
|
||||
6. cpython [#cpython-repo]_
|
||||
|
||||
The devinabox, benchmarksm and tracker repositories are code-only.
|
||||
The peps and devguide repositories involve the generation of webpages.
|
||||
And the cpython repository has special requirements for integration
|
||||
with bugs.python.org [#b.p.o]_.
|
||||
|
||||
Migration Plan
|
||||
==============
|
||||
The migration plan is separated into sections based on what is
|
||||
required to migrate the repositories listed in the
|
||||
`Repositories to Migrate`_ section. Completion of requirements
|
||||
outlined in each section should unblock the migration of the related
|
||||
repositories. The sections are expected to be completed in order, but
|
||||
not necessarily the requirements within a section.
|
||||
|
||||
Requirements for Code-Only Repositories
|
||||
---------------------------------------
|
||||
Completion of the requirements in this section will allow the
|
||||
devinabox, benchmarks, and tracker repositories to move to
|
||||
GitHub. While devinabox has a sufficiently descriptive name, the
|
||||
benchmarks repository does not; therefore, it will be named
|
||||
"python-benchmark-suite". The tracker repo also has a misleading name
|
||||
and will be renamed "bugs.python.org".
|
||||
|
||||
Create a 'python-dev' team
|
||||
''''''''''''''''''''''''''
|
||||
To manage permissions, a 'python-dev' team will be created as part of
|
||||
the python organization [#github-python-org]_. Any repository that is
|
||||
moved will have the 'python-dev' team added to it with write
|
||||
permissions [#github-org-perms]_. Anyone who previously had rights to
|
||||
manage SSH keys on hg.python.org will become a team maintainer for the
|
||||
'python-dev' team.
|
||||
|
||||
Define commands to move a Mercurial repository to Git
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
Since moving to GitHub also entails moving to Git [#git]_, we must
|
||||
decide what tools and commands we will run to translate a Mercurial
|
||||
repository to Git. The exact tools and steps to use are an
|
||||
open issue; see `Tools and commands to move from Mercurial to Git`_.
|
||||
|
||||
CLA enforcement
|
||||
'''''''''''''''
|
||||
A key part of any open source project is making sure that its source
|
||||
code can be properly licensed. This requires making sure all people
|
||||
making contributions have signed a contributor license agreement
|
||||
(CLA) [#cla]_. Up until now, enforcement of CLA signing of
|
||||
contributed code has been enforced by core developers checking
|
||||
whether someone had an ``*`` by their username on
|
||||
bugs.python.org [#b.p.o]_. With this migration, the plan is to start
|
||||
off with automated checking and enforcement of contributors signing
|
||||
the CLA.
|
||||
|
||||
Adding GitHub username support to bugs.python.org
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
To keep tracking of CLA signing under the direct control of the PSF,
|
||||
tracking who has signed the PSF CLA will be continued by marking that
|
||||
fact as part of someone's bugs.python.org user profile. What this
|
||||
means is that an association will be needed between a person's
|
||||
bugs.python.org [#b.p.o]_ account and their GitHub account, which
|
||||
will be done through a new field in a user's profile.
|
||||
|
||||
A bot to enforce CLA signing
|
||||
++++++++++++++++++++++++++++
|
||||
With an association between someone's GitHub account and their
|
||||
bugs.python.org [#b.p.o]_ account, which has the data as to whether
|
||||
someone has signed the CLA, a bot can monitor pull requests on
|
||||
GitHub and denote whether the contributor has signed the CLA.
|
||||
|
||||
If the user has signed the CLA, the bot will add a positive label to
|
||||
the issue to denote the pull request has no CLA issues (e.g., a green
|
||||
label stating, "CLA: ✓"). If the contributor has not signed a CLA,
|
||||
a negative label will be added to the pull request will be blocked
|
||||
using GitHub's status API (e.g., a red label stating, "CLA: ✗"). If a
|
||||
contributor lacks a bugs.python.org account, that will lead to
|
||||
another label (e.g., "CLA: ✗ (no account)"). Using a label for both
|
||||
positive and negative cases provides a fallback notification if the
|
||||
bot happens to fail, preventing potential false-positives or
|
||||
false-negatives. It also allows for an easy way to trigger the bot
|
||||
again by simply removing a CLA-related label.
|
||||
|
||||
Requirements for Web-Related Repositories
|
||||
-----------------------------------------
|
||||
Due to their use for generating webpages, the
|
||||
devguide [#devguide-repo]_ and peps [#peps-repo]_ repositories need
|
||||
their respective processes updated to pull from their new Git
|
||||
repositories.
|
||||
|
||||
The devguide repository might also need to be named
|
||||
``python-devguide`` to make sure the repository is not ambiguous
|
||||
when viewed in isolation from the
|
||||
python organization [#github-python-org]_.
|
||||
|
||||
Requirements for the cpython Repository
|
||||
---------------------------------------
|
||||
Obviously the most active and important repository currently hosted
|
||||
at hg.python.org [#h.p.o]_ is the cpython
|
||||
repository [#cpython-repo]_. Because of its importance and high-
|
||||
frequency use, it requires more tooling before being moved to GitHub
|
||||
compared to the other repositories mentioned in this PEP.
|
||||
|
||||
Document steps to commit a pull request
|
||||
'''''''''''''''''''''''''''''''''''''''
|
||||
During the process of choosing a new development workflow, it was
|
||||
decided that a linear history is desired. This means that the
|
||||
convenient "Merge" button in GitHub pull requests is undesireable, as
|
||||
it creates a merge commit along with all of the contributor's
|
||||
individual commits (this does not affect the other repositories where
|
||||
the desire for a linear history doesn't exist).
|
||||
|
||||
Luckily, Git [#git]_ does not require GitHub's workflow and so one can
|
||||
be chosen which gives us a linear history by using Git's CLI. The
|
||||
expectation is that all pull requests will be fast-forwarded and
|
||||
rebased before being pushed to the master repository. This should
|
||||
give proper attribution to the pull request author in the Git
|
||||
history.
|
||||
|
||||
A second set of recommended commands will also be written for
|
||||
committing a contribution from a patch file uploaded to
|
||||
bugs.python.org [#b.p.o]_. This will obviously help keep the linear
|
||||
history, but it will need to be made to have attribution to the patch
|
||||
author.
|
||||
|
||||
The exact sequence of commands that will be given as guidelines to
|
||||
core developers is an open issue:
|
||||
`Git CLI commands for committing a pull request to cpython`_.
|
||||
|
||||
Handling Misc/NEWS
|
||||
''''''''''''''''''
|
||||
Traditionally the ``Misc/NEWS`` file [#news-file]_ has been problematic
|
||||
for changes which spanned Python releases. Often times there will be
|
||||
merge conflicts when committing a change between e.g., 3.5 and 3.6
|
||||
only in the ``Misc/NEWS`` file. It's so common, in fact, that the
|
||||
example instructions in the devguide explicitly mention how to
|
||||
resolve conflicts in the ``Misc/NEWS`` file
|
||||
[#devguide-merge-across-branches]_. As part of our tool
|
||||
modernization, working with the ``Misc/NEWS`` file will be
|
||||
simplified.
|
||||
|
||||
There are currently two competing approaches to solving the
|
||||
``Misc/NEWS`` problem which are discussed in an open issue:
|
||||
`How to handle the Misc/NEWS file`_.
|
||||
|
||||
Handling Misc/ACKS
|
||||
''''''''''''''''''
|
||||
Traditionally the ``Misc/ACKS`` file [#acks-file]_ has been managed
|
||||
by hand. But thanks to Git supporting an ``author`` value as well as
|
||||
a ``committer`` value per commit, authorship of a commit can be part
|
||||
of the history of the code itself.
|
||||
|
||||
As such, manual management of ``Misc/ACKS`` will become optional. A
|
||||
script will be written that will collect all author and committer
|
||||
names and merge them into ``Misc/ACKS`` with all of the names listed
|
||||
prior to the move to Git. Running this script will become part of the
|
||||
release process.
|
||||
|
||||
Linking pull requests to issues
|
||||
'''''''''''''''''''''''''''''''
|
||||
Historically, external contributions were attached to an issue on
|
||||
bugs.python.org [#b.p.o]_ thanks to the fact that all external
|
||||
contributions were uploaded as a file. For changes committed by a
|
||||
core developer who committed a change directly, the specifying of an
|
||||
issue number in the commit message of the format ``Issue #`` at the
|
||||
start of the message led to a comment being posted to the issue
|
||||
linking to the commit.
|
||||
|
||||
Linking a pull request to an issue
|
||||
++++++++++++++++++++++++++++++++++
|
||||
An association between a pull request and an issue is needed to track
|
||||
when a fix has been proposed. The association needs to be many-to-one
|
||||
as there can take multiple pull requests to solve a single issue
|
||||
(technically it should be a many-to-many association for when a
|
||||
single fix solves multiple issues, but this is fairly rare and issues
|
||||
can be merged into one using the ``Superceder`` field on the issue
|
||||
tracker).
|
||||
|
||||
Association between a pull request and an issue will be done based on
|
||||
detecting the regular expression``[Ii]ssue #(?P<bpo_id>\d+)``. If
|
||||
this is specified in either the title or in the body of a message on
|
||||
a pull request then connection will be made on
|
||||
bugs.python.org [#b.p.o]_. A label will also be added to the pull
|
||||
request to signify that the connection was made successfully. This
|
||||
could lead to incorrect associations if the wrong issue or
|
||||
referencing another issue was done, but these are rare occasions.
|
||||
|
||||
Notify the issue if the pull request is committed
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
Once a pull request is closed (merged or not), the issue should be
|
||||
updated to reflect this fact.
|
||||
|
||||
Update linking service for mapping commit IDs to URLs
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
Currently you can use https://hg.python.org/lookup/ with a revision
|
||||
ID from either the Subversion or Mercurial copies of the
|
||||
cpython repo [#cpython-repo]_ to get redirected to the URL for that
|
||||
revision in the Mercurial repository. The URL rewriter will need to
|
||||
be updated to redirect to the Git repository and to support the new
|
||||
revision IDs created for the Git repository.
|
||||
|
||||
Create https://git.python.org
|
||||
'''''''''''''''''''''''''''''
|
||||
Just as hg.python.org [#h.p.o]_ currently points to the Mercurial
|
||||
repository for Python, git.python.org should do the equivalent for
|
||||
the Git repository.
|
||||
|
||||
Backup of pull request data
|
||||
'''''''''''''''''''''''''''
|
||||
Since GitHub [#github]_ is going to be used for code hosting and code
|
||||
review, those two things need to be backed up. In the case of code
|
||||
hosting, the backup is implicit as all non-shallow Git [#git]_ clones
|
||||
contain the full history of the repository, hence there will be many
|
||||
backups of the repository.
|
||||
|
||||
The code review history does not have the same implicit backup
|
||||
mechanism as the repository itself. That means a daily backup of code
|
||||
review history should be done so that it is not lost in case of any
|
||||
issues with GitHub. It also helps guarantee that a migration from
|
||||
GitHub to some other code review system is feasible were GitHub to
|
||||
disappear overnight.
|
||||
|
||||
Change ``sys._mercurial``
|
||||
'''''''''''''''''''''''''
|
||||
Once Python is no longer kept in Mercurial, the ``sys._mercurial``
|
||||
attribute will need to be removed. An equivalent ``sys._git``
|
||||
attribute will be needed to take its place.
|
||||
|
||||
Optional, Planned Features
|
||||
--------------------------
|
||||
Once the cpython repository [#cpython-repo]_ is migrated, all
|
||||
repositories will have been moved to GitHub [#github]_ and the
|
||||
development process should be on equal footing as before. But a key
|
||||
reason for this migration is to improve the development process,
|
||||
making it better than it has ever been. This section outlines some
|
||||
plans on how to improve things.
|
||||
|
||||
It should be mentioned that overall feature planning for
|
||||
bugs.python.org [#b.p.o]_ -- which includes plans independent of this
|
||||
migration -- are tracked on their own wiki page [#tracker-plans]_.
|
||||
|
||||
Bot to handle pull request merging
|
||||
''''''''''''''''''''''''''''''''''
|
||||
As stated in the section entitled
|
||||
"`Document steps to commit a pull request`_", the desire is to
|
||||
maintain a linear history for cpython. Unfortunately,
|
||||
Github's [#github]_ web-based workflow does not support a linear
|
||||
history. Because of this, a bot should be written to substitute for
|
||||
GitHub's in-browser commit abilities.
|
||||
|
||||
To start, the bot should accept commands to commit a pull request
|
||||
against a list of branches. This allows for committing a pull request
|
||||
that fixes a bug in multiple versions of Python.
|
||||
|
||||
More advanced features such as a commit queue can come later. This
|
||||
would linearly apply accepted pull requests and verify that the
|
||||
commits did not interfere with each other by running the test suite
|
||||
and backing out commits if the test run failed. To help facilitate
|
||||
the speed of testing, all patches committed since the last test run
|
||||
can be applied and run in a single test run as the optimistic
|
||||
assumption is that the patches will work in tandem.
|
||||
|
||||
Inspiration or basis of the bot could be taken from pre-existig bots
|
||||
such as Homu [#homu]_ or Zuul [#zuul]_.
|
||||
|
||||
The name given to this bot in order to give it commands is an open
|
||||
issue: `Naming the commit bot`_.
|
||||
|
||||
Continuous integration per pull request
|
||||
'''''''''''''''''''''''''''''''''''''''
|
||||
To help speed up pull request approvals, continuous integration
|
||||
testing should be used. This helps mitigate the need for a core
|
||||
developer to download a patch simply to run the test suite against
|
||||
the patch.
|
||||
|
||||
Which free CI service to use is an open issue:
|
||||
`Choosing a CI service`_.
|
||||
|
||||
Test coverage report
|
||||
''''''''''''''''''''
|
||||
Getting an up-to-date test coverage report for Python's standard
|
||||
library would be extremely beneficial as generating such a report can
|
||||
take quite a while to produce.
|
||||
|
||||
There are a couple pre-existing services that provide free test
|
||||
coverage for open source projects. Which option is best is an open
|
||||
issue: `Choosing a test coverage service`_.
|
||||
|
||||
Notifying issues of pull request comments
|
||||
'''''''''''''''''''''''''''''''''''''''''
|
||||
The current development process does not include notifying an issue
|
||||
on bugs.python.org [#b.p.o]_ when a review comment is left on
|
||||
Rietveld [#rietveld]_. It would be nice to fix this so that people
|
||||
can subscribe only to comments at bugs.python.org and not
|
||||
GitHub [#github]_ and yet still know when something occurs on GitHub
|
||||
in terms of review comments on relevant pull requests. Current
|
||||
thinking is to post a comment to bugs.python.org to the relevant
|
||||
issue when at least one review comment has been made over a certain
|
||||
period of time (e.g., 15 or 30 minutes). This keeps the email volume
|
||||
down for those that receive both GitHub and bugs.python.org email
|
||||
notifications while still making sure that those only following
|
||||
bugs.python.org know when there might be a review comment to address.
|
||||
|
||||
Allow bugs.python.org to use GitHub as a login provider
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
As of right now, bugs.python.org [#b.p.o]_ allows people to log in
|
||||
using Google, Launchpad, or OpenID credentials. It would be good to
|
||||
expand this to GitHub credentials.
|
||||
|
||||
Web hooks for re-generating web content
|
||||
'''''''''''''''''''''''''''''''''''''''
|
||||
The content at https://docs.python.org/,
|
||||
https://docs.python.org/devguide, and
|
||||
https://www.python.org/dev/peps/ are all derived from files kept in
|
||||
one of the repositories to be moved as part of this migration. As
|
||||
such, it would be nice to set up appropriate webhooks to trigger
|
||||
rebuilding the appropriate web content when the files they are based
|
||||
on change instead of having to wait for, e.g., a cronjob to trigger.
|
||||
|
||||
Link web content back to files that it is generated from
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
It would be helpful for people who find issues with any of the
|
||||
documentation that is generated from a file to have a link on each
|
||||
page which points back to the file on GitHub [#github]_ that stores
|
||||
the content of the page. That would allow for quick pull requests to
|
||||
fix simple things such as spelling mistakes.
|
||||
|
||||
Splitting out parts of the documentation into their own repositories
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
While certain parts of the documentation at https://docs.python.org
|
||||
change with the code, other parts are fairly static and are not
|
||||
tightly bound to the CPython code itself. The following sections of
|
||||
the documentation fit this category of slow-changing,
|
||||
loosely-coupled:
|
||||
|
||||
* `Tutorial <https://docs.python.org/3/tutorial/index.html>`__
|
||||
* `Python Setup and Usage <https://docs.python.org/3/using/index.html>`__
|
||||
* `HOWTOs <https://docs.python.org/3/howto/index.html>`__
|
||||
* `Installing Python Modules <https://docs.python.org/3/installing/index.html>`__
|
||||
* `Distributing Python Modules <https://docs.python.org/3/distributing/index.html>`__
|
||||
* `Extending and Embedding <https://docs.python.org/3/extending/index.html>`__
|
||||
* `FAQs <https://docs.python.org/3/faq/index.html>`__
|
||||
|
||||
These parts of the documentation could be broken out into their own
|
||||
repositories to simplify their maintenance and to expand who has
|
||||
commit rights to them to ease in their maintenance.
|
||||
|
||||
Status
|
||||
======
|
||||
XXX list all sections (w/ backlinks) stating the status
|
||||
|
||||
Open Issues
|
||||
===========
|
||||
For this PEP, open issues are ones where a decision needs to be made
|
||||
to how to approach or solve a problem. Open issues do not entail
|
||||
coordination issues such as who is going to write a certain bit of
|
||||
code.
|
||||
|
||||
The fate of hg.python.org
|
||||
-------------------------
|
||||
With the code repositories moving over to Git [#git]_, there is no
|
||||
technical need to keep hg.python.org [#h.p.o]_ running. Having said
|
||||
that, some in the community would like to have it stay functioning as
|
||||
a Mercurial [#hg]_ mirror of the Git repositories. Others have said
|
||||
that they still want a mirror, but one using Git.
|
||||
|
||||
As maintaining hg.python.org is not necessary, it will be up to the
|
||||
PSF infrastructure committee to decide if they want to spend the
|
||||
time and resources to keep it running. They may also choose whether
|
||||
they want to host a Git mirror on PSF infrastructure.
|
||||
|
||||
Tools and commands to move from Mercurial to Git
|
||||
------------------------------------------------
|
||||
A decision needs to be made on exactly what tooling and what commands
|
||||
involving those tools will be used to convert a Mercurial repository
|
||||
to Git. Currently a suggestion has been made to use
|
||||
https://github.com/frej/fast-export. Another suggestion is to use
|
||||
https://github.com/felipec/git-remote-hg. Finally,
|
||||
http://hg-git.github.io/ has been suggested.
|
||||
|
||||
Git CLI commands for committing a pull request to cpython
|
||||
---------------------------------------------------------
|
||||
Because Git [#git]_ may be a new version control system for core
|
||||
developers, the commands people are expected to run will need to be
|
||||
written down. These commands also need to keep a linear history while
|
||||
giving proper attribution to the pull request author.
|
||||
|
||||
Another set of commands will also be necessary for when working with
|
||||
a patch file uploaded to bugs.python.org [#b.p.o]_. Here the linear
|
||||
history will be kept implicitly, but it will need to make sure to
|
||||
keep/add attribution.
|
||||
|
||||
How to handle the Misc/NEWS file
|
||||
--------------------------------
|
||||
There are two competing approaches to handling
|
||||
``Misc/NEWS`` [#news-file]_. One is to add a news entry for issues on
|
||||
bugs.python.org [#b.p.o]_. This would mean an issue that is marked
|
||||
as "resolved" could not be closed until a news entry is added in the
|
||||
"news" field in the issue tracker. The benefit of tying the news
|
||||
entry to the issue is it makes sure that all changes worthy of a news
|
||||
entry have an accompanying issue. It also makes classifying a news
|
||||
entry automatic thanks to the Component field of the issue. The
|
||||
Versions field of the issue also ties the news entry to which Python
|
||||
releases were affected. A script would be written to query
|
||||
bugs.python.org for relevant new entries for a release and to produce
|
||||
the output needed to be checked into the code repository. This
|
||||
approach is agnostic to whether a commit was done by CLI or bot.
|
||||
|
||||
The competing approach is to use an individual file per news entry,
|
||||
containg the text for the entry. In this scenario each feature
|
||||
release would have its own directory for news entries and a separate
|
||||
file would be created in that directory that was either named after
|
||||
the issue it closed or a timestamp value (which prevents collisions).
|
||||
Merges across branches would have no issue as the news entry file
|
||||
would still be uniqeuely named and in the directory of the latest
|
||||
version that contained the fix. A script would collect all news entry
|
||||
files no matter what directory they reside in and create an
|
||||
appropriate news file (the release directory can be ignored as the
|
||||
mere fact that the file exists is enough to represent that the entry
|
||||
belongs to the release). Classification can either be done by keyword
|
||||
in the new entry file itself or by using subdirectories representing
|
||||
each news entry classification in each release directory (or
|
||||
classification of news entries could be dropped since critical
|
||||
information is captured by the "What's New" documents which are
|
||||
organized). The benefit of this approach is that it keeps the changes
|
||||
with the code that was actually changed. It also ties the message to
|
||||
being part of the commit which introduced the change. For a commit
|
||||
made through the CLI, a script will be provided to help generate the
|
||||
file. In a bot-driven scenario, the merge bot will have a way to
|
||||
specify a specific news entry and create the file as part of its
|
||||
flattened commit (while most likely also supporting using the first
|
||||
line of the commit message if no specific news entry was specified).
|
||||
Code for this approach has been written previously for the Mercurial
|
||||
workflow at http://bugs.python.org/issue18967. There is also a tool
|
||||
from the community at https://pypi.python.org/pypi/towncrier.
|
||||
|
||||
Naming the commit bot
|
||||
---------------------
|
||||
As naming things can lead to bikeshedding of epic proportions, Brett
|
||||
Cannon will choose the final name of the commit bot (the name of the
|
||||
project for the bot itself can be anything, this is purely for the
|
||||
name used in giving commands to the bot). The name will come from
|
||||
Monty Python, which is only fitting since Python is named after the
|
||||
comedy troupe. It will most likely come from
|
||||
'Monty Python and the Holy Grail' [#holy-grail]_ (which happens to be
|
||||
how Brett was introduced to Monty Python). Current ideas on the name
|
||||
include:
|
||||
|
||||
"Black Knight" sketch [#black-knight-sketch]_:
|
||||
|
||||
* black-knight
|
||||
* none-shall-pass
|
||||
* just-a-flesh-wound
|
||||
|
||||
"Bridge of Death" sketch [#bridge-of-death-sketch]_:
|
||||
|
||||
* bridge-keeper
|
||||
* man-from-scene-24
|
||||
* five-questions
|
||||
* what-is-your-quest
|
||||
* blue-no-green
|
||||
* air-speed-velocity
|
||||
* your-favourite-colour
|
||||
(and that specific spelling; Monty Python is British, after all)
|
||||
|
||||
"Killer rabbit" sketch [#killer-rabbit-sketch]_:
|
||||
|
||||
* killer-rabbit
|
||||
* holy-hand-grenade
|
||||
* 5-is-right-out
|
||||
|
||||
"Witch Village" sketch [#witch-village-sketch]_:
|
||||
|
||||
* made-of-wood
|
||||
* burn-her
|
||||
|
||||
"French Taunter" sketch [#french-taunter-sketch]_:
|
||||
|
||||
* elderberries
|
||||
* kanigget
|
||||
|
||||
"Constitutional Peasants" sketch [#constitutional-peasants-sketch]_:
|
||||
|
||||
* dennis
|
||||
* from-the-masses
|
||||
* watery-tart
|
||||
|
||||
"Knights Who Say Ni" sketch [#ni-sketch]_:
|
||||
|
||||
* shubbery
|
||||
* ni
|
||||
|
||||
From "Monty Python and the Holy Grail" in general:
|
||||
|
||||
* brave-sir-robin
|
||||
|
||||
Choosing a CI service
|
||||
---------------------
|
||||
There are various CI services that provide free support for open
|
||||
source projects hosted on GitHub [#github]_. Two such examples are
|
||||
Travis [#travis]_ and Codeship [#codeship]_. Whatever solution is
|
||||
chosen will need to not time out in the time it takes to execute
|
||||
Python's test suite. It should optimally provide access to multiple C
|
||||
compilers for more thorough testing. Network access is also
|
||||
beneficial.
|
||||
|
||||
The current CI service for Python is Pypatcher [#pypatcher]_. A
|
||||
request can be made in IRC to try a patch from
|
||||
bugs.python.org [#b.p.o]_. The results can be viewed at
|
||||
https://ci.centos.org/job/cPython-build-patch/ .
|
||||
|
||||
Choosing a test coverage service
|
||||
--------------------------------
|
||||
Getting basic test coverage of Python's standard library can be
|
||||
created simply by using coverage.py [#coverage]_. Getting
|
||||
thorough test coverage is actually quite tricky, with the details
|
||||
outlined in the devinabox's README [#devinabox-repo]_. It would be
|
||||
best if a service could be found that would allow for thorough test
|
||||
coverage, but it might not be feasible.
|
||||
|
||||
Free test coverage services include Coveralls [#coveralls]_ and
|
||||
Codecov [#codecov]_.
|
||||
|
||||
Rejected Ideas
|
||||
==============
|
||||
Separate Python 2 and Python 3 repositories
|
||||
-------------------------------------------
|
||||
It was discussed whether separate repositories for Python 2 and
|
||||
Python 3 were desired. The thinking was that this would shrink the
|
||||
overall repository size which benefits people with slow Internet
|
||||
connections or small bandwidth caps.
|
||||
|
||||
In the end it was decided that it was easier logistically to simply
|
||||
keep all of CPython's history in a single repository.
|
||||
|
||||
Commit multi-release changes in bugfix branch first
|
||||
---------------------------------------------------
|
||||
As the current development process has changes committed in the
|
||||
oldest branch first and then merged up to the default branch, the
|
||||
question came up as to whether this workflow should be perpetuated.
|
||||
In the end it was decided that committing in the newest branch and
|
||||
then cherrypicking changes into older branches would work best as
|
||||
most people will instinctively work off the newest branch and it is a
|
||||
more common workflow when using Git [#git]_.
|
||||
|
||||
Deriving ``Misc/NEWS`` from the commit logs
|
||||
-------------------------------------------
|
||||
As part of the discussion surrounding `Handling Misc/NEWS`_, the
|
||||
suggestion has come up of deriving the file from the commit logs
|
||||
itself. In this scenario, the first line of a commit message would be
|
||||
taken to represent the news entry for the change. Some heuristic to
|
||||
tie in whether a change warranted a news entry would be used, e.g.,
|
||||
whether an issue number is listed.
|
||||
|
||||
This idea has been rejected due to some core developers preferring to
|
||||
write a news entry separate from the commit message. The argument is
|
||||
the first line of a commit message compared to that of a news entry
|
||||
have different requirements in terms of brevity, what should be said,
|
||||
etc.
|
||||
|
||||
References
|
||||
==========
|
||||
.. [#h.p.o] https://hg.python.org
|
||||
|
||||
.. [#GitHub] GitHub (https://github.com)
|
||||
|
||||
.. [#hg] Mercurial (https://www.mercurial-scm.org/)
|
||||
|
||||
.. [#git] Git (https://git-scm.com/)
|
||||
|
||||
.. [#b.p.o] https://bugs.python.org
|
||||
|
||||
.. [#rietveld] Rietveld (https://github.com/rietveld-codereview/rietveld)
|
||||
|
||||
.. [#gitlab] GitLab (https://about.gitlab.com/)
|
||||
|
||||
.. [#core-workflow] core-workflow mailing list (https://mail.python.org/mailman/listinfo/core-workflow)
|
||||
|
||||
.. [#guido-keynote] Guido van Rossum's keynote at PyCon US (https://www.youtube.com/watch?v=G-uKNd5TSBw)
|
||||
|
||||
.. [#reasons] Email to core-workflow outlining reasons why GitHub was selected
|
||||
(https://mail.python.org/pipermail/core-workflow/2016-January/000345.html)
|
||||
|
||||
.. [#benchmarks-repo] Mercurial repository for the Unified Benchmark Suite
|
||||
(https://hg.python.org/benchmarks/)
|
||||
|
||||
.. [#devinabox-repo] Mercurial repository for devinabox (https://hg.python.org/devinabox/)
|
||||
|
||||
.. [#peps-repo] Mercurial repository of the Python Enhancement Proposals (https://hg.python.org/peps/)
|
||||
|
||||
.. [#tracker-repo] bugs.python.org code repository (https://hg.python.org/tracker/python-dev/)
|
||||
|
||||
.. [#devguide-repo] Mercurial repository for the Python Developer's Guide (https://hg.python.org/devguide/)
|
||||
|
||||
.. [#cpython-repo] Mercurial repository for CPython (https://hg.python.org/cpython/)
|
||||
|
||||
.. [#github-python-org] Python organization on GitHub (https://github.com/python)
|
||||
|
||||
.. [#github-org-perms] GitHub repository permission levels
|
||||
(https://help.github.com/enterprise/2.4/user/articles/repository-permission-levels-for-an-organization/)
|
||||
|
||||
.. [#cla] Python Software Foundation Contributor Agreement (https://www.python.org/psf/contrib/contrib-form/)
|
||||
|
||||
.. [#news-file] ``Misc/NEWS`` (https://hg.python.org/cpython/file/default/Misc/NEWS)
|
||||
|
||||
.. [#acks-file] ``Misc/ACKS`` (https://hg.python.org/cpython/file/default/Misc/ACKS)
|
||||
|
||||
.. [#devguide-merge-across-branches] Devguide instructions on how to merge across branches
|
||||
(https://docs.python.org/devguide/committing.html#merging-between-different-branches-within-the-same-major-version)
|
||||
|
||||
.. [#pythocat] Pythocat (https://octodex.github.com/pythocat/)
|
||||
|
||||
.. [#tracker-plans] Wiki page for bugs.python.org feature development
|
||||
(https://wiki.python.org/moin/TrackerDevelopmentPlanning)
|
||||
|
||||
.. [#black-knight-sketch] The "Black Knight" sketch from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=dhRUe-gz690)
|
||||
|
||||
.. [#bridge-of-death-sketch] The "Bridge of Death" sketch from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=cV0tCphFMr8)
|
||||
|
||||
.. [#holy-grail] "Monty Python and the Holy Grail" sketches
|
||||
(https://www.youtube.com/playlist?list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp)
|
||||
|
||||
.. [#killer-rabbit-sketch] "Killer rabbit" sketch from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=Nvs5pqf-DMA&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=11)
|
||||
|
||||
.. [#witch-village-sketch] "Witch Village" from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=k3jt5ibfRzw&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=12)
|
||||
|
||||
.. [#french-taunter-sketch] "French Taunter" from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=A8yjNbcKkNY&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=13)
|
||||
|
||||
.. [#constitutional-peasants-sketch] "Constitutional Peasants" from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=JvKIWjnEPNY&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=14)
|
||||
|
||||
.. [#ni-sketch] "Knights Who Say Ni" from "Monty Python and the Holy Grail"
|
||||
(https://www.youtube.com/watch?v=zIV4poUZAQo&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=15)
|
||||
|
||||
.. [#homu] Homu (http://homu.io/)
|
||||
|
||||
.. [#zuul] Zuul (http://docs.openstack.org/infra/zuul/)
|
||||
|
||||
.. [#travis] Travis (https://travis-ci.org/)
|
||||
|
||||
.. [#codeship] Codeship (https://codeship.com/)
|
||||
|
||||
.. [#coverage] coverage.py (https://pypi.python.org/pypi/coverage)
|
||||
|
||||
.. [#coveralls] Coveralls (https://coveralls.io/)
|
||||
|
||||
.. [#codecov] Codecov (https://codecov.io/)
|
||||
|
||||
.. [#pypatcher] Pypatcher (https://github.com/kushaldas/pypatcher)
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
|
||||
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
Loading…
Reference in New Issue