iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update status of GH migration

This commit is contained in:
Brett Cannon 2016-05-08 13:46:14 -07:00
parent b08220c867
commit b529907818
1 changed files with 93 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

98
pep-0512.txt
View File

@ -12,6 +12,7 @@ Post-History: 17-Jan-2016, 19-Jan-2016, 23-Jan-2016
Abstract
========
This PEP outlines the steps required to migrate Python's development
process from Mercurial [#hg]_ as hosted at
hg.python.org [#h.p.o]_ to Git [#git]_ on GitHub [#GitHub]_. Meeting
@ -19,8 +20,10 @@ the minimum goals of this PEP should allow for the development
process of Python to be as productive as it currently is, and meeting
its extended goals should improve it.
Rationale
=========
In 2014, it became obvious that Python's custom development
process was becoming a hindrance. As an example, for an external
contributor to submit a fix for a bug that eventually was committed,
@ -104,8 +107,10 @@ GitHub's default workflow). All of this will be done in such a way
that if an external contributor chooses not to use GitHub then they
will continue to have that option.
Repositories to Migrate
=======================
While hg.python.org [#h.p.o]_ hosts many repositories, there are only
five key repositories that should move:
@ -122,6 +127,7 @@ with bugs.python.org [#b.p.o]_.
Migration Plan
==============
The migration plan is separated into sections based on what is
required to migrate the repositories listed in the
`Repositories to Migrate`_ section. Completion of requirements
@ -129,16 +135,20 @@ outlined in each section should unblock the migration of the related
repositories. The sections are expected to be completed in order, but
not necessarily the requirements within a section.
Requirements for Code-Only Repositories
---------------------------------------
Completion of the requirements in this section will allow the
devinabox and benchmarks repositories to move to
GitHub. While devinabox has a sufficiently descriptive name, the
benchmarks repository does not; therefore, it will be named
"python-benchmarks".
Create a 'Python core' team
'''''''''''''''''''''''''''
To manage permissions, a 'Python core' team will be created as part of
the python organization [#github-python-org]_. Any repository that is
moved will have the 'Python core' team added to it with write
@ -146,15 +156,19 @@ permissions [#github-org-perms]_. Anyone who previously had rights to
manage SSH keys on hg.python.org will become a team maintainer for the
'Python core' team.
Define commands to move a Mercurial repository to Git
'''''''''''''''''''''''''''''''''''''''''''''''''''''
Since moving to GitHub also entails moving to Git [#git]_, we must
decide what tools and commands we will run to translate a Mercurial
repository to Git. The exact tools and steps to use are an
open issue; see `Tools and commands to move from Mercurial to Git`_.
CLA enforcement
'''''''''''''''
A key part of any open source project is making sure that its source
code can be properly licensed. This requires making sure all people
making contributions have signed a contributor license agreement
@ -165,8 +179,10 @@ bugs.python.org [#b.p.o]_. With this migration, the plan is to start
off with automated checking and enforcement of contributors signing
the CLA.
Adding GitHub username support to bugs.python.org
+++++++++++++++++++++++++++++++++++++++++++++++++
To keep tracking of CLA signing under the direct control of the PSF,
tracking who has signed the PSF CLA will be continued by marking that
fact as part of someone's bugs.python.org user profile. What this
@ -185,8 +201,10 @@ returns a JSON dictionary with the keys of the usernames requested
and a ``true`` value if they have sigend the CLA, ``false`` if they
have not, and ``null`` if no corresponding GitHub username was found.
A bot to enforce CLA signing
++++++++++++++++++++++++++++
With an association between someone's GitHub account and their
bugs.python.org [#b.p.o]_ account, which has the data as to whether
someone has signed the CLA, a bot can monitor pull requests on
@ -211,8 +229,10 @@ Heroku [#heroku]_ and written to target Python 3.5 to act as a
showcase for asynchronous programming. The code for the bot is hosted
in the Knights Who Say Ni project [#ni]_.
Requirements for Web-Related Repositories
-----------------------------------------
Due to their use for generating webpages, the
devguide [#devguide-repo]_ and peps [#peps-repo]_ repositories need
their respective processes updated to pull from their new Git
@ -223,16 +243,20 @@ The devguide repository might also need to be named
when viewed in isolation from the
python organization [#github-python-org]_.
Requirements for the cpython Repository
---------------------------------------
Obviously the most active and important repository currently hosted
at hg.python.org [#h.p.o]_ is the cpython
repository [#cpython-repo]_. Because of its importance and high-
frequency use, it requires more tooling before being moved to GitHub
compared to the other repositories mentioned in this PEP.
Document steps to commit a pull request
'''''''''''''''''''''''''''''''''''''''
During the process of choosing a new development workflow, it was
decided that a linear history is desired. People preferred having a
single commit representing a single change instead of having a set of
@ -262,8 +286,10 @@ The exact sequence of commands that will be given as guidelines to
core developers is an open issue:
`Git CLI commands for committing a pull request to cpython`_.
Handling Misc/NEWS
''''''''''''''''''
Traditionally the ``Misc/NEWS`` file [#news-file]_ has been problematic
for changes which spanned Python releases. Often times there will be
merge conflicts when committing a change between e.g., 3.5 and 3.6
@ -278,8 +304,10 @@ There are currently two competing approaches to solving the
``Misc/NEWS`` problem which are discussed in an open issue:
`How to handle the Misc/NEWS file`_.
Handling Misc/ACKS
''''''''''''''''''
Traditionally the ``Misc/ACKS`` file [#acks-file]_ has been managed
by hand. But thanks to Git supporting an ``author`` value as well as
a ``committer`` value per commit, authorship of a commit can be part
@ -305,8 +333,10 @@ issue number in the commit message of the format ``Issue #`` at the
start of the message led to a comment being posted to the issue
linking to the commit.
Linking a pull request to an issue
++++++++++++++++++++++++++++++++++
An association between a pull request and an issue is needed to track
when a fix has been proposed. The association needs to be many-to-one
as there can take multiple pull requests to solve a single issue
@ -324,13 +354,17 @@ request to signify that the connection was made successfully. This
could lead to incorrect associations if the wrong issue or
referencing another issue was done, but these are rare occasions.
Notify the issue if the pull request is committed
+++++++++++++++++++++++++++++++++++++++++++++++++
Once a pull request is closed (merged or not), the issue should be
updated to reflect this fact.
Update linking service for mapping commit IDs to URLs
'''''''''''''''''''''''''''''''''''''''''''''''''''''
Currently you can use https://hg.python.org/lookup/ with a revision
ID from either the Subversion or Mercurial copies of the
cpython repo [#cpython-repo]_ to get redirected to the URL for that
@ -338,14 +372,18 @@ revision in the Mercurial repository. The URL rewriter will need to
be updated to redirect to the Git repository and to support the new
revision IDs created for the Git repository.
Create https://git.python.org
'''''''''''''''''''''''''''''
Just as hg.python.org [#h.p.o]_ currently points to the Mercurial
repository for Python, git.python.org should do the equivalent for
the Git repository.
Backup of pull request data
'''''''''''''''''''''''''''
Since GitHub [#github]_ is going to be used for code hosting and code
review, those two things need to be backed up. In the case of code
hosting, the backup is implicit as all non-shallow Git [#git]_ clones
@ -359,25 +397,33 @@ issues with GitHub. It also helps guarantee that a migration from
GitHub to some other code review system is feasible were GitHub to
disappear overnight.
Deprecate sys._mercurial
''''''''''''''''''''''''
Once Python is no longer kept in Mercurial, the ``sys._mercurial``
attribute will need to be changed to return ``('CPython', '', '')``.
An equivalent ``sys._git`` attribute will be added which fulfills the
same use-cases.
Update the devguide
'''''''''''''''''''
The devguide will need to be updated with details of the new
workflow. Mostly likely work will take place in a separate branch
until the migration actually occurs.
Update PEP 101
''''''''''''''
The release process will need to be updated as necessary.
Optional, Planned Features
--------------------------
Once the cpython repository [#cpython-repo]_ is migrated, all
repositories will have been moved to GitHub [#github]_ and the
development process should be on equal footing as before. But a key
@ -389,8 +435,10 @@ It should be mentioned that overall feature planning for
bugs.python.org [#b.p.o]_ -- which includes plans independent of this
migration -- are tracked on their own wiki page [#tracker-plans]_.
Bot to handle pull request merging
''''''''''''''''''''''''''''''''''
As stated in the section entitled
"`Document steps to commit a pull request`_", the desire is to
maintain a linear history for cpython. Unfortunately,
@ -419,8 +467,10 @@ such as Homu [#homu]_ or Zuul [#zuul]_.
The name given to this bot in order to give it commands is an open
issue: `Naming the bots`_.
Continuous integration per pull request
'''''''''''''''''''''''''''''''''''''''
To help speed up pull request approvals, continuous integration
testing should be used. This helps mitigate the need for a core
developer to download a patch simply to run the test suite against
@ -429,8 +479,10 @@ the patch.
Which free CI service to use is an open issue:
`Choosing a CI service`_.
Test coverage report
''''''''''''''''''''
Getting an up-to-date test coverage report for Python's standard
library would be extremely beneficial as generating such a report can
take quite a while to produce.
@ -439,8 +491,10 @@ There are a couple pre-existing services that provide free test
coverage for open source projects. Which option is best is an open
issue: `Choosing a test coverage service`_.
Notifying issues of pull request comments
'''''''''''''''''''''''''''''''''''''''''
The current development process does not include notifying an issue
on bugs.python.org [#b.p.o]_ when a review comment is left on
Rietveld [#rietveld]_. It would be nice to fix this so that people
@ -454,14 +508,18 @@ down for those that receive both GitHub and bugs.python.org email
notifications while still making sure that those only following
bugs.python.org know when there might be a review comment to address.
Allow bugs.python.org to use GitHub as a login provider
'''''''''''''''''''''''''''''''''''''''''''''''''''''''
As of right now, bugs.python.org [#b.p.o]_ allows people to log in
using Google, Launchpad, or OpenID credentials. It would be good to
expand this to GitHub credentials.
Web hooks for re-generating web content
'''''''''''''''''''''''''''''''''''''''
The content at https://docs.python.org/,
https://docs.python.org/devguide, and
https://www.python.org/dev/peps/ are all derived from files kept in
@ -470,16 +528,20 @@ such, it would be nice to set up appropriate webhooks to trigger
rebuilding the appropriate web content when the files they are based
on change instead of having to wait for, e.g., a cronjob to trigger.
Link web content back to files that it is generated from
''''''''''''''''''''''''''''''''''''''''''''''''''''''''
It would be helpful for people who find issues with any of the
documentation that is generated from a file to have a link on each
page which points back to the file on GitHub [#github]_ that stores
the content of the page. That would allow for quick pull requests to
fix simple things such as spelling mistakes.
Splitting out parts of the documentation into their own repositories
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
While certain parts of the documentation at https://docs.python.org
change with the code, other parts are fairly static and are not
tightly bound to the CPython code itself. The following sections of
@ -505,15 +567,19 @@ developed where it would be difficult to forget to update
What's New (potentially through a label added to PRs, like
"What's New needed").
Backup of Git repositories
''''''''''''''''''''''''''
While not necessary, it would be good to have official backups of the
various Git repositories for disaster protection. It will be up to
the PSF infrastructure committee to decide if this is worthwhile or
unnecessary.
Identify potential new core developers
''''''''''''''''''''''''''''''''''''''
The Python development team has long-standing guidelines for
selecting new core developers. The key part of the guidelines is that
a person needs to have contributed multiple patches which have been
@ -528,14 +594,10 @@ filled in properly.
Status
======
Requirements for migrating the devinabox [#devinabox-repo]_ and
benchmarks [#benchmarks-repo]_ repositories:
* In progress
- `Define commands to move a Mercurial repository to Git`_:
https://github.com/orsenthil/cpython-hg-to-git (Senthil Kumaran)
* Completed
- `Adding GitHub username support to bugs.python.org`_
@ -543,6 +605,8 @@ benchmarks [#benchmarks-repo]_ repositories:
- `A bot to enforce CLA signing`_:
https://github.com/python/the-knights-who-say-ni (Brett Cannon)
- `Create a 'Python core' team`_
- `Define commands to move a Mercurial repository to Git`_:
https://github.com/orsenthil/cpython-hg-to-git (Senthil Kumaran)
Repositories whose build steps need updating:
@ -606,15 +670,19 @@ Optional features:
- None
Open Issues
===========
For this PEP, open issues are ones where a decision needs to be made
to how to approach or solve a problem. Open issues do not entail
coordination issues such as who is going to write a certain bit of
code.
The fate of hg.python.org
-------------------------
With the code repositories moving over to Git [#git]_, there is no
technical need to keep hg.python.org [#h.p.o]_ running. Having said
that, some in the community would like to have it stay functioning as
@ -630,8 +698,10 @@ Depending on the decision reached, other ancillary repositories will
either be forced to migration or they can choose to simply stay on
hg.python.org.
Tools and commands to move from Mercurial to Git
------------------------------------------------
A decision needs to be made on exactly what tooling and what commands
involving those tools will be used to convert a Mercurial repository
to Git. Currently a suggestion has been made to use
@ -639,8 +709,10 @@ https://github.com/frej/fast-export. Another suggestion is to use
https://github.com/felipec/git-remote-hg. Finally,
http://hg-git.github.io/ has been suggested.
Git CLI commands for committing a pull request to cpython
---------------------------------------------------------
Because Git [#git]_ may be a new version control system for core
developers, the commands people are expected to run will need to be
written down. These commands also need to keep a linear history while
@ -651,8 +723,10 @@ a patch file uploaded to bugs.python.org [#b.p.o]_. Here the linear
history will be kept implicitly, but it will need to make sure to
keep/add attribution.
How to handle the Misc/NEWS file
--------------------------------
There are three competing approaches to handling
``Misc/NEWS`` [#news-file]_. One is to add a news entry for issues on
bugs.python.org [#b.p.o]_. This would mean an issue that is marked
@ -701,8 +775,10 @@ approach allows for keeping the NEWS file as a single file. It does
run the risk, though, of failure and thus blocking a commit until it
can be manually resolved.
Naming the bots
---------------
As naming things can lead to bikeshedding of epic proportions, Brett
Cannon will choose the final name of the various bots (the name of
the project for the bots themselves can be anything, this is purely
@ -713,6 +789,7 @@ Python is named after the comedy troupe.
Choosing a CI service
---------------------
There are various CI services that provide free support for open
source projects hosted on GitHub [#github]_. Two such examples are
Travis [#travis]_ and Codeship [#codeship]_. Whatever solution is
@ -726,8 +803,10 @@ request can be made in IRC to try a patch from
bugs.python.org [#b.p.o]_. The results can be viewed at
https://ci.centos.org/job/cPython-build-patch/ .
Choosing a test coverage service
--------------------------------
Getting basic test coverage of Python's standard library can be
created simply by using coverage.py [#coverage]_. Getting
thorough test coverage is actually quite tricky, with the details
@ -738,10 +817,13 @@ coverage, but it might not be feasible.
Free test coverage services include Coveralls [#coveralls]_ and
Codecov [#codecov]_.
Rejected Ideas
==============
Separate Python 2 and Python 3 repositories
-------------------------------------------
It was discussed whether separate repositories for Python 2 and
Python 3 were desired. The thinking was that this would shrink the
overall repository size which benefits people with slow Internet
@ -750,8 +832,10 @@ connections or small bandwidth caps.
In the end it was decided that it was easier logistically to simply
keep all of CPython's history in a single repository.
Commit multi-release changes in bugfix branch first
---------------------------------------------------
As the current development process has changes committed in the
oldest branch first and then merged up to the default branch, the
question came up as to whether this workflow should be perpetuated.
@ -776,8 +860,10 @@ older branch, possibly push to another clone representing the
Cherry-picking should decouple this so that you don't have to rush
your multi-branch changes as the cherry-pick can be done separately.
Deriving ``Misc/NEWS`` from the commit logs
-------------------------------------------
As part of the discussion surrounding `Handling Misc/NEWS`_, the
suggestion has come up of deriving the file from the commit logs
itself. In this scenario, the first line of a commit message would be
@ -793,6 +879,7 @@ etc.
References
==========
.. [#h.p.o] https://hg.python.org
.. [#GitHub] GitHub (https://github.com)
@ -889,6 +976,7 @@ References
.. [#ni] The Knights Who Say Ni project
(https://github.com/python/the-knights-who-say-ni)
Copyright
=========