2014-11-29 18:12:06 -05:00
|
|
|
|
PEP: 481
|
|
|
|
|
Title: Migrate Some Supporting Repositories to Git and Github
|
|
|
|
|
Version: $Revision$
|
|
|
|
|
Last-Modified: $Date$
|
|
|
|
|
Author: Donald Stufft <donald@stufft.io>
|
|
|
|
|
Status: Draft
|
|
|
|
|
Type: Process
|
|
|
|
|
Content-Type: text/x-rst
|
|
|
|
|
Created: 29-Nov-2014
|
|
|
|
|
Post-History: 29-Nov-2014
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Abstract
|
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
This PEP proposes migrating to Git and Github for certain supporting
|
|
|
|
|
repositories (such as the repository for Python Enhancement Proposals) in a way
|
|
|
|
|
that is more accessible to new contributors, and easier to manage for core
|
|
|
|
|
developers. This is offered as an alternative to PEP 474 which aims to achieve
|
|
|
|
|
the same overall benefits but while continuing to use the Mercurial DVCS and
|
|
|
|
|
without relying on a commerical entity.
|
|
|
|
|
|
|
|
|
|
In particular this PEP proposes changes to the following repositories:
|
|
|
|
|
|
|
|
|
|
* https://hg.python.org/devguide/
|
|
|
|
|
* https://hg.python.org/devinabox/
|
|
|
|
|
* https://hg.python.org/peps/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This PEP does not propose any changes to the core development workflow for
|
|
|
|
|
CPython itself.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rationale
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
As PEP 474 mentions, there are currently a number of repositories hosted on
|
|
|
|
|
hg.python.org which are not directly used for the development of CPython but
|
|
|
|
|
instead are supporting or ancillary repositories. These supporting repositories
|
|
|
|
|
do not typically have complex workflows or often branches at all other than the
|
|
|
|
|
primary integration branch. This simplicity makes them very good targets for
|
|
|
|
|
the "Pull Request" workflow that is commonly found on sites like Github.
|
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
However whereas PEP 474 proposes to continue to use Mercurial and restricts
|
|
|
|
|
itself to only solutions which are OSS and self-hosted, this PEP expands the
|
|
|
|
|
scope of that to include migrating to Git and using Github.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
The existing method of contributing to these repositories generally includes
|
|
|
|
|
generating a patch and either uploading them to bugs.python.org or emailing
|
|
|
|
|
them to peps@python.org. This process is unfriendly towards non-comitter
|
2014-11-29 19:21:23 -05:00
|
|
|
|
contributors as well as cumbersome for comitters seeking to accept the patches
|
|
|
|
|
sent by users. In contrast, the Pull Request workflow style enables non
|
2014-11-30 18:17:40 -05:00
|
|
|
|
technical contributors, especially those who do not know their way around the
|
2014-11-29 19:21:23 -05:00
|
|
|
|
DVCS of choice, to contribute using the web based editor. On the committer
|
|
|
|
|
side, the Pull Requests enable them to tell, before merging, whether or not
|
2014-11-29 18:12:06 -05:00
|
|
|
|
a particular Pull Request will break anything. It also enables them to do a
|
|
|
|
|
simple "push button" merge which does not require them to check out the
|
|
|
|
|
changes locally. Another such feature that is useful in particular for docs,
|
2014-11-29 19:21:23 -05:00
|
|
|
|
is the ability to view a "prose" diff. This Github-specific feature enables
|
2014-11-29 18:12:06 -05:00
|
|
|
|
a committer to view a diff of the rendered output which will hide things like
|
|
|
|
|
reformatting a paragraph and show you what the actual "meat" of the change
|
|
|
|
|
actually is.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Why Git?
|
|
|
|
|
--------
|
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
Looking at the variety of DVCS which are available today, it becomes fairly
|
2014-11-30 18:17:40 -05:00
|
|
|
|
clear that git has the largest mindshare. The Open Hub (previously Ohloh)
|
2014-11-29 19:21:23 -05:00
|
|
|
|
statistics [#openhub-stats]_ show that currently 37% of the repositories
|
|
|
|
|
indexed by Open Hub are using git which is second only to SVN (which has 48%),
|
|
|
|
|
while Mercurial has just 2% of the indexed repositories (beating only bazaar
|
|
|
|
|
which has 1%). In additon to the Open Hub statistics, a look at the top 100
|
|
|
|
|
projects on PyPI (ordered by total download counts) shows that within the
|
|
|
|
|
Python space itself, the majority of projects use git.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
=== ========= ========== ====== === ====
|
|
|
|
|
Git Mercurial Subversion Bazaar CVS None
|
|
|
|
|
=== ========= ========== ====== === ====
|
|
|
|
|
62 22 7 4 1 1
|
|
|
|
|
=== ========= ========== ====== === ====
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Chosing a DVCS which has the larger mindshare will make it more likely that any
|
|
|
|
|
particular person who has experience with DVCS at all will be able to
|
2014-11-29 19:21:23 -05:00
|
|
|
|
meaningfully contribute without having to learn a new tool.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
In addition to simply making it more likely that any individual will already
|
|
|
|
|
know how to use git, the number of projects and people using it means that the
|
2014-11-29 19:21:23 -05:00
|
|
|
|
resources for learning the tool are likely to be more fully fleshed out.
|
2014-11-30 18:17:40 -05:00
|
|
|
|
When you run into problems, the likelihood that someone else had that problem
|
|
|
|
|
and posted a question and recieved an answer is also far higher.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
Thirdly, by using a more popular tool you also increase your options for
|
|
|
|
|
tooling *around* the DVCS itself. Looking at the various options for hosting
|
|
|
|
|
repositories, it's extremely rare to find a hosting solution (whether OSS or
|
|
|
|
|
commerical) that supports Mercurial but does not support Git. On the flip side,
|
2014-11-29 18:12:06 -05:00
|
|
|
|
there are a number of tools which support Git but do not support Mercurial.
|
|
|
|
|
Therefore the popularity of git increases the flexibility of our options going
|
|
|
|
|
into the future for what toolchain these projects use.
|
|
|
|
|
|
2014-11-30 18:17:40 -05:00
|
|
|
|
Also, by moving to the more popular DVCS, we increase the likelihood that the
|
2014-11-29 18:12:06 -05:00
|
|
|
|
knowledge that the person has learned in contributing to these support
|
|
|
|
|
repositories will transfer to projects outside of the immediate CPython project
|
|
|
|
|
such as to the larger Python community which is primarily using Git hosted on
|
|
|
|
|
Github.
|
|
|
|
|
|
|
|
|
|
In previous years there was concern about how well supported git was on Windows
|
2014-11-29 19:21:23 -05:00
|
|
|
|
in comparison to Mercurial. However, git has grown to support Windows as a
|
|
|
|
|
first class citizen. In addition to that, for Windows users who are not well
|
2014-11-30 18:17:40 -05:00
|
|
|
|
acquainted with the Windows command line, there are GUI options as well.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Why Github?
|
|
|
|
|
-----------
|
|
|
|
|
|
|
|
|
|
There are a number of software projects or web services which offer
|
|
|
|
|
functionality similar to that of Github. These range from commerical web
|
2014-11-30 18:17:40 -05:00
|
|
|
|
services such as Bitbucket to self-hosted OSS solutions such as Kallithea or
|
2014-11-29 18:12:06 -05:00
|
|
|
|
Gitlab. This PEP proposes that we move these repositories to Github.
|
|
|
|
|
|
|
|
|
|
There are two primary reasons for selecting Github: Popularity and
|
|
|
|
|
Quality/Polish.
|
|
|
|
|
|
|
|
|
|
Github is currently the most popular hosted repository hosting according to
|
2014-11-29 19:21:23 -05:00
|
|
|
|
Alexa, where it currently has a global rank of 121. Much like for Git itself,
|
2014-11-30 18:17:40 -05:00
|
|
|
|
by choosing the most popular tool we gain benefits in increasing the likelihood
|
2014-11-29 18:12:06 -05:00
|
|
|
|
that a new contributor will have already experienced the toolchain, the quality
|
|
|
|
|
and availablity of the help, more and better tooling being built around it, and
|
|
|
|
|
the knowledge transfer to other projects. A look again at the top 100 projects
|
|
|
|
|
by download counts on PyPI shows the following hosting locations:
|
|
|
|
|
|
|
|
|
|
====== ========= =========== ========= =========== ==========
|
|
|
|
|
GitHub BitBucket Google Code Launchpad SourceForge Other/Self
|
|
|
|
|
====== ========= =========== ========= =========== ==========
|
|
|
|
|
62 18 6 4 3 7
|
|
|
|
|
====== ========= =========== ========= =========== ==========
|
|
|
|
|
|
|
|
|
|
In addition to all of those reasons, Github also has the benefit that while
|
|
|
|
|
many of the options have similar features when you look at them in a feature
|
|
|
|
|
matrix the Github version of each of those features tend to work better and be
|
|
|
|
|
far more polished. This is hard to quantify objectively however it is a fairly
|
|
|
|
|
common sentiment if you go around and ask people who are using these services
|
|
|
|
|
often.
|
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
Finally, a reason to choose a web service at all over something that is
|
2014-11-29 18:12:06 -05:00
|
|
|
|
self-hosted is to be able to more efficiently use volunteer time and donated
|
2014-11-30 18:17:40 -05:00
|
|
|
|
resources. Every additional service hosted on the PSF infrastructure by the
|
2014-11-29 18:12:06 -05:00
|
|
|
|
PSF infrastructure team further spreads out the amount of time that the
|
|
|
|
|
volunteers on that team have to spend and uses some chunk of resources that
|
2014-11-30 18:17:40 -05:00
|
|
|
|
could potentially be used for something where there is no free or affordable
|
2014-11-29 18:12:06 -05:00
|
|
|
|
hosted solution available.
|
|
|
|
|
|
|
|
|
|
One concern that people do have with using a hosted service is that there is a
|
|
|
|
|
lack of control and that at some point in the future the service may no longer
|
|
|
|
|
be suitable. It is the opinion of this PEP that Github does not currently and
|
|
|
|
|
has not in the past engaged in any attempts to lock people into their platform
|
|
|
|
|
and that if at some point in the future Github is no longer suitable for one
|
2014-11-29 19:21:23 -05:00
|
|
|
|
reason or another, then at that point we can look at migrating away from Github
|
2014-11-29 18:12:06 -05:00
|
|
|
|
onto a different solution. In other words, we'll cross that bridge if and when
|
|
|
|
|
we come to it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Example: Scientific Python
|
|
|
|
|
--------------------------
|
|
|
|
|
|
|
|
|
|
One of the key ideas behind the move to both git and Github is that a feature
|
|
|
|
|
of a DVCS, the repository hosting, and the workflow used is the social network
|
|
|
|
|
and size of the community using said tools. We can see this is true by looking
|
|
|
|
|
at an example from a sub-community of the Python community: The Scientific
|
|
|
|
|
Python community. They have already migrated most of the key pieces of the
|
2014-11-29 19:21:23 -05:00
|
|
|
|
SciPy stack onto Github using the Pull Request based workflow. This process
|
|
|
|
|
started with IPython, and as more projects moved over it became a natural
|
|
|
|
|
default for new projects in the community.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
They claim to have seen a great benefit from this move, in that it enables
|
|
|
|
|
casual contributors to easily move between different projects within their
|
2014-11-29 18:12:06 -05:00
|
|
|
|
sub-community without having to learn a special, bespoke workflow and a
|
|
|
|
|
different toolchain for each project. They've found that when people can use
|
|
|
|
|
their limited time on actually contributing instead of learning the different
|
2014-11-30 18:17:40 -05:00
|
|
|
|
tools and workflows, not only do they contribute more to one project, but
|
2014-11-29 19:21:23 -05:00
|
|
|
|
that they also expand out and contribute to other projects. This move has also
|
|
|
|
|
been attributed to the increased tendency for members of that community to go
|
|
|
|
|
so far as publishing their research and educational materials on Github as
|
|
|
|
|
well.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
This example showcases the real power behind moving to a highly popular
|
|
|
|
|
toolchain and workflow, as each variance introduces yet another hurdle for new
|
|
|
|
|
and casual contributors to get past and it makes the time spent learning that
|
|
|
|
|
workflow less reusable with other projects.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Migration
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
Through the use of hg-git [#hg-git]_ we can easily convert a Mercurial
|
|
|
|
|
repository to a Git repository by simply pushing the Mercurial repository to
|
2014-11-30 18:17:40 -05:00
|
|
|
|
the Git repository. People who wish to continue to use Mercurial locally can
|
2014-11-29 19:21:23 -05:00
|
|
|
|
then use hg-git going into the future using the new Github URL. However they
|
2014-11-29 18:12:06 -05:00
|
|
|
|
will need to re-clone their repositories as using Git as the server seems to
|
|
|
|
|
trigger a one time change of the changeset ids.
|
|
|
|
|
|
|
|
|
|
As none of the selected repositories have any tags, branches, or bookmarks
|
|
|
|
|
other than the ``default`` branch the migration will simply map the ``default``
|
|
|
|
|
branch in Mercurial to the ``master`` branch in git.
|
|
|
|
|
|
2014-11-29 19:21:23 -05:00
|
|
|
|
In addition, since none of the selected projects have any great need of a
|
2014-11-29 18:12:06 -05:00
|
|
|
|
complex bug tracker, they will also migrate their issue handling to using the
|
|
|
|
|
GitHub issues.
|
|
|
|
|
|
|
|
|
|
In addition to the migration of the repository hosting itself there are a
|
|
|
|
|
number of locations for each particular repository which will require updating.
|
2014-11-30 18:17:40 -05:00
|
|
|
|
The bulk of these will simply be changing commands from the hg equivalent to
|
|
|
|
|
the git equivalent.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
In particular this will include:
|
|
|
|
|
|
|
|
|
|
* Updating www.python.org to generate PEPs using a git clone and link to
|
|
|
|
|
Github.
|
|
|
|
|
* Updating docs.python.org to pull from Github instead of hg.python.org for the
|
|
|
|
|
devguide.
|
|
|
|
|
* Enabling the ability to send an email to python-checkins@python.org for each
|
|
|
|
|
push.
|
|
|
|
|
* Enabling the ability to send an IRC message to #python-dev on Freenode for
|
|
|
|
|
each push.
|
|
|
|
|
* Migrate any issues for these projects to their respective bug tracker on
|
|
|
|
|
Github.
|
2014-11-30 12:27:06 -05:00
|
|
|
|
* Use hg-git to provide a read-only mirror on hg.python.org which will enable
|
|
|
|
|
read-only uses of the hg.python.org instances of the specified repositories
|
|
|
|
|
to remain the same.
|
2014-11-29 18:12:06 -05:00
|
|
|
|
|
|
|
|
|
This will restore these repositories to similar functionality as they currently
|
|
|
|
|
have. In addition to this the migration will also include enabling testing for
|
|
|
|
|
each pull request using Travis CI [#travisci]_ where possible to ensure that
|
|
|
|
|
a new PR does not break the ability to render the documentation or PEPs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
User Access
|
|
|
|
|
===========
|
|
|
|
|
|
|
|
|
|
Moving to Github would involve adding an additional user account that will need
|
|
|
|
|
to be managed, however it also offers finer grained control, allowing the
|
|
|
|
|
ability to grant someone access to only one particular repository instead of
|
|
|
|
|
the coarser grained ACLs available on hg.python.org.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
References
|
|
|
|
|
==========
|
|
|
|
|
|
|
|
|
|
.. [#openhub-stats] `Open Hub Statistics <https://www.openhub.net/repositories/compare>`
|
|
|
|
|
.. [#hg-git] `hg-git <https://hg-git.github.io/>`
|
|
|
|
|
.. [#travisci] `Travis CI <https://travis-ci.org/>`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Copyright
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
..
|
|
|
|
|
Local Variables:
|
|
|
|
|
mode: indented-text
|
|
|
|
|
indent-tabs-mode: nil
|
|
|
|
|
sentence-end-double-space: t
|
|
|
|
|
fill-column: 70
|
|
|
|
|
coding: utf-8
|
|
|
|
|
End:
|