Split software forge proposal into a new PEP
- moved the software forge proposal out to its own PEP - switched forge proposal to Kallithea based on SFC concerns with the current RhodeCode licensing structure
This commit is contained in:
Nick Coghlan
parent
89ae8bb813
commit
209a860d13
161
pep-0462.txt
161
pep-0462.txt
|
|
@ -21,11 +21,6 @@ available to contribute to CPython, which should also result in an improved
|
|||
experience for other contributors that are reliant on the core team to get
|
||||
their changes incorporated.
|
||||
|
||||
This PEP also proposes changes to the way certain supporting repositories
|
||||
(such as the repository for Python Enhancement Proposals) are managed to
|
||||
make them more accessible to new contributors, and easier to manage for
|
||||
core developers.
|
||||
|
||||
|
||||
Rationale for changes to the core development workflow
|
||||
======================================================
|
||||
|
|
@ -158,23 +153,6 @@ incorporation doesn't score well on that front for anyone:
|
|||
the case for OpenStack).
|
||||
|
||||
|
||||
Rationale for changes to source code repository management
|
||||
==========================================================
|
||||
|
||||
Currently, hg.python.org hosts more than just the core CPython repository,
|
||||
it also hosts other repositories such as those for the CPython developer
|
||||
guide and for Python Enhancement Proposals, along with various "sandbox"
|
||||
repositories for core developer experimentation.
|
||||
|
||||
These supporting repositories could benefit greatly from user the simple
|
||||
"pull request" style workflow made popular by code hosting sites like
|
||||
GitHub and BitBucket.
|
||||
|
||||
This PEP proposes introducing a more sophisticated approach to repository
|
||||
management that includes more "self service" features, including support
|
||||
for pull requests.
|
||||
|
||||
|
||||
Current Tools
|
||||
=============
|
||||
|
||||
|
|
@ -198,53 +176,8 @@ tools to see which, if any, may be candidates for replacement.
|
|||
Proposal
|
||||
========
|
||||
|
||||
This proposal consists of two key components:
|
||||
|
||||
* Introducing RhodeCode for improved repository management
|
||||
* Introducing automated merge gating for the CPython project
|
||||
|
||||
|
||||
Improved repository management
|
||||
------------------------------
|
||||
|
||||
The rise of free source code hosting sites like GitHub and BitBucket with a
|
||||
focus on improving the user experience have increased user expectations
|
||||
for the web based interface offered for source code repositories. This
|
||||
includes features like dynamic control of user permissions for repository
|
||||
administrators, online editing for documentation updates and similar small
|
||||
fixes, easy cloning and branching, etc.
|
||||
|
||||
While GitHub and BitBucket are proprietary solutions to this problem that
|
||||
cannot be self-hosted, RhodeCode_ is a popular primarily open source solution
|
||||
that offers similar features to the major free hosting sites, while also
|
||||
allowing installation on your own servers. RhodeCode is also implemented
|
||||
in Python, allowing us to preserve our current situation of having our
|
||||
workflow tools being almost entirely self-hosting.
|
||||
|
||||
There's nothing too complicated with this part of the proposal: it is
|
||||
essentially just a suggestion to convert hg.python.org to being backed
|
||||
by a RhodeCode Enterprise instance hosted on PSF infrastructure.
|
||||
|
||||
All of the functional parts of RhodeCode Enterprise are open source under
|
||||
the GPLv3 license. The "look & feel" components are licensed under a
|
||||
`business source license`_ that is free for up to 20 users, as well as for
|
||||
organisations that qualify for one of RhodeCode's exemption categories
|
||||
(in the case of ``hg.python.org``, the relevant category is "public open
|
||||
source project").
|
||||
|
||||
The RhodeCode developers have also expressed interest in helping out with
|
||||
ensuring a python.org RhodeCode deployment is successful.
|
||||
|
||||
.. _RhodeCode: https://rhodecode.com/
|
||||
.. _business source license: https://rhodecode.com/licenses
|
||||
|
||||
|
||||
Automated merge gating for CPython
|
||||
----------------------------------
|
||||
|
||||
The essence of this part of the proposal is that CPython aim to adopt a
|
||||
"core reviewer" development model, similar to that used by the OpenStack
|
||||
project.
|
||||
The essence of this proposal is that CPython aim to adopt a "core reviewer"
|
||||
development model, similar to that used by the OpenStack project.
|
||||
|
||||
The workflow problems experienced by the CPython core development team are
|
||||
not unique. The OpenStack infrastructure team have come up with a well
|
||||
|
|
@ -255,7 +188,7 @@ designed automated workflow that is designed to ensure:
|
|||
* patches never get merged without being tested relative to the current
|
||||
state of the branch
|
||||
* the main development branch always stays green. Patches that do not pass
|
||||
the automated tests do not get merged.
|
||||
the automated tests do not get merged
|
||||
|
||||
If a core developer wants to tweak a patch prior to merging, they download
|
||||
it from the review tool, modify and *upload it back to the review tool*
|
||||
|
|
@ -360,25 +293,6 @@ manage them using the simpler pull request based model.
|
|||
Perceived Benefits
|
||||
==================
|
||||
|
||||
Improved repository management
|
||||
------------------------------
|
||||
|
||||
The primary benefit of using RhodeCode to manage hg.python.org is that
|
||||
supporting repositories such as the developer guide and the PEP repo could
|
||||
now be managed using pull requests and online editing. This would be
|
||||
much simpler then the current workflow which requires PEP editors and
|
||||
other core developers to act as intermediaries to apply updates suggested
|
||||
by other users.
|
||||
|
||||
The richer administrative functionality would also make it substantially
|
||||
easier to grant users access to particular repositories for collaboration
|
||||
purposes, without having to grant them general access to the entire
|
||||
installation.
|
||||
|
||||
|
||||
Automated merge gating for CPython
|
||||
----------------------------------
|
||||
|
||||
The benefits of this proposal accrue most directly to the core development
|
||||
team. First and foremost, it means that once we mark a patch as "Approved"
|
||||
in the updated code review system, *we're usually done*. The extra 20-30
|
||||
|
|
@ -438,35 +352,6 @@ infrastructure will at least require the development of additional
|
|||
Zuul trigger and action plugins, and may require additional development
|
||||
in some of our existing tools.
|
||||
|
||||
RhodeCode may also require some adjustment to fit in with existing
|
||||
infrastructure.
|
||||
|
||||
|
||||
User account management
|
||||
-----------------------
|
||||
|
||||
Ideally we'd be able to offer a single account that spans all python.org
|
||||
services, including RhodeCode, Roundup/Rietveld, PyPI and the back end for
|
||||
the new python.org site, but actually implementing that would be a distinct
|
||||
infrastructure project, independent of this PEP.
|
||||
|
||||
A potentially simpler near term solution would be if RhodeCode's user
|
||||
account management could be integrated with the existing account management
|
||||
in Roundup, similar to what was done for Rietveld. However, if that also
|
||||
turns out to be impractical in the near term, we would merely end up with
|
||||
another identity silo to be integrated at a later date, suggesting that
|
||||
this doesn't need to be considered a blocker for an initial RhodeCode
|
||||
deployment.
|
||||
|
||||
|
||||
Preserving existing SSH access and links for Mercurial repositories
|
||||
-------------------------------------------------------------------
|
||||
|
||||
Adopting RhodeCode may result in some changes to where repositories are
|
||||
located on the hg.python.org filesystem. We may want to ensure that
|
||||
existing links into the hg.python.org web service are preserved, and should
|
||||
definitely ensure that existing SSH based clones continue to work correctly.
|
||||
|
||||
|
||||
Rietveld/Roundup vs Gerrit
|
||||
--------------------------
|
||||
|
|
@ -512,18 +397,10 @@ works directly on patches, and is completely decoupled from any specific
|
|||
version control system.
|
||||
|
||||
Zuul is also directly integrated with git for patch manipulation - as far
|
||||
as I am aware, this part of the design isn't pluggable.
|
||||
|
||||
Rather than trying to adapt Zuul to work directly with Mercurial, it will
|
||||
likely be more practical to let Zuul continue to use git internally, and
|
||||
then use the hg-git Mercurial plugin to pull the output from Zuul into the
|
||||
master repo at hg.python.org. (While there are various plugins that are
|
||||
designed to let git push to Mercurial repos, the influence of GitHub is
|
||||
such that the hg-git plugin appears to be the most mature of the available
|
||||
options for hg-git interoperability).
|
||||
|
||||
One advantage of adopting RhodeCode to manage the repository infrastructure
|
||||
is that it supports git repositories in addition to Mercurial repositories.
|
||||
as I am aware, this part of the design isn't pluggable. However, at PyCon
|
||||
US 2014, the Mercurial core developers at the sprints expressed some
|
||||
interest in collaborating with the core development team and the Zuul
|
||||
developers on enabling the use of Zuul with Mercurial in addition to git.
|
||||
|
||||
|
||||
Buildbot vs Jenkins
|
||||
|
|
@ -779,10 +656,9 @@ unpaid contributors.
|
|||
Open Questions
|
||||
==============
|
||||
|
||||
Pretty much everything in the PEP. Do we want to adopt RhodeCode? Do we
|
||||
want to adopt merge gating and Zuul? Is Rietveld the right place to hook
|
||||
Zuul into our current workflows? How do we want to address the various
|
||||
technical challenges?
|
||||
Pretty much everything in the PEP. Do we want to adopt merge gating and
|
||||
Zuul? Is Rietveld the right place to hook Zuul into our current workflows?
|
||||
How do we want to address the various technical challenges?
|
||||
|
||||
Assuming we do want to do it (or something like it), how is the work going
|
||||
to get done? Do we try to get it done solely as a volunteer effort? Do we
|
||||
|
|
@ -798,15 +674,10 @@ OpenStack currently dwarf those for CPython?
|
|||
Next Steps
|
||||
==========
|
||||
|
||||
The topic of CPython workflow automation is on the agenda for the Language
|
||||
Summit at PyCon US 2014 in Montreal, and we will be inviting additional
|
||||
participants (specifically Mercurial and Zuul developers) to be involved
|
||||
in the discussions (Guido van Rossum is the creator of Rietveld, and these
|
||||
workflow changes are not expected to require any significant changes in
|
||||
Roundup or Buildbot).
|
||||
|
||||
Unfortunately, the lead RhodeCode developers aren't able to attend PyCon US
|
||||
this year, or we would have invited them, too.
|
||||
Unfortunately, we ran out of time at the PyCon 2014 language summit to really
|
||||
discuss these issues. However, the `core-workflow mailing list
|
||||
<https://mail.python.org/mailman/listinfo/core-workflow>`__ has now been set
|
||||
up to discuss workflow issues in general.
|
||||
|
||||
|
||||
Acknowledgements
|
||||
|
|
@ -817,10 +688,6 @@ feedback on a preliminary draft of this proposal, and to James and Monty
|
|||
Taylor for additional technical feedback following publication of the
|
||||
initial draft.
|
||||
|
||||
Thanks also to Marcin Kuzminski (CTO of RhodeCode) for getting in touch to
|
||||
express their interest in helping to ensure the success of a RhodeCode
|
||||
deployment if we choose to proceed down that path.
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
|
|
|||
|
|
@ -0,0 +1,161 @@
|
|||
PEP: 474
|
||||
Title: Creating forge.python.org
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||||
Status: Draft
|
||||
Type: Process
|
||||
Content-Type: text/x-rst
|
||||
Created: 19-Jul-2014
|
||||
Post-History: 19-Jul-2014
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes setting up a new PSF provided resource, forge.python.org,
|
||||
as a location for maintaining various supporting repositories
|
||||
(such as the repository for Python Enhancement Proposals) in a way that is
|
||||
more accessible to new contributors, and easier to manage for core
|
||||
developers.
|
||||
|
||||
This PEP does *not* propose any changes to the core development workflow
|
||||
for CPython itself (see PEP 462 in relation to that).
|
||||
|
||||
|
||||
Proposal
|
||||
========
|
||||
|
||||
This PEP proposes that, once the Kallithea project has made an official
|
||||
release, that a Kallithea instance be deployed as "forge.python.org".
|
||||
|
||||
Individual repositories (such as the developer guide or the PEPs repository)
|
||||
may then be migrated from the existing hg.python.org infrastructure to the
|
||||
new forge.python.org infrastructure on a case by case basis. Each migration
|
||||
will need to decide whether to retain a read-only mirror on hg.python.org,
|
||||
or whether to just migrate wholesale to the new location.
|
||||
|
||||
This would not be a general purpose hosting site for arbitrary Python
|
||||
projects, but a more narrowly focused site akin to the existing
|
||||
hg.python.org.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
Currently, hg.python.org hosts more than just the core CPython repository,
|
||||
it also hosts other repositories such as those for the CPython developer
|
||||
guide and for Python Enhancement Proposals, along with various "sandbox"
|
||||
repositories for core developer experimentation.
|
||||
|
||||
While the simple "pull request" style workflow made popular by code hosting
|
||||
sites like GitHub and BitBucket isn't adequate for the complex branching
|
||||
model needed for parallel maintenance and development of the various
|
||||
CPython releases, it's a good fit for several of the ancillary projects
|
||||
that surround CPython that we don't wish to move to a proprietary hosting
|
||||
site.
|
||||
|
||||
The key requirements proposed for a PSF provided software forge are:
|
||||
|
||||
* Must support self-hosting on PSF infrastructure without ongoing fees
|
||||
* Must support Mercurial (for consistency with existing tooling)
|
||||
* Must support simple "pull request" style workflows
|
||||
* Must support online editing for simple changes
|
||||
|
||||
Ideally, the chosen solution would also be a fully open source application
|
||||
written in Python.
|
||||
|
||||
The requirement for self-hosting without ongoing fees rules out the
|
||||
free-as-in-beer providers like GitHub and BitBucket, in addition to the
|
||||
various proprietary source code management offerings.
|
||||
|
||||
The requirement for Mercurial support not only rules out GitHub, but also
|
||||
other Git-only solutions like GitLab and Gitorious.
|
||||
|
||||
The requirement for online editing support rules out the Apache
|
||||
Allura/HgForge combination.
|
||||
|
||||
That leaves two main contenders from a technical perspective:
|
||||
|
||||
* `RhodeCode <https://code.rhodecode.com/rhodecode>`__
|
||||
* `Kallithea SCM <https://kallithea-scm.org/>`__
|
||||
|
||||
The `legal uncertainty
|
||||
<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ over the
|
||||
interaction between RhodeCode's current "Business Source" licensing model and
|
||||
the various GPL components it relies on currently make it unclear whether it
|
||||
is legally permissible to deploy it.
|
||||
|
||||
By contrast, Kallithea is a full GPLv3 application (derived from the clearly
|
||||
and unambiguously GPLv3 licensed components of RhodeCode) that is being
|
||||
developed under the auspices of the `Software Freedom Conservancy
|
||||
<http://sfconservancy.org/news/2014/jul/04/kallithea-joins/>`__. The
|
||||
Conservancy has `affirmed
|
||||
<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ that the
|
||||
Kallithea codebase is completely and validly licensed under GPLv3. In
|
||||
addition to their role in building the initial Kallithea community, the
|
||||
Conservancy is also the legal home of both the Mercurial and Git projects.
|
||||
Other SFC member projects that may be familiar to Python users include
|
||||
Twisted, Gevent, BuildBot and PyPy.
|
||||
|
||||
|
||||
Perceived Benefits
|
||||
==================
|
||||
|
||||
The primary benefit of deploying Kallithea as forge.python.org is that
|
||||
supporting repositories such as the developer guide and the PEP repo could
|
||||
potentially be managed using pull requests and online editing. This would be
|
||||
much simpler than the current workflow which requires PEP editors and
|
||||
other core developers to act as intermediaries to apply updates suggested
|
||||
by other users.
|
||||
|
||||
The richer administrative functionality would also make it substantially
|
||||
easier to grant users access to particular repositories for collaboration
|
||||
purposes, without having to grant them general access to the entire
|
||||
installation.
|
||||
|
||||
|
||||
Technical Challenges
|
||||
====================
|
||||
|
||||
User account management
|
||||
-----------------------
|
||||
|
||||
Ideally we'd be able to offer a single account that spans all python.org
|
||||
services, including Kallithea, Roundup/Rietveld, PyPI and the back end for
|
||||
the new python.org site, but actually implementing that would be a distinct
|
||||
infrastructure project, independent of this PEP.
|
||||
|
||||
A potentially simpler near term solution would be if Kallithea's user
|
||||
account management could be integrated with the existing account management
|
||||
in Roundup, similar to what was done for Rietveld. However, if that also
|
||||
turns out to be impractical in the near term, we would merely end up with
|
||||
another identity silo to be integrated at a later date, suggesting that
|
||||
this doesn't need to be considered a blocker for an initial Kallithea
|
||||
deployment.
|
||||
|
||||
|
||||
Breaking existing SSH access and links for Mercurial repositories
|
||||
-----------------------------------------------------------------
|
||||
|
||||
This PEP proposes leaving the existing hg.python.org installation alone,
|
||||
and setting up Kallithea on a new host. This approach minimises the risk
|
||||
of interfering with the development of CPython itself (and any other
|
||||
projects that don't migrate to the new software forge), but does make any
|
||||
migrations of existing repos more disruptive (since existing checkouts will
|
||||
break).
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
||||
Loading…
Reference in New Issue