iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Split software forge proposal into a new PEP 

- moved the software forge proposal out to its own PEP
- switched forge proposal to Kallithea based on SFC concerns
  with the current RhodeCode licensing structure
This commit is contained in:
Nick Coghlan 2014-07-19 19:49:30 +10:00
parent 89ae8bb813
commit 209a860d13
2 changed files with 175 additions and 147 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

161
pep-0462.txt
View File

@ -21,11 +21,6 @@ available to contribute to CPython, which should also result in an improved
experience for other contributors that are reliant on the core team to get experience for other contributors that are reliant on the core team to get
their changes incorporated. their changes incorporated.
This PEP also proposes changes to the way certain supporting repositories
(such as the repository for Python Enhancement Proposals) are managed to
make them more accessible to new contributors, and easier to manage for
core developers.
Rationale for changes to the core development workflow Rationale for changes to the core development workflow
====================================================== ======================================================
@ -158,23 +153,6 @@ incorporation doesn't score well on that front for anyone:
the case for OpenStack). the case for OpenStack).
Rationale for changes to source code repository management
==========================================================
Currently, hg.python.org hosts more than just the core CPython repository,
it also hosts other repositories such as those for the CPython developer
guide and for Python Enhancement Proposals, along with various "sandbox"
repositories for core developer experimentation.
These supporting repositories could benefit greatly from user the simple
"pull request" style workflow made popular by code hosting sites like
GitHub and BitBucket.
This PEP proposes introducing a more sophisticated approach to repository
management that includes more "self service" features, including support
for pull requests.
Current Tools Current Tools
============= =============
@ -198,53 +176,8 @@ tools to see which, if any, may be candidates for replacement.
Proposal Proposal
======== ========
This proposal consists of two key components: The essence of this proposal is that CPython aim to adopt a "core reviewer"
development model, similar to that used by the OpenStack project.
* Introducing RhodeCode for improved repository management
* Introducing automated merge gating for the CPython project
Improved repository management
------------------------------
The rise of free source code hosting sites like GitHub and BitBucket with a
focus on improving the user experience have increased user expectations
for the web based interface offered for source code repositories. This
includes features like dynamic control of user permissions for repository
administrators, online editing for documentation updates and similar small
fixes, easy cloning and branching, etc.
While GitHub and BitBucket are proprietary solutions to this problem that
cannot be self-hosted, RhodeCode_ is a popular primarily open source solution
that offers similar features to the major free hosting sites, while also
allowing installation on your own servers. RhodeCode is also implemented
in Python, allowing us to preserve our current situation of having our
workflow tools being almost entirely self-hosting.
There's nothing too complicated with this part of the proposal: it is
essentially just a suggestion to convert hg.python.org to being backed
by a RhodeCode Enterprise instance hosted on PSF infrastructure.
All of the functional parts of RhodeCode Enterprise are open source under
the GPLv3 license. The "look & feel" components are licensed under a
`business source license`_ that is free for up to 20 users, as well as for
organisations that qualify for one of RhodeCode's exemption categories
(in the case of ``hg.python.org``, the relevant category is "public open
source project").
The RhodeCode developers have also expressed interest in helping out with
ensuring a python.org RhodeCode deployment is successful.
.. _RhodeCode: https://rhodecode.com/
.. _business source license: https://rhodecode.com/licenses
Automated merge gating for CPython
----------------------------------
The essence of this part of the proposal is that CPython aim to adopt a
"core reviewer" development model, similar to that used by the OpenStack
project.
The workflow problems experienced by the CPython core development team are The workflow problems experienced by the CPython core development team are
not unique. The OpenStack infrastructure team have come up with a well not unique. The OpenStack infrastructure team have come up with a well
@ -255,7 +188,7 @@ designed automated workflow that is designed to ensure:
* patches never get merged without being tested relative to the current * patches never get merged without being tested relative to the current
state of the branch state of the branch
* the main development branch always stays green. Patches that do not pass * the main development branch always stays green. Patches that do not pass
the automated tests do not get merged. the automated tests do not get merged
If a core developer wants to tweak a patch prior to merging, they download If a core developer wants to tweak a patch prior to merging, they download
it from the review tool, modify and *upload it back to the review tool* it from the review tool, modify and *upload it back to the review tool*
@ -360,25 +293,6 @@ manage them using the simpler pull request based model.
Perceived Benefits Perceived Benefits
================== ==================
Improved repository management
------------------------------
The primary benefit of using RhodeCode to manage hg.python.org is that
supporting repositories such as the developer guide and the PEP repo could
now be managed using pull requests and online editing. This would be
much simpler then the current workflow which requires PEP editors and
other core developers to act as intermediaries to apply updates suggested
by other users.
The richer administrative functionality would also make it substantially
easier to grant users access to particular repositories for collaboration
purposes, without having to grant them general access to the entire
installation.
Automated merge gating for CPython
----------------------------------
The benefits of this proposal accrue most directly to the core development The benefits of this proposal accrue most directly to the core development
team. First and foremost, it means that once we mark a patch as "Approved" team. First and foremost, it means that once we mark a patch as "Approved"
in the updated code review system, *we're usually done*. The extra 20-30 in the updated code review system, *we're usually done*. The extra 20-30
@ -438,35 +352,6 @@ infrastructure will at least require the development of additional
Zuul trigger and action plugins, and may require additional development Zuul trigger and action plugins, and may require additional development
in some of our existing tools. in some of our existing tools.
RhodeCode may also require some adjustment to fit in with existing
infrastructure.
User account management
-----------------------
Ideally we'd be able to offer a single account that spans all python.org
services, including RhodeCode, Roundup/Rietveld, PyPI and the back end for
the new python.org site, but actually implementing that would be a distinct
infrastructure project, independent of this PEP.
A potentially simpler near term solution would be if RhodeCode's user
account management could be integrated with the existing account management
in Roundup, similar to what was done for Rietveld. However, if that also
turns out to be impractical in the near term, we would merely end up with
another identity silo to be integrated at a later date, suggesting that
this doesn't need to be considered a blocker for an initial RhodeCode
deployment.
Preserving existing SSH access and links for Mercurial repositories
-------------------------------------------------------------------
Adopting RhodeCode may result in some changes to where repositories are
located on the hg.python.org filesystem. We may want to ensure that
existing links into the hg.python.org web service are preserved, and should
definitely ensure that existing SSH based clones continue to work correctly.
Rietveld/Roundup vs Gerrit Rietveld/Roundup vs Gerrit
-------------------------- --------------------------
@ -512,18 +397,10 @@ works directly on patches, and is completely decoupled from any specific
version control system. version control system.
Zuul is also directly integrated with git for patch manipulation - as far Zuul is also directly integrated with git for patch manipulation - as far
as I am aware, this part of the design isn't pluggable. as I am aware, this part of the design isn't pluggable. However, at PyCon
US 2014, the Mercurial core developers at the sprints expressed some
Rather than trying to adapt Zuul to work directly with Mercurial, it will interest in collaborating with the core development team and the Zuul
likely be more practical to let Zuul continue to use git internally, and developers on enabling the use of Zuul with Mercurial in addition to git.
then use the hg-git Mercurial plugin to pull the output from Zuul into the
master repo at hg.python.org. (While there are various plugins that are
designed to let git push to Mercurial repos, the influence of GitHub is
such that the hg-git plugin appears to be the most mature of the available
options for hg-git interoperability).
One advantage of adopting RhodeCode to manage the repository infrastructure
is that it supports git repositories in addition to Mercurial repositories.
Buildbot vs Jenkins Buildbot vs Jenkins
@ -779,10 +656,9 @@ unpaid contributors.
Open Questions Open Questions
============== ==============
Pretty much everything in the PEP. Do we want to adopt RhodeCode? Do we Pretty much everything in the PEP. Do we want to adopt merge gating and
want to adopt merge gating and Zuul? Is Rietveld the right place to hook Zuul? Is Rietveld the right place to hook Zuul into our current workflows?
Zuul into our current workflows? How do we want to address the various How do we want to address the various technical challenges?
technical challenges?
Assuming we do want to do it (or something like it), how is the work going Assuming we do want to do it (or something like it), how is the work going
to get done? Do we try to get it done solely as a volunteer effort? Do we to get done? Do we try to get it done solely as a volunteer effort? Do we
@ -798,15 +674,10 @@ OpenStack currently dwarf those for CPython?
Next Steps Next Steps
========== ==========
The topic of CPython workflow automation is on the agenda for the Language Unfortunately, we ran out of time at the PyCon 2014 language summit to really
Summit at PyCon US 2014 in Montreal, and we will be inviting additional discuss these issues. However, the `core-workflow mailing list
participants (specifically Mercurial and Zuul developers) to be involved <https://mail.python.org/mailman/listinfo/core-workflow>`__ has now been set
in the discussions (Guido van Rossum is the creator of Rietveld, and these up to discuss workflow issues in general.
workflow changes are not expected to require any significant changes in
Roundup or Buildbot).
Unfortunately, the lead RhodeCode developers aren't able to attend PyCon US
this year, or we would have invited them, too.
Acknowledgements Acknowledgements
@ -817,10 +688,6 @@ feedback on a preliminary draft of this proposal, and to James and Monty
Taylor for additional technical feedback following publication of the Taylor for additional technical feedback following publication of the
initial draft. initial draft.
Thanks also to Marcin Kuzminski (CTO of RhodeCode) for getting in touch to
express their interest in helping to ensure the success of a RhodeCode
deployment if we choose to proceed down that path.
Copyright Copyright
========= =========

161
pep-0474.txt Normal file
View File

@ -0,0 +1,161 @@
PEP: 474
Title: Creating forge.python.org
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 19-Jul-2014
Post-History: 19-Jul-2014
Abstract
========
This PEP proposes setting up a new PSF provided resource, forge.python.org,
as a location for maintaining various supporting repositories
(such as the repository for Python Enhancement Proposals) in a way that is
more accessible to new contributors, and easier to manage for core
developers.
This PEP does *not* propose any changes to the core development workflow
for CPython itself (see PEP 462 in relation to that).
Proposal
========
This PEP proposes that, once the Kallithea project has made an official
release, that a Kallithea instance be deployed as "forge.python.org".
Individual repositories (such as the developer guide or the PEPs repository)
may then be migrated from the existing hg.python.org infrastructure to the
new forge.python.org infrastructure on a case by case basis. Each migration
will need to decide whether to retain a read-only mirror on hg.python.org,
or whether to just migrate wholesale to the new location.
This would not be a general purpose hosting site for arbitrary Python
projects, but a more narrowly focused site akin to the existing
hg.python.org.
Rationale
=========
Currently, hg.python.org hosts more than just the core CPython repository,
it also hosts other repositories such as those for the CPython developer
guide and for Python Enhancement Proposals, along with various "sandbox"
repositories for core developer experimentation.
While the simple "pull request" style workflow made popular by code hosting
sites like GitHub and BitBucket isn't adequate for the complex branching
model needed for parallel maintenance and development of the various
CPython releases, it's a good fit for several of the ancillary projects
that surround CPython that we don't wish to move to a proprietary hosting
site.
The key requirements proposed for a PSF provided software forge are:
* Must support self-hosting on PSF infrastructure without ongoing fees
* Must support Mercurial (for consistency with existing tooling)
* Must support simple "pull request" style workflows
* Must support online editing for simple changes
Ideally, the chosen solution would also be a fully open source application
written in Python.
The requirement for self-hosting without ongoing fees rules out the
free-as-in-beer providers like GitHub and BitBucket, in addition to the
various proprietary source code management offerings.
The requirement for Mercurial support not only rules out GitHub, but also
other Git-only solutions like GitLab and Gitorious.
The requirement for online editing support rules out the Apache
Allura/HgForge combination.
That leaves two main contenders from a technical perspective:
* `RhodeCode <https://code.rhodecode.com/rhodecode>`__
* `Kallithea SCM <https://kallithea-scm.org/>`__
The `legal uncertainty
<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ over the
interaction between RhodeCode's current "Business Source" licensing model and
the various GPL components it relies on currently make it unclear whether it
is legally permissible to deploy it.
By contrast, Kallithea is a full GPLv3 application (derived from the clearly
and unambiguously GPLv3 licensed components of RhodeCode) that is being
developed under the auspices of the `Software Freedom Conservancy
<http://sfconservancy.org/news/2014/jul/04/kallithea-joins/>`__. The
Conservancy has `affirmed
<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ that the
Kallithea codebase is completely and validly licensed under GPLv3. In
addition to their role in building the initial Kallithea community, the
Conservancy is also the legal home of both the Mercurial and Git projects.
Other SFC member projects that may be familiar to Python users include
Twisted, Gevent, BuildBot and PyPy.
Perceived Benefits
==================
The primary benefit of deploying Kallithea as forge.python.org is that
supporting repositories such as the developer guide and the PEP repo could
potentially be managed using pull requests and online editing. This would be
much simpler than the current workflow which requires PEP editors and
other core developers to act as intermediaries to apply updates suggested
by other users.
The richer administrative functionality would also make it substantially
easier to grant users access to particular repositories for collaboration
purposes, without having to grant them general access to the entire
installation.
Technical Challenges
====================
User account management
-----------------------
Ideally we'd be able to offer a single account that spans all python.org
services, including Kallithea, Roundup/Rietveld, PyPI and the back end for
the new python.org site, but actually implementing that would be a distinct
infrastructure project, independent of this PEP.
A potentially simpler near term solution would be if Kallithea's user
account management could be integrated with the existing account management
in Roundup, similar to what was done for Rietveld. However, if that also
turns out to be impractical in the near term, we would merely end up with
another identity silo to be integrated at a later date, suggesting that
this doesn't need to be considered a blocker for an initial Kallithea
deployment.
Breaking existing SSH access and links for Mercurial repositories
-----------------------------------------------------------------
This PEP proposes leaving the existing hg.python.org installation alone,
and setting up Kallithea on a new host. This approach minimises the risk
of interfering with the development of CPython itself (and any other
projects that don't migrate to the new software forge), but does make any
migrations of existing repos more disruptive (since existing checkouts will
break).
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: