python-peps/pep-8002.rst

PEP: 8002
Title: Open Source Governance Survey
Author: Barry Warsaw <barry@python.org>, Łukasz Langa <lukasz@python.org>,
   Antoine Pitrou <solipsis@pitrou.net>, Doug Hellmann <doug@doughellmann.com>,
   Carol Willing <willingc@gmail.com>
Status: Active
Type: Informational
Content-Type: text/x-rst
Created: 2018-08-24


Abstract
========

This PEP surveys existing and similar open source and free software projects
for their governance models, providing summaries that will serve as useful
references for Python's own selection of a new governance model in the wake of
`Guido's retirement
<https://mail.python.org/pipermail/python-committers/2018-July/005664.html>`_.

Rather than an individual PEP for each of these community surveys, they will
all be collected here in this PEP.


Rationale
=========

CPython is not the first open source project to undergo a governance crisis.
Other projects have experimented various governance options, sometimes several
times during their existence.  There are useful lessons to take away of their
experience, which will help inform our own decision.

Project choice
--------------

There are many open source projects out there, but it will be most fruitful
to survey those which are similar enough to CPython on a couple key metrics:

1. the number of contributors and their activity (there are scaling issues that
   don't make the governance models of very small projects very enlightening
   for our purposes) ;
2. being mostly or partly community-driven (company-driven projects can afford
   different governance options, since the company hierarchy has power over
   the main participants) ;
3. being faced with important design decisions that require a somewhat formal
   decision process.


Rust
====

The governance structure is documented in `Rust RFC #1068
<https://github.com/rust-lang/rfcs/blob/master/text/1068-rust-governance.md>`_.

The effective governance process grows organically over time without being entirely
codified as RFCs, especially in case of day-to-day operation details.  One example is
the `formation of Domain Working Groups
<https://internals.rust-lang.org/t/announcing-the-2018-domain-working-groups/6737>`_ in
February 2018.

Key people and their functions
------------------------------

In the Rust project there are teams responsible for certain areas. For language features
there is a "lang team", for tooling there's "dev tools" and "Cargo", and so on.
Contentious issues have facilitators to drive discussion who often aren't the decision
makers.  Typically the faciliators are authors of the proposed changes (see
"Controversial decision process" below).  They ensure all key decision makers are
involved along with interested community members.  They push towards an agreeable
outcome via iteration.

In practice this means decisions are rarely escalated to the core team.

The most common role of a contributor is team membership.  Issue triage/code review
privileges without team membership is rare.  Contributors have full commit access,
code ownership separation is based on trust.  Writing to the compiler repository is
frowned upon, all changes go through pull requests and get merged by an integration
bot after they were reviewed and approved.

New team members are added by nomination by an existing team member.

Regular decision process
------------------------

Primary work happens via GitHub issues and pull requests.  Approving a pull request
by any team member allows it to be merged without further process.  All merged pull
requests end up in the next stable version of Rust.

Notifying relevant people by mentions is important.  Listening to the firehose of
e-mails for all GitHub activity is not popular.

There are planning and triage meetings open to the public happening on IRC and Discord.
They are not very popular because most of work happens through GitHub.  Discussions also
happen on official Rust forums (https://users.rust-lang.org/ and
https://internals.rust-lang.org/).  There is a dedicated moderation team responsible for
taking notes and enforcing `code of conduct
<https://www.rust-lang.org/en-US/conduct.html>`_.

Controversial decision process
------------------------------

Larger or controversial work goes through a `RFC process
<https://github.com/rust-lang/rfcs>`_.  It allows everyone to express their thoughts and
iterates towards a resolution.  At some point when all blocking concerns of relevant
team members are addressed, they sign off on the RFC and it reaches a "final comment
period". That does not require consensus amongst all participants, rather there should
not be a strong consensus against the proposal.

After 10 days the RFC is *merged* unless any new blocking concerns are raised by team
members.  A "merge" signifies that work towards implementing the feature and integrating
it can now happen without interruption.  An RFC doesn't have to have a reference
implementation for it to be accepted.

The other possible results of the "final comment period" are to:

* *postpone* the RFC (similar to the Deferred status in PEPs),
* get it *back into discussion* if blocking concerns can be addressed, or
* *close it* if blocking concerns are not solvable.  When an RFC is marked as
  *closed*, there is a 7 day grace period to debate whether it should be closed.

In practice registering concerns with an RFC happens very often initially but rarely
causes for the RFC to be entirely killed.

This process scales well for small-contention changes and/or smaller changes.  For the
largest controversial changes the discussion gets unwieldy.  This is a topic currently
(as of August 2018) on the minds of the Rust team (see:
`"Listening and Trust, part 1" <http://aturon.github.io/2018/05/25/listening-part-1/>`_,
`"Listening and Trust, part 2" <http://aturon.github.io/2018/06/02/listening-part-2/>`_,
`"Listening and Trust, part 3" <http://aturon.github.io/2018/06/18/listening-part-3/>`_,
`"Proposal for a staged RFC process"
<http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/>`_).

Planning a new release
----------------------

Every six weeks the Rust compiler is released with whatever it contained at the time.
There are no LTS channels or releases yet but this concept is planned to make
redistributors able to keep up with development better.

Every few years a so-called `"Edition"
<https://rust-lang-nursery.github.io/edition-guide/editions/index.html>`_ is released.
Those are milestone releases with full sets of updated documentation and tooling. They
can be backwards incompatible with previous editions.  External packages opt into
breaking changes in their crate metadata.  The Rust compiler supports all editions that
existed prior to its release.  Linking between crates of any supported edition is
possible.

Changes in the process over time
--------------------------------

The Rust programming language was started by Graydon Hoare who developed it as
a personal project for a few years.  When Mozilla started sponsoring the project,
the team slowly grew with Graydon as a BDFL-style figure.  He `left the project
<https://www.reddit.com/r/rust/comments/7qels2/i_wonder_why_graydon_hoare_the_author_of_rust/dsqeh1d/>`_
in 2013.  Rust functions without a BDFL since.  The RFC process was put in place later.
Initially some design discussions happened during closed-door weekly video meetings
which was `shut down
<https://github.com/rust-lang/meeting-minutes/blob/master/weekly-meetings/2015-05-26.md#future-of-weekly-meeting>`_
in May 2015 (before the 1.0 release of Rust), organically replaced with open discussion
and direct influence of teams.

The number of teams is growing in time.  The number of technical decisions made by the
core team is decreasing, instead those get delegated to respective teams.

The concept of a "final comment period" was introduced to encourage more public
discussion and enable reacting to a change *about to* being made, instead of having to
revert a rushed decision that was already made.


OpenStack
=========

The OpenStack Foundation Bylaws lay out the basic structure for
project governance, with `Article IV
<https://www.openstack.org/legal/bylaws-of-the-openstack-foundation/>`__
delegating day-to-day management of the open source project to the
OpenStack Technical Committee (TC), and `The TC member policy
<https://www.openstack.org/legal/technical-committee-member-policy/>`__
defining broadly how the Technical Committee shall be elected. The TC
publishes a set of more detailed `governance documents
<https://governance.openstack.org/tc/>`__, including `the TC charter
<https://governance.openstack.org/tc/reference/charter.html>`__, which
describes the team structure, precise rules for establishing
eligibility to run for office, and criteria for establishing the
various electorates.

Key people and their functions
------------------------------

The OpenStack community is made up of many distinct `project teams
<https://governance.openstack.org/tc/reference/projects/index.html>`__,
responsible for producing different components of the software (block
storage management, compute management, etc.) or managing different
parts of the processes the community follows (such as tracking the
release schedule).  Each team is led by a *Project Team Lead* (PTL),
elected by the *Active Project Contributors* for that project.

Active Project Contributors (APCs) are recent contributors to a given
project team.  APC status formally requires two things: becoming an
individual member of the OpenStack Foundation (membership is free) and
having a change merged within the last year (two development cycles)
in a repository managed by a project team.

The elected PTL serves a term equal to one development cycle (roughly
6 months). There is no restriction on the number of consecutive terms
a person may serve as PTL, and it is common for someone to serve for
several terms in a row. It is also not unusual for a team to have only
one candidate volunteer to serve as PTL for a given cycle, in which
case there is no need for an election.

The PTL represents the team in all cases except where they have
explicitly delegated some responsibility. For example, many teams
designate a separate *release liaison* to manage the release process
for a development cycle.  The PTL also serves as a final decision
maker in cases where consensus cannot be reached between the team
members.

While the APCs all vote for the PTL of a team, in many other cases
only the *core reviewer* team will be consulted on policy decisions
for the team. Anyone may review any patch for any OpenStack
project. After someone demonstrates that they have a good grasp of the
technical issues of a project, that they provide useful feedback on
reviews, and that they understand the direction the project is going,
they may be invited to become a member of the core review team. Unlike
in many other communities, this status does not grant them the right
to submit code without having it reviewed. Rather, it asks them to
commit to reviewing code written by other contributors, and to
participate in team decision-making discussions. Asking someone to
become a member of the core review team is a strong indication of
trust.

The Technical Committee (TC) is responsible for managing the
development of OpenStack as a whole.  The 13 members of the Technical
Committee are directly elected by APCs from all project teams. Each
member serves a term of two development cycles (roughly 1 year), with
the elections split so that only about half of the members' terms
expire at any time, to ensure continuity. The TC establishes overall
policies, such as the criteria for adding new project teams, the
deprecation policy for Python 2, testing requirements, etc.

Regular decision process
------------------------

All elections for PTL or TC members use https://civs.cs.cornell.edu to
run a *Condorcet* election. This system was selected because it
emphasizes consensus candidates over strict popularity.

The OpenStack contributor community relies on 3 primary tools for
discussion: the `openstack-dev mailing list
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>`__,
a gerrit code review instance at https://review.openstack.org, and a
set of `OpenStack-specific IRC channels
<http://eavesdrop.openstack.org>`__ on Freenode. There are a few teams
whose contributors are based primarily in China, and they have trouble
accessing IRC. Those teams tend to use alternative platforms such as
WeChat, instead.

The tool used for discussing any given decision will vary based on its
weight and impact. Everyone is encouraged to use either the mailing
list or gerrit to support asynchronous discussion across a wider range
of timezones and and firewalls, especially for publicizing final
decisions for the rest of the community.

Policy decisions limited to a single team are usually made by the core
review team for a project, and the policies and decision processes may
vary between teams.  Some groups write down their team policies in
their documentation repository, and use the code review tool (gerrit)
to vote on them. Some teams discuss policies on IRC, either ad hoc or
during a regularly scheduled meeting, and make decisions there.  Some
teams use the mailing list for those discussions. The PTL for the team
is responsible for ensuring the discussion is managed and the outcome
is communicated (either by doing so directly or ensuring that the task
is delegated to someone else).

All team policy decisions need to be compatible with the overall
policies set by the Technical Committee. Because the TC tends to make
broader governance decisions that apply to the entire contributor
community, the process for discussing and voting on those decisions is
described more formally, including specifying the number of votes
needed to pass and the minimum length of time required for
discussion. For example, most motions require 1/3 of the members (5)
to pass and must stay open at least 3 days after receiving sufficient
votes to pass, ensuring that there is time for dissent to be
registered. See the `Technical Committee Charter
<https://governance.openstack.org/tc/reference/charter.html#motions>`__
and `house rules
<https://governance.openstack.org/tc/reference/house-rules.html>`__
for more details.

Significant design decisions are usually discussed by reviewing a
`specification document <http://specs.openstack.org>`__, somewhat
similar to a PEP, that covers the requirements, alternatives, and
implementation details. Feedback is solicited from all contributors,
and then specifications are eventually approved or rejected by members
of the core review team for a project. Some teams require only 2
reviewers to approve a design, while other teams require a stronger
indication of consensus before a design is approved. Each team sets a
`deadline for approving specifications within each development cycle
<https://releases.openstack.org/rocky/schedule.html>`__, to encourage
contributors to work out designs for significant new features early
and avoid risk from changes late in the cycle.

Smaller technical decisions are typically made by reviewing the
patch(es) needed to implement the change. Anyone may review any patch
and provide technical feedback, but ultimately two core reviewers for
a team are needed to approve most changes (exceptions are often made
for trivial changes such as typos or for fixes that unblock the CI
gating system).

Controversial decision process
------------------------------

Controversial, or merely complicated, decisions frequently expand
outside of specification reviews to mailing list discussions. They
often also result in discussions at one of the regularly scheduled
in-person community gatherings. Because many members of the community
cannot attend these events, the discussions are summarized and final
decisions are made using on-line tools as much as possible.

The PTL is responsible for deciding when consensus has been reached
for decisions that affect a single team, and to make a final call in
rare cases where consensus has not been reached and a decision
absolutely needs to be made.  The TC acts as a similar decision-making
group of last resort for cases where issues *between* teams cannot be
resolved in another way. Such escalation of decision-making ends up
being rarely necessary, because the contributors directly involved
generally prefer to come to a consensual agreement rather than
escalate the decision to others.

Planning a new release
----------------------

OpenStack has a major release about every 6 months. These are
coordinated date-based releases, which include the work finished up to
that point in time in all of the member projects. Some project teams
release more often than every 6 months (this is especially true for
teams building libraries consumed by other teams). Those smaller
releases tend to be produced when there is content (new features or
bug fixes) to justify them.

The schedule for each development cycle, with deadlines and a final
release date, is proposed by the release management team, in
coordination with the Foundation staff (releases are generally aligned
with the calendar of in-person events), and then the community has an
opportunity to provide feedback before the final dates are set.

Decisions about priorities for each development cycle are made at the
team level and the TC level. Core review teams prioritize internal
work, such as fixing bugs and implementing new features. The TC
selects `community goals
<https://governance.openstack.org/tc/goals/index.html>`__, which
usually require some amount of work from all teams. Agreeing to these
priorities at the start of each cycle helps the teams coordinate their
work, which is especially important because the implementation will
require reviews from multiple team members.

Changes in the process over time
--------------------------------

Over the last 8 years the number of OpenStack project teams has grown
from 2 to 63. The makeup of the Technical Committee has changed to
accommodate that growth. Originally the TC was made up of PTLs, but as
the membership grew it became impractical for the group to function
effectively.

The community also used to be organized around "program areas" rather
than project teams. A program area covered a feature set, such as
gathering telemetry or managing block storage. This organization
failed when multiple teams of people wanted to work on the same
feature set using different solutions. Organizing teams around the
code they deliver allows different teams to have different
interpretations of the same requirements. For example, there are now
several teams working on different deployment tools.


Jupyter
=======

The governance structure is documented in the `Main Governance Document
<https://github.com/jupyter/governance/blob/master/governance.md>`_
within the `Jupyter Governance repo <https://github.com/jupyter/governance>`_.

The effective governance process grows organically over time as the needs of
the project evolve. Formal changes to the Governance Document are submitted via
Pull Request, with an open period for comments. After the open period, a
Steering Council may call for a vote to ratify the PR changes. Acceptance
requires a minimum of 80% of the Steering Council to vote and at least 2/3 of
the vote must be positive. The BDFL can act alone to accept or reject changes
or override the Steering Council decision; though this would be an extremely
rare event.

Key people and their functions
------------------------------

The key people in Jupyter's Governance are the BDFL, Fernando Perez, and the
Steering Council. Contributors can be given a special status of core contributor.
Some may also be Institutional Contributors, who are individuals who contribute
to the project as part of their official duties at an Institutional Partner.

Fernando Perez, the project founder, is the current and first BDFL. The BDFL
may serve as long as desired. The `BDFL succession plan <https://github.com/jupyter/governance/blob/master/governance.md#bdfl>`_
is described in the Main Governance Document. In summary, the BDFL may appoint
the next BDFL. As a courtesy, it is expected that the BDFL will consult with the
Steering Council. In the event that the BDFL can not appoint a successor, the
Steering Council will recommend one.

Core contributors are individuals who are given rights, such as commit privileges,
to act in the best interest of the project within their area of expertise or
`subproject <https://github.com/jupyter/governance/blob/master/newsubprojects.md>`_.
An existing core contributor typically recommends someone be given
core contributor rights by gathering consensus from project leads, who are
experienced core contributors as listed in the README of the project repo.

To be recommended and invited as a Steering Council member, an individual must
be a Project Contributor who has produced contributions that are substantial in
quality and quantity, and sustained over at least one year. Potential Council
Members are nominated by existing Council members and voted upon by the 
existing Council after asking if the potential Member is interested and willing
to serve in that capacity.

Regular decision process
------------------------

Project Jupyter is made up of a number of GitHub organizations and subprojects
within those organizations. Primary work happens via GitHub issues and pull
requests. Approving a pull request by any team member allows it to be merged
without further process. All merged pull requests end up in the next stable
release of a subproject.

There is a weekly, public Project-wide meeting that is recorded and posted on
YouTube. Some larger GitHub organizations, which are subprojects of 
Project Jupyter, e.g. JupyterLab and JupyterHub, may
have additional public team meetings on a weekly or monthly schedule. 
Discussions occur on Gitter, the Jupyter mailing list, and most frequently an
open issue and/or pull request on GitHub.

Controversial decision process
------------------------------

The foundations of Project Jupyter's governance are:

* Openness & Transparency
* Active Contribution
* Institutional Neutrality

During the everyday project activities, Steering Council members participate in
all discussions, code review and other project activities as peers with all
other Contributors and the Community. In these everyday activities, 
Council Members do not have any special power or privilege through their 
membership on the Council. However, it is expected that because of the quality
and quantity of their contributions and their expert knowledge of the 
Project Software and Services that Council Members will provide useful guidance,
both technical and in terms of project direction, to potentially less 
experienced contributors.

For controversial issues, the contributor community works together to refine
potential solutions, iterate as necessary, and build consensus by sharing
information and views constructively and openly. The Steering Council may 
make decisions when regular community discussion doesn’t produce consensus
on an issue in a reasonable time frame.

Voting
------

Rarely, if ever, is voting done for technical decisions.

For other Project issues, the Steering Council may call for a vote for a
decision via a Governance PR or email proposal. Acceptance
requires a minimum of 80% of the Steering Council to vote and at least 2/3 of
the vote must be positive. 

The BDFL can act alone to accept or reject changes or override the Steering
Council decision; though this would be an extremely rare event. As Benevolent,
the BDFL, in practice chooses to defer that authority to the consensus of the
community discussion channels and the Steering Council.

Planning releases
-----------------

Since Project Jupyter has a number of projects, not just a single project, the
release planning is largely driven by the core contributors of a project.

Changes in the process over time
--------------------------------

The process has remained consistent over time, and the approach has served us
well. Moving forward The Project leadership will consist of a BDFL and
Steering Council. This governance model was a formalization of what
the Project was doing (prior to 2015 when the Main Governance Document was
adopted by the Steering Council), rather than a change in direction.


Django
======

The governance structure is documented in `Organization of the Django Project
<https://docs.djangoproject.com/en/2.1/internals/organization/>`_.


Key people and their functions
------------------------------

The project recognizes three kinds of contributors.  Members of the
core team, the Technical Board, and Fellows. Regular core committers
no longer exercise their "commit bit", instead they rely on pull
requests being reviewed and accepted.  There is a bot which merges
the pull request.  The Technical Board steers technical choices.
Fellows are hired contractors who triage new tickets and review patches
from the community, including non-trivial ones.

Core team members are added by nomination and vote within the core
team, with technical board veto (so far not exercised). Technical
board is elected by and from the core team membership every 18 months
(every major Django release).  Sub-teams within the core team are
self-selected by interest.


Regular decision process
------------------------

Most day-to-day decisions are made by Fellows and sometimes other active
core team members.

The core team votes on new members which requires a 4/5 majority of
votes cast, no quorum requirement.  The Technical Board has veto power.
This power was never exercised


Controversial decision process
------------------------------

The Technical Board occasionally approves Django
Enhancement Proposals (DEPs) but those are rare.  The DEP process is
roughly modeled after PEPs and documented in `DEP 1
<https://github.com/django/deps/blob/master/final/0001-dep-process.rst>`_.
DEPs are mostly used to design major new features, but also for
information on general guidelines and process.

An idea for a DEP should be first publically vetted on the
django-developers mailing list.  After it was roughly validated, the
author forms a team with three roles:

* *authors* who write the DEP and steers the discussion;
* *implementers* who prepare the implementation of the DEP;
* a *shepherd* who is a core developer and will be the primary reviewer
  of the DEP.

The DEP's draft is submitted, assigned a number, and discussed.  Authors
collect feedback and steer discussion as they see fit.  Suggested venues
to avoid endless open-ended discussions are: separate mailing lists,
Wiki pages, working off of pull requests on the DEP.

Once the feedback round is over, the shepherd asks the Technical Board
for review and pronouncement.  The Board can rule on a DEP as a team or
designate one member to review and decide.

In any case where consensus can't be reached, the Technical Board has
final say. This was never exercised.

Differences between DEPs and PEPs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The main difference is that the entire workflow is based on pull
requests rather than e-mail.  They are pronounced upon by the Technical
Board.  They need to have the key roles identified before submission
and throughout the process.  The *shepherd* role exists to guide a DEP
to completion without engaging the Technical Board.

Those changes to the process make it more distributed and workable in
a governance model without a BDFL.


Planning a new release
----------------------

Releases are done on a fixed time-based schedule, with a major version
every 18 months.  With paid Fellows to ensure the necessary work gets
down, on-time releases are routine.


Changes in the process over time
--------------------------------

Django originally had two BDFLs: Jacob Kaplan-Moss and Adrian Holovaty.
They retired (`Adrian's post
<http://www.holovaty.com/writing/bdfls-retiring/>`_, `Jacob's post
<https://jacobian.org/writing/retiring-as-bdfls/>`_)
9 years into the project's history.  Following the stepping down,
the DEP process was defined.


Acknowledgements
================

Thank you to Alex Crichton from the Rust team for an extensive explanation of how the
core team governs the project.

Jeremy Stanley, Chris Dent, Julia Kreger, Sean McGinnis, Emmet Hikory,
and Thierry Carrez contributed to the OpenStack section.

The Project Jupyter Steering Council created the Main Governance Document for
Project Jupyter, and Carol Willing summarized the key points of that documennt
for the Jupyter section.


Annex 1: Template questions
===========================

The following set of questions was used as a template to guide evaluation and
interaction with the surveyed projects:

1. Do you have any open documentation on how the governance model is set up?

2. How does the process look like in practice?

   * Who are the key people?
   * What "special statuses" can contributors have?
   * How are they elected/how are the statuses assigned?
   * How are regular decisions made?
   * How are controversial decisions made?
   * Is there a voting mechanism? how does it work? how often do votes actually happen?
   * Is there a veto mechanism? how often was it actually used?

3. How do you like the process?

   * Which parts work well?
   * Which parts could work better?
   * When it doesn't work well, how does it look like?
   * What would you change if it were only up to you?

4. Related project work:

   * How do you decide when a release happens and what goes into it?
   * How do you decide who gets commit access?
   * Where do you hold discussions? (GitHub, mailing lists, face-to-face meetings, and so on)
   * Do you have a RFC/PEP-like process?
   * Who has access to those discussion channels?
   * How is this access granted/revoked?
   * Who moderates those discussions?
   * Do you (and how) censure participants and how?

5. Process evolution

   * How did this process evolve historically?
   * How can it be changed in the future?


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: