1004 lines
45 KiB
ReStructuredText
1004 lines
45 KiB
ReStructuredText
PEP: 8002
|
|
Title: Open Source Governance Survey
|
|
Author: Barry Warsaw <barry@python.org>, Łukasz Langa <lukasz@python.org>,
|
|
Antoine Pitrou <solipsis@pitrou.net>, Doug Hellmann <doug@doughellmann.com>,
|
|
Carol Willing <willingc@gmail.com>
|
|
Status: Final
|
|
Type: Informational
|
|
Topic: Governance
|
|
Content-Type: text/x-rst
|
|
Created: 24-Aug-2018
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP surveys existing and similar open source and free software projects
|
|
for their governance models, providing summaries that will serve as useful
|
|
references for Python's own selection of a new governance model in the wake of
|
|
`Guido's retirement
|
|
<https://mail.python.org/pipermail/python-committers/2018-July/005664.html>`_.
|
|
|
|
Rather than an individual PEP for each of these community surveys, they will
|
|
all be collected here in this PEP.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
CPython is not the first open source project to undergo a governance crisis.
|
|
Other projects have experimented various governance options, sometimes several
|
|
times during their existence. There are useful lessons to take away of their
|
|
experience, which will help inform our own decision.
|
|
|
|
Project choice
|
|
--------------
|
|
|
|
There are many open source projects out there, but it will be most fruitful
|
|
to survey those which are similar enough to CPython on a couple key metrics:
|
|
|
|
1. the number of contributors and their activity (there are scaling issues that
|
|
don't make the governance models of very small projects very enlightening
|
|
for our purposes) ;
|
|
2. being mostly or partly community-driven (company-driven projects can afford
|
|
different governance options, since the company hierarchy has power over
|
|
the main participants) ;
|
|
3. being faced with important design decisions that require a somewhat formal
|
|
decision process.
|
|
|
|
|
|
Rust
|
|
====
|
|
|
|
The governance structure is documented in `Rust RFC #1068
|
|
<https://github.com/rust-lang/rfcs/blob/master/text/1068-rust-governance.md>`_.
|
|
|
|
The effective governance process grows organically over time without being entirely
|
|
codified as RFCs, especially in case of day-to-day operation details. One example is
|
|
the `formation of Domain Working Groups
|
|
<https://internals.rust-lang.org/t/announcing-the-2018-domain-working-groups/6737>`_ in
|
|
February 2018.
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
In the Rust project there are teams responsible for certain areas. For language features
|
|
there is a "lang team", for tooling there's "dev tools" and "Cargo", and so on.
|
|
Contentious issues have facilitators to drive discussion who often aren't the decision
|
|
makers. Typically the facilitators are authors of the proposed changes (see
|
|
"Controversial decision process" below). They ensure all key decision makers are
|
|
involved along with interested community members. They push towards an agreeable
|
|
outcome via iteration.
|
|
|
|
In practice this means decisions are rarely escalated to the core team.
|
|
|
|
The most common role of a contributor is team membership. Issue triage/code review
|
|
privileges without team membership is rare. Contributors have full commit access,
|
|
code ownership separation is based on trust. Writing to the compiler repository is
|
|
frowned upon, all changes go through pull requests and get merged by an integration
|
|
bot after they were reviewed and approved.
|
|
|
|
New team members are added by nomination by an existing team member.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Primary work happens via GitHub issues and pull requests. Approving a pull request
|
|
by any team member allows it to be merged without further process. All merged pull
|
|
requests end up in the next stable version of Rust.
|
|
|
|
Notifying relevant people by mentions is important. Listening to the firehose of
|
|
e-mails for all GitHub activity is not popular.
|
|
|
|
There are planning and triage meetings open to the public happening on IRC and Discord.
|
|
They are not very popular because most of work happens through GitHub. Discussions also
|
|
happen on official Rust forums (https://users.rust-lang.org/ and
|
|
https://internals.rust-lang.org/). There is a dedicated moderation team responsible for
|
|
taking notes and enforcing `code of conduct
|
|
<https://www.rust-lang.org/en-US/conduct.html>`_.
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
Larger or controversial work goes through a `RFC process
|
|
<https://github.com/rust-lang/rfcs>`_. It allows everyone to express their thoughts and
|
|
iterates towards a resolution. At some point when all blocking concerns of relevant
|
|
team members are addressed, they sign off on the RFC and it reaches a "final comment
|
|
period". That does not require consensus amongst all participants, rather there should
|
|
not be a strong consensus against the proposal.
|
|
|
|
After 10 days the RFC is *merged* unless any new blocking concerns are raised by team
|
|
members. A "merge" signifies that work towards implementing the feature and integrating
|
|
it can now happen without interruption. An RFC doesn't have to have a reference
|
|
implementation for it to be accepted.
|
|
|
|
The other possible results of the "final comment period" are to:
|
|
|
|
* *postpone* the RFC (similar to the Deferred status in PEPs),
|
|
* get it *back into discussion* if blocking concerns can be addressed, or
|
|
* *close it* if blocking concerns are not solvable. When an RFC is marked as
|
|
*closed*, there is a 7-day grace period to debate whether it should be closed.
|
|
|
|
In practice registering concerns with an RFC happens very often initially but rarely
|
|
causes for the RFC to be entirely killed.
|
|
|
|
This process scales well for small-contention changes and/or smaller changes. For the
|
|
largest controversial changes the discussion gets unwieldy. This is a topic currently
|
|
(as of August 2018) on the minds of the Rust team (see:
|
|
`"Listening and Trust, part 1" <http://aturon.github.io/2018/05/25/listening-part-1/>`_,
|
|
`"Listening and Trust, part 2" <http://aturon.github.io/2018/06/02/listening-part-2/>`_,
|
|
`"Listening and Trust, part 3" <http://aturon.github.io/2018/06/18/listening-part-3/>`_,
|
|
`"Proposal for a staged RFC process"
|
|
<http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/>`_).
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
Every six weeks the Rust compiler is released with whatever it contained at the time.
|
|
There are no LTS channels or releases yet but this concept is planned to make
|
|
redistributors able to keep up with development better.
|
|
|
|
Every few years a so-called `"Edition"
|
|
<https://rust-lang-nursery.github.io/edition-guide/editions/index.html>`_ is released.
|
|
Those are milestone releases with full sets of updated documentation and tooling. They
|
|
can be backwards incompatible with previous editions. External packages opt into
|
|
breaking changes in their crate metadata. The Rust compiler supports all editions that
|
|
existed prior to its release. Linking between crates of any supported edition is
|
|
possible.
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
The Rust programming language was started by Graydon Hoare who developed it as
|
|
a personal project for a few years. When Mozilla started sponsoring the project,
|
|
the team slowly grew with Graydon as a BDFL-style figure. He `left the project
|
|
<https://www.reddit.com/r/rust/comments/7qels2/i_wonder_why_graydon_hoare_the_author_of_rust/dsqeh1d/>`_
|
|
in 2013. Rust functions without a BDFL since. The RFC process was put in place later.
|
|
Initially some design discussions happened during closed-door weekly video meetings
|
|
which was `shut down
|
|
<https://github.com/rust-lang/meeting-minutes/blob/master/weekly-meetings/2015-05-26.md#future-of-weekly-meeting>`_
|
|
in May 2015 (before the 1.0 release of Rust), organically replaced with open discussion
|
|
and direct influence of teams.
|
|
|
|
The number of teams is growing in time. The number of technical decisions made by the
|
|
core team is decreasing, instead those get delegated to respective teams.
|
|
|
|
The concept of a "final comment period" was introduced to encourage more public
|
|
discussion and enable reacting to a change *about to* being made, instead of having to
|
|
revert a rushed decision that was already made.
|
|
|
|
|
|
OpenStack
|
|
=========
|
|
|
|
The OpenStack Foundation Bylaws lay out the basic structure for
|
|
project governance, with `Article IV
|
|
<https://www.openstack.org/legal/bylaws-of-the-openstack-foundation/>`__
|
|
delegating day-to-day management of the open source project to the
|
|
OpenStack Technical Committee (TC), and `The TC member policy
|
|
<https://www.openstack.org/legal/technical-committee-member-policy/>`__
|
|
defining broadly how the Technical Committee shall be elected. The TC
|
|
publishes a set of more detailed `governance documents
|
|
<https://governance.openstack.org/tc/>`__, including `the TC charter
|
|
<https://governance.openstack.org/tc/reference/charter.html>`__, which
|
|
describes the team structure, precise rules for establishing
|
|
eligibility to run for office, and criteria for establishing the
|
|
various electorates.
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
The OpenStack community is made up of many distinct `project teams
|
|
<https://governance.openstack.org/tc/reference/projects/index.html>`__,
|
|
responsible for producing different components of the software (block
|
|
storage management, compute management, etc.) or managing different
|
|
parts of the processes the community follows (such as tracking the
|
|
release schedule). Each team is led by a *Project Team Lead* (PTL),
|
|
elected by the *Active Project Contributors* for that project.
|
|
|
|
Active Project Contributors (APCs) are recent contributors to a given
|
|
project team. APC status formally requires two things: becoming an
|
|
individual member of the OpenStack Foundation (membership is free) and
|
|
having a change merged within the last year (two development cycles)
|
|
in a repository managed by a project team.
|
|
|
|
The elected PTL serves a term equal to one development cycle (roughly
|
|
6 months). There is no restriction on the number of consecutive terms
|
|
a person may serve as PTL, and it is common for someone to serve for
|
|
several terms in a row. It is also not unusual for a team to have only
|
|
one candidate volunteer to serve as PTL for a given cycle, in which
|
|
case there is no need for an election.
|
|
|
|
The PTL represents the team in all cases except where they have
|
|
explicitly delegated some responsibility. For example, many teams
|
|
designate a separate *release liaison* to manage the release process
|
|
for a development cycle. The PTL also serves as a final decision
|
|
maker in cases where consensus cannot be reached between the team
|
|
members.
|
|
|
|
While the APCs all vote for the PTL of a team, in many other cases
|
|
only the *core reviewer* team will be consulted on policy decisions
|
|
for the team. Anyone may review any patch for any OpenStack
|
|
project. After someone demonstrates that they have a good grasp of the
|
|
technical issues of a project, that they provide useful feedback on
|
|
reviews, and that they understand the direction the project is going,
|
|
they may be invited to become a member of the core review team. Unlike
|
|
in many other communities, this status does not grant them the right
|
|
to submit code without having it reviewed. Rather, it asks them to
|
|
commit to reviewing code written by other contributors, and to
|
|
participate in team decision-making discussions. Asking someone to
|
|
become a member of the core review team is a strong indication of
|
|
trust.
|
|
|
|
The Technical Committee (TC) is responsible for managing the
|
|
development of OpenStack as a whole. The 13 members of the Technical
|
|
Committee are directly elected by APCs from all project teams. Each
|
|
member serves a term of two development cycles (roughly 1 year), with
|
|
the elections split so that only about half of the members' terms
|
|
expire at any time, to ensure continuity. The TC establishes overall
|
|
policies, such as the criteria for adding new project teams, the
|
|
deprecation policy for Python 2, testing requirements, etc.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
All elections for PTL or TC members use https://civs.cs.cornell.edu to
|
|
run a *Condorcet* election. This system was selected because it
|
|
emphasizes consensus candidates over strict popularity.
|
|
|
|
The OpenStack contributor community relies on 3 primary tools for
|
|
discussion: the `openstack-dev mailing list
|
|
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>`__,
|
|
a gerrit code review instance at https://review.openstack.org, and a
|
|
set of `OpenStack-specific IRC channels
|
|
<http://eavesdrop.openstack.org>`__ on Freenode. There are a few teams
|
|
whose contributors are based primarily in China, and they have trouble
|
|
accessing IRC. Those teams tend to use alternative platforms such as
|
|
WeChat, instead.
|
|
|
|
The tool used for discussing any given decision will vary based on its
|
|
weight and impact. Everyone is encouraged to use either the mailing
|
|
list or gerrit to support asynchronous discussion across a wider range
|
|
of timezones and firewalls, especially for publicizing final
|
|
decisions for the rest of the community.
|
|
|
|
Policy decisions limited to a single team are usually made by the core
|
|
review team for a project, and the policies and decision processes may
|
|
vary between teams. Some groups write down their team policies in
|
|
their documentation repository, and use the code review tool (gerrit)
|
|
to vote on them. Some teams discuss policies on IRC, either ad hoc or
|
|
during a regularly scheduled meeting, and make decisions there. Some
|
|
teams use the mailing list for those discussions. The PTL for the team
|
|
is responsible for ensuring the discussion is managed and the outcome
|
|
is communicated (either by doing so directly or ensuring that the task
|
|
is delegated to someone else).
|
|
|
|
All team policy decisions need to be compatible with the overall
|
|
policies set by the Technical Committee. Because the TC tends to make
|
|
broader governance decisions that apply to the entire contributor
|
|
community, the process for discussing and voting on those decisions is
|
|
described more formally, including specifying the number of votes
|
|
needed to pass and the minimum length of time required for
|
|
discussion. For example, most motions require 1/3 of the members (5)
|
|
to pass and must stay open at least 3 days after receiving sufficient
|
|
votes to pass, ensuring that there is time for dissent to be
|
|
registered. See the `Technical Committee Charter
|
|
<https://governance.openstack.org/tc/reference/charter.html#motions>`__
|
|
and `house rules
|
|
<https://governance.openstack.org/tc/reference/house-rules.html>`__
|
|
for more details.
|
|
|
|
Significant design decisions are usually discussed by reviewing a
|
|
`specification document <http://specs.openstack.org>`__, somewhat
|
|
similar to a PEP, that covers the requirements, alternatives, and
|
|
implementation details. Feedback is solicited from all contributors,
|
|
and then specifications are eventually approved or rejected by members
|
|
of the core review team for a project. Some teams require only 2
|
|
reviewers to approve a design, while other teams require a stronger
|
|
indication of consensus before a design is approved. Each team sets a
|
|
`deadline for approving specifications within each development cycle
|
|
<https://releases.openstack.org/rocky/schedule.html>`__, to encourage
|
|
contributors to work out designs for significant new features early
|
|
and avoid risk from changes late in the cycle.
|
|
|
|
Smaller technical decisions are typically made by reviewing the
|
|
patch(es) needed to implement the change. Anyone may review any patch
|
|
and provide technical feedback, but ultimately two core reviewers for
|
|
a team are needed to approve most changes (exceptions are often made
|
|
for trivial changes such as typos or for fixes that unblock the CI
|
|
gating system).
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
Controversial, or merely complicated, decisions frequently expand
|
|
outside of specification reviews to mailing list discussions. They
|
|
often also result in discussions at one of the regularly scheduled
|
|
in-person community gatherings. Because many members of the community
|
|
cannot attend these events, the discussions are summarized and final
|
|
decisions are made using on-line tools as much as possible.
|
|
|
|
The PTL is responsible for deciding when consensus has been reached
|
|
for decisions that affect a single team, and to make a final call in
|
|
rare cases where consensus has not been reached and a decision
|
|
absolutely needs to be made. The TC acts as a similar decision-making
|
|
group of last resort for cases where issues *between* teams cannot be
|
|
resolved in another way. Such escalation of decision-making ends up
|
|
being rarely necessary, because the contributors directly involved
|
|
generally prefer to come to a consensual agreement rather than
|
|
escalate the decision to others.
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
OpenStack has a major release about every 6 months. These are
|
|
coordinated date-based releases, which include the work finished up to
|
|
that point in time in all of the member projects. Some project teams
|
|
release more often than every 6 months (this is especially true for
|
|
teams building libraries consumed by other teams). Those smaller
|
|
releases tend to be produced when there is content (new features or
|
|
bug fixes) to justify them.
|
|
|
|
The schedule for each development cycle, with deadlines and a final
|
|
release date, is proposed by the release management team, in
|
|
coordination with the Foundation staff (releases are generally aligned
|
|
with the calendar of in-person events), and then the community has an
|
|
opportunity to provide feedback before the final dates are set.
|
|
|
|
Decisions about priorities for each development cycle are made at the
|
|
team level and the TC level. Core review teams prioritize internal
|
|
work, such as fixing bugs and implementing new features. The TC
|
|
selects `community goals
|
|
<https://governance.openstack.org/tc/goals/index.html>`__, which
|
|
usually require some amount of work from all teams. Agreeing to these
|
|
priorities at the start of each cycle helps the teams coordinate their
|
|
work, which is especially important because the implementation will
|
|
require reviews from multiple team members.
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
Over the last 8 years the number of OpenStack project teams has grown
|
|
from 2 to 63. The makeup of the Technical Committee has changed to
|
|
accommodate that growth. Originally the TC was made up of PTLs, but as
|
|
the membership grew it became impractical for the group to function
|
|
effectively.
|
|
|
|
The community also used to be organized around "program areas" rather
|
|
than project teams. A program area covered a feature set, such as
|
|
gathering telemetry or managing block storage. This organization
|
|
failed when multiple teams of people wanted to work on the same
|
|
feature set using different solutions. Organizing teams around the
|
|
code they deliver allows different teams to have different
|
|
interpretations of the same requirements. For example, there are now
|
|
several teams working on different deployment tools.
|
|
|
|
|
|
Jupyter
|
|
=======
|
|
|
|
The governance structure is documented in the `Main Governance Document
|
|
<https://github.com/jupyter/governance/blob/master/governance.md>`_
|
|
within the `Jupyter Governance repo <https://github.com/jupyter/governance>`_.
|
|
|
|
The effective governance process grows organically over time as the needs of
|
|
the project evolve. Formal changes to the Governance Document are submitted via
|
|
Pull Request, with an open period for comments. After the open period, a
|
|
Steering Council may call for a vote to ratify the PR changes. Acceptance
|
|
requires a minimum of 80% of the Steering Council to vote and at least 2/3 of
|
|
the vote must be positive. The BDFL can act alone to accept or reject changes
|
|
or override the Steering Council decision; though this would be an extremely
|
|
rare event.
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
The key people in Jupyter's Governance are the BDFL, Fernando Perez, and the
|
|
Steering Council. Contributors can be given a special status of core contributor.
|
|
Some may also be Institutional Contributors, who are individuals who contribute
|
|
to the project as part of their official duties at an Institutional Partner.
|
|
|
|
Fernando Perez, the project founder, is the current and first BDFL. The BDFL
|
|
may serve as long as desired. The `BDFL succession plan <https://github.com/jupyter/governance/blob/master/governance.md#bdfl>`_
|
|
is described in the Main Governance Document. In summary, the BDFL may appoint
|
|
the next BDFL. As a courtesy, it is expected that the BDFL will consult with the
|
|
Steering Council. In the event that the BDFL can not appoint a successor, the
|
|
Steering Council will recommend one.
|
|
|
|
Core contributors are individuals who are given rights, such as commit privileges,
|
|
to act in the best interest of the project within their area of expertise or
|
|
`subproject <https://github.com/jupyter/governance/blob/master/newsubprojects.md>`_.
|
|
An existing core contributor typically recommends someone be given
|
|
core contributor rights by gathering consensus from project leads, who are
|
|
experienced core contributors as listed in the README of the project repo.
|
|
|
|
To be recommended and invited as a Steering Council member, an individual must
|
|
be a Project Contributor who has produced contributions that are substantial in
|
|
quality and quantity, and sustained over at least one year. Potential Council
|
|
Members are nominated by existing Council members and voted upon by the
|
|
existing Council after asking if the potential Member is interested and willing
|
|
to serve in that capacity.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Project Jupyter is made up of a number of GitHub organizations and subprojects
|
|
within those organizations. Primary work happens via GitHub issues and pull
|
|
requests. Approving a pull request by any team member allows it to be merged
|
|
without further process. All merged pull requests end up in the next stable
|
|
release of a subproject.
|
|
|
|
There is a weekly, public Project-wide meeting that is recorded and posted on
|
|
YouTube. Some larger GitHub organizations, which are subprojects of
|
|
Project Jupyter, e.g. JupyterLab and JupyterHub, may
|
|
have additional public team meetings on a weekly or monthly schedule.
|
|
Discussions occur on Gitter, the Jupyter mailing list, and most frequently an
|
|
open issue and/or pull request on GitHub.
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
The foundations of Project Jupyter's governance are:
|
|
|
|
* Openness & Transparency
|
|
* Active Contribution
|
|
* Institutional Neutrality
|
|
|
|
During the everyday project activities, Steering Council members participate in
|
|
all discussions, code review and other project activities as peers with all
|
|
other Contributors and the Community. In these everyday activities,
|
|
Council Members do not have any special power or privilege through their
|
|
membership on the Council. However, it is expected that because of the quality
|
|
and quantity of their contributions and their expert knowledge of the
|
|
Project Software and Services that Council Members will provide useful guidance,
|
|
both technical and in terms of project direction, to potentially less
|
|
experienced contributors.
|
|
|
|
For controversial issues, the contributor community works together to refine
|
|
potential solutions, iterate as necessary, and build consensus by sharing
|
|
information and views constructively and openly. The Steering Council may
|
|
make decisions when regular community discussion doesn't produce consensus
|
|
on an issue in a reasonable time frame.
|
|
|
|
Voting
|
|
------
|
|
|
|
Rarely, if ever, is voting done for technical decisions.
|
|
|
|
For other Project issues, the Steering Council may call for a vote for a
|
|
decision via a Governance PR or email proposal. Acceptance
|
|
requires a minimum of 80% of the Steering Council to vote and at least 2/3 of
|
|
the vote must be positive.
|
|
|
|
The BDFL can act alone to accept or reject changes or override the Steering
|
|
Council decision; though this would be an extremely rare event. As Benevolent,
|
|
the BDFL, in practice chooses to defer that authority to the consensus of the
|
|
community discussion channels and the Steering Council.
|
|
|
|
Planning releases
|
|
-----------------
|
|
|
|
Since Project Jupyter has a number of projects, not just a single project, the
|
|
release planning is largely driven by the core contributors of a project.
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
The process has remained consistent over time, and the approach has served us
|
|
well. Moving forward The Project leadership will consist of a BDFL and
|
|
Steering Council. This governance model was a formalization of what
|
|
the Project was doing (prior to 2015 when the Main Governance Document was
|
|
adopted by the Steering Council), rather than a change in direction.
|
|
|
|
|
|
Django
|
|
======
|
|
|
|
The governance structure is documented in `Organization of the Django Project
|
|
<https://docs.djangoproject.com/en/2.1/internals/organization/>`_.
|
|
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
The project recognizes three kinds of contributors. Members of the
|
|
core team, the Technical Board, and Fellows. Regular core committers
|
|
no longer exercise their "commit bit", instead they rely on pull
|
|
requests being reviewed and accepted. The Technical Board steers
|
|
technical choices. Fellows are hired contractors who triage new
|
|
tickets, review and merge patches from the committers and community,
|
|
including non-trivial ones.
|
|
|
|
Core team members are added by nomination and vote within the core
|
|
team, with technical board veto (so far not exercised). Technical
|
|
board is elected by and from the core team membership every 18 months
|
|
(every major Django release). Sub-teams within the core team are
|
|
self-selected by interest.
|
|
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Most day-to-day decisions are made by Fellows and sometimes other active
|
|
core team members.
|
|
|
|
The core team votes on new members which requires a 4/5 majority of
|
|
votes cast, no quorum requirement. The Technical Board has veto power.
|
|
This power was never exercised
|
|
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
The Technical Board occasionally approves Django
|
|
Enhancement Proposals (DEPs) but those are rare. The DEP process is
|
|
roughly modeled after PEPs and documented in `DEP 1
|
|
<https://github.com/django/deps/blob/main/final/0001-dep-process.rst>`_.
|
|
DEPs are mostly used to design major new features, but also for
|
|
information on general guidelines and process.
|
|
|
|
An idea for a DEP should be first publicly vetted on the
|
|
django-developers mailing list. After it was roughly validated, the
|
|
author forms a team with three roles:
|
|
|
|
* *authors* who write the DEP and steers the discussion;
|
|
* *implementers* who prepare the implementation of the DEP;
|
|
* a *shepherd* who is a core developer and will be the primary reviewer
|
|
of the DEP.
|
|
|
|
The DEP's draft is submitted, assigned a number, and discussed. Authors
|
|
collect feedback and steer discussion as they see fit. Suggested venues
|
|
to avoid endless open-ended discussions are: separate mailing lists,
|
|
Wiki pages, working off of pull requests on the DEP.
|
|
|
|
Once the feedback round is over, the shepherd asks the Technical Board
|
|
for review and pronouncement. The Board can rule on a DEP as a team or
|
|
designate one member to review and decide.
|
|
|
|
In any case where consensus can't be reached, the Technical Board has
|
|
final say. This was never exercised.
|
|
|
|
Differences between DEPs and PEPs
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The main difference is that the entire workflow is based on pull
|
|
requests rather than e-mail. They are pronounced upon by the Technical
|
|
Board. They need to have the key roles identified before submission
|
|
and throughout the process. The *shepherd* role exists to guide a DEP
|
|
to completion without engaging the Technical Board.
|
|
|
|
Those changes to the process make it more distributed and workable in
|
|
a governance model without a BDFL.
|
|
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
Releases are done on a fixed time-based schedule, with a major version
|
|
every 18 months. With paid Fellows to ensure the necessary work gets
|
|
down, on-time releases are routine.
|
|
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
Django originally had two BDFLs: Jacob Kaplan-Moss and Adrian Holovaty.
|
|
They retired (`Adrian's post
|
|
<http://www.holovaty.com/writing/bdfls-retiring/>`_, `Jacob's post
|
|
<https://jacobian.org/writing/retiring-as-bdfls/>`_)
|
|
9 years into the project's history. Following the stepping down,
|
|
the DEP process was defined.
|
|
|
|
|
|
TypeScript
|
|
==========
|
|
|
|
The governance structure is not externally documented besides the
|
|
`CONTRIBUTING.md
|
|
<https://github.com/Microsoft/TypeScript/blob/main/CONTRIBUTING.md>`_
|
|
document in the main TypeScript repository.
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
There is a formal design team and a release management team working at
|
|
Microsoft. The main person behind the project is currently Anders
|
|
Hejlsberg as some of the original members of the team have left the
|
|
company.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Microsoft, where the project is developed, has a strong planning culture
|
|
so development roadmaps are released long in advanced, notes from
|
|
design discussions held at Microsoft get published quickly and meetings
|
|
are sometimes broadcast using Skype.
|
|
|
|
External contributions are encouraged through pull requests on GitHub.
|
|
Suggestions for new use cases or features are given by issues on GitHub.
|
|
This serves like an ad-hoc PEP-like process. There is some discussion
|
|
over social media (Twitter) as well.
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
Hejlsberg is the central figure of the project in terms of language
|
|
design, synthesizing community needs into a cohesive whole. There is
|
|
no formal process to externally contribute to the design of the
|
|
language.
|
|
|
|
The TypeScript team filters through and integrates community
|
|
suggestions. The main advantages of this setup are that there is
|
|
strong and consistent design with dependable scheduling and
|
|
execution. While there is transparency of intentions and plans, the
|
|
disadvantage of this model is that community involvement is limited
|
|
to pull requests and suggestions.
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
Microsoft determines the release schedule, communicates dates and
|
|
features well in advance. Nightly builds are usually stable (with
|
|
a significant portion of users on this release form).
|
|
|
|
Versioned releases are done every 1 - 3 months, with a roadmap available
|
|
on GitHub.
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
TypeScript is likely the first notable project by Microsoft developed
|
|
fully in the open (versus source-available).
|
|
|
|
Open-sourcing of TypeScript by Microsoft was a planned feature from the
|
|
inception of the project. Before the first open release was made, the
|
|
language was driven fully by needs identified by the original teams and
|
|
the early in-house users. The initial open-sourcing happened via
|
|
the now-defunct Microsoft CodePlex platform. It didn't have
|
|
a well-defined routine of accepting external contributions. Community
|
|
engagement rose significantly after the project got moved.
|
|
|
|
|
|
Astropy
|
|
=======
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
The Astropy Project team's responsibilities are spread over many different
|
|
roles [#astropy-team]_, though frequently a person will have several roles.
|
|
|
|
The main body overseeing the Astropy Project is the Astropy
|
|
*Coordination Committee* (CoCo) . Its key roles are dealing with any
|
|
financial issues, approving new packages wanting to join the Astropy
|
|
Project, approving or rejecting *Astropy Proposals for Enhancement*
|
|
(APEs) [#astropy-apes]_, and generally anything that's "leadership"-oriented
|
|
or time-sensitive. As of this writing, the committee has four members,
|
|
and might grow or shrink as the demands on the committee change.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Code-level decisions
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The Astropy Project includes the *core Astropy package* and other
|
|
*affiliated packages*. For the sake of simplicity, we will avoid
|
|
discussing affiliated packages, which can have their own rules.
|
|
Therefore, everything below will concern the core Astropy package.
|
|
|
|
The core Astropy package is organized as *sub-packages*. Each sub-package
|
|
has an official *maintainer* as well as one or more *deputies*, who are
|
|
responsible for ensuring code is reviewed and generally architecting the
|
|
subpackage. Code-level decisions are therefore made in GitHub issues or
|
|
pull requests (PRs), usually on the basis of consensus, moderated by the
|
|
maintainer and deputies of that sub-package.
|
|
|
|
When there is specific disagreement, majority vote of those who are involved
|
|
in the discussion (e.g. PR) determines the winner, with the CoCo called on
|
|
to break ties or mediate disagreements.
|
|
|
|
Non-code decisions
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
Non-code decisions (like sprint scheduling, bugfix release timing, etc)
|
|
are usually announced on the *astropy-dev mailing list* [#astropy-dev]_ with
|
|
a vote-by-message format, or a "if there are no objections"-style message
|
|
for highly uncontroversial items. In general, on astropy-dev the expectation
|
|
is a concrete proposal which other members are welcome to comment or vote on.
|
|
|
|
Voting
|
|
~~~~~~
|
|
|
|
Voting usually involves either using the +1/-1 format on GitHub or the
|
|
astropy-dev mailing list. There, any interested person can vote regardless
|
|
of their official role in the project, or lack thereof. Furthermore, there
|
|
is no veto mechanism for the CoCo to override decisions of the majority.
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
Simpler controversial decisions are generally discussed on the astropy-dev
|
|
mailing list [#astropy-dev]_, and after a reasonable time either there is
|
|
a clear consensus/compromise (this happens most of the time), or the CoCo
|
|
makes a decision to avoid stalling.
|
|
|
|
More complicated decisions follow the APE process, which is modeled after the
|
|
PEP process. Here, the CoCo makes the final decision after a discussion
|
|
period, open to everyone, has passed. Generally the CoCo would follow the
|
|
consensus or majority will.
|
|
|
|
Ethical issues
|
|
~~~~~~~~~~~~~~
|
|
|
|
The Project has an *Ombudsperson* who ensures there is an alternate contact
|
|
for sensitive issues, such as Code of Conduct violations, independent
|
|
from the Coordination Committee. In practice, the CoCo, the Community
|
|
engagement coordinators and the Ombudsperson would work together privately
|
|
to try and communicate with the violator to address the situation.
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
The major release timing is on a fixed schedule (every 6 months); whatever
|
|
is in at that time goes in.
|
|
|
|
Changes in the process over time
|
|
--------------------------------
|
|
|
|
The CoCo and the "Open Development" ethos came from the inception of the
|
|
Project after a series of votes by interested Python-oriented astronomers
|
|
and allied software engineers. The core results of that initial discussion
|
|
were embodied in the *Vision for Astropy* document [#astropy-vision]_.
|
|
|
|
The existence of the formal roles and most of the rest of the above
|
|
came as evolutionary steps as the community grew larger, each following
|
|
either the APE process, or the regular process of a proposal being brought
|
|
up for discussion and vote in astropy-dev [#astropy-dev]_. In general all
|
|
evolved as sort of ratification of already-existing practices, only after
|
|
they were first tested in the wild.
|
|
|
|
Self-appreciation
|
|
-----------------
|
|
|
|
The fact that anyone who has the time can step in and suggest something
|
|
(usually via PR) or vote on their preference, leads to a sense that
|
|
"we are all in this together", leading to better-coordinated effort.
|
|
|
|
Additionally, the function of the CoCo as mostly a tie-breaking body means
|
|
that there's no sense of a dictator who enforces their will, while still
|
|
giving clear points of contact for external organizations that are
|
|
leery of fully-democratic governance models.
|
|
|
|
References
|
|
----------
|
|
|
|
.. [#astropy-team] Astropy roles and responsibilities
|
|
https://www.astropy.org/team.html
|
|
|
|
.. [#astropy-apes] Astropy Proposals for Enhancement
|
|
https://github.com/astropy/astropy-APEs
|
|
|
|
.. [#astropy-dev] Astropy-dev mailing list
|
|
https://groups.google.com/forum/#!forum/astropy-dev
|
|
|
|
.. [#astropy-vision] Vision for a Common Astronomy Python Package
|
|
https://docs.astropy.org/en/stable/development/vision.html
|
|
|
|
|
|
Bonus: Microsoft
|
|
================
|
|
|
|
Despite the selection process for "relevant projects" described above,
|
|
it is worthwhile considering how companies that are held financially
|
|
accountable for their decisions go about making them. This is not
|
|
intended as a readily-usable model for Python, but as additional insight
|
|
that may influence the final design or selection.
|
|
|
|
This section is not taken from any official documentation, but has been
|
|
abstracted by Steve Dower, a current Microsoft employee, to reflect the
|
|
processes that are most applicable to individual projects in the
|
|
engineering departments. Role titles are used (and defined) rather than
|
|
identifying specific individuals, and all names are examples and should
|
|
not be taken as a precise description of the company at any particular
|
|
time in history.
|
|
|
|
This is also highly simplified and idealised. There are plenty of
|
|
unhealthy teams that do not look like this description, and those
|
|
typically have high attrition (people leave the team more frequently
|
|
than other teams). Teams that retain their people are usually closer to
|
|
the model described here, but ultimately everything involving humans is
|
|
imperfect and Microsoft is no exception.
|
|
|
|
Key people and their functions
|
|
------------------------------
|
|
|
|
Microsoft has a hierarchy that ultimately reports to the CEO. Below the
|
|
CEO are a number of organisations, some of which are focused on
|
|
engineering projects (as opposed to sales, marketing or other functions).
|
|
These engineering organisations roughly break down into significant
|
|
product families - for example, there has been a "Windows group", an
|
|
"Xbox group", and a "server and tools group". These are typically led by
|
|
*Executive Vice Presidents* (EVPs), who report to the CEO.
|
|
|
|
Below each EVP are many *Corporate Vice Presidents* (CVPs), each of which
|
|
is responsible for one or more products. This level is where the hierarchy
|
|
becomes relevant for the purposes of this PEP - the CEO and EVPs are
|
|
rarely involved in most decision processes, but set the direction under
|
|
which CVPs make decisions.
|
|
|
|
Each product under a CVP has a team consisting of *Program Managers*
|
|
(PMs) and *Engineering Managers*. Engineering Managers have teams of
|
|
engineers who are largely uninvolved in decision making, though may be
|
|
used as specialists in some cases. For the rest of this section,
|
|
*Engineering* refers to anyone from the engineering team who is
|
|
contributing with a technical-focus, and *PM* refers to anyone from the
|
|
program management team contributing with a customer-focus. After
|
|
decisions are made, Engineering does the implementation and testing work,
|
|
and PM validates with users that their problem has been solved.
|
|
|
|
(This is actually a huge simplification, to the point where some people
|
|
in these roles are offended by this characterisation. In reality, most
|
|
people in PM or Engineering do work that crosses the boundary between
|
|
the two roles, and so they should be treated as a term describing the
|
|
work that somebody is doing in the moment, rather than an identifier or
|
|
restriction for a person.)
|
|
|
|
Teams generally represent a feature, while the CVP represents a product.
|
|
For example, Visual Studio Code has a CVP who is ultimately responsible
|
|
for decisions about that product and its overall direction (in the context
|
|
set by their EVP). But many teams contribute features into Visual Studio
|
|
Code.
|
|
|
|
For complete clarity, the CEO, EVPs, and CVPs do not ever directly
|
|
modify source code. Their entire role is to provide direction for
|
|
whoever is immediately below them and to adjudicate on controversial
|
|
decisions.
|
|
|
|
Regular decision process
|
|
------------------------
|
|
|
|
Changes to product code that are not visible to external users are made
|
|
solely by Engineering. Individual engineers will be assigned tasks by a
|
|
designated engineering manager, or may self-assign. Promotion to
|
|
increasingly senior positions generally reflects trust in the
|
|
individual's decision-making ability, and more senior engineers are
|
|
trusted to make decisions with less validation from the rest of the team.
|
|
Most bugs are covered by this process (that is, fixing a user-visible
|
|
problem without changing the intended experience is an Engineering
|
|
decision).
|
|
|
|
Decisions affecting users of a particular feature are made by the PM
|
|
team for that feature. They will use whatever data sources available to
|
|
identify an issue, experiment with alternatives, and ultimately prepare
|
|
a design document. Senior members from PM and Engineering will review
|
|
designs to clarify the details, and ultimately an artifact is created
|
|
that the feature team agrees on. Engineering will use this artifact to
|
|
implement the work, and PM will later use this artifact to validate that
|
|
the original issue has been resolved.
|
|
|
|
Senior members of Engineering and PM teams for a feature are expected to
|
|
make decisions in the spirit of the direction set by their CVP. Teams
|
|
have regular meetings with their CVP to discuss recent decisions and
|
|
ensure consistency. Decisions that are not obviously in line with CVP
|
|
expectations are escalated to the controversial process.
|
|
|
|
Controversial decision process
|
|
------------------------------
|
|
|
|
When decisions require cross-team coordination, or do not obviously
|
|
align with previous CVP guidance, teams will escalate decision making.
|
|
These often include decisions that involve changing direction,
|
|
attempting to reach a new or different group of users, deprecating and
|
|
removing significant features (or on a short timeframe), or changes that
|
|
require quick releases.
|
|
|
|
In general, CVPs are not intimately familiar with all aspects of the
|
|
feature team's work. As a result, the feature team must provide both a
|
|
recommendation and sufficient context for the decision that the CVP can
|
|
decide *without additional knowledge*. Most of the time, the first
|
|
attempt results in a series of questions from the CVP, which the team
|
|
will research, answer and attempt the decision again at a later date.
|
|
|
|
Common questions asked by CVPs are:
|
|
|
|
* how many users are affected by this decision?
|
|
* what is the plan for minimizing impact on current users?
|
|
* how will the change be "sold"/described to potential users?
|
|
|
|
CVPs are expected to have a strong understanding of the entire field, so
|
|
that they can evaluate some questions for themselves, such as:
|
|
|
|
* what similar decisions have been made by other projects within Microsoft?
|
|
* what other projects have plans that may impact this decision?
|
|
* what similar decisions have been made by projects outside Microsoft?
|
|
* do users need it?
|
|
* is it in line with the direction set by their EVP?
|
|
|
|
Decisions made by CVPs are generally arbitrary and final, though they
|
|
typically will provide their rationale.
|
|
|
|
Planning a new release
|
|
----------------------
|
|
|
|
Releases involve coordinating a number of feature teams, and so rarely
|
|
attempt to include input from all teams. A schedule will be determined
|
|
based on broader ecosystem needs, such as planned events/conferences or
|
|
opportunities to take advantage of media attention.
|
|
|
|
Teams are informed of the release date, the theme of the release, and
|
|
make their own plans around it following the above decision making
|
|
process. Changing the release date is considered a controversial
|
|
decision.
|
|
|
|
|
|
Acknowledgements
|
|
================
|
|
|
|
Thank you to Alex Crichton from the Rust team for an extensive explanation of how the
|
|
core team governs the project.
|
|
|
|
Jeremy Stanley, Chris Dent, Julia Kreger, Sean McGinnis, Emmet Hikory,
|
|
and Thierry Carrez contributed to the OpenStack section.
|
|
|
|
The Project Jupyter Steering Council created the Main Governance Document for
|
|
Project Jupyter, and Carol Willing summarized the key points of that document
|
|
for the Jupyter section.
|
|
|
|
Thank you to Carl Meyer from the Django team for explanation how their
|
|
project's governance is set up.
|
|
|
|
The TypeScript and Swift sections were created after conversations with
|
|
Joe Pamer and Vlad Matveev. Thanks!
|
|
|
|
Answers about the Astropy project were kindly contributed, in significant
|
|
detail, by Erik Tollerud and reviewed by other members of the project.
|
|
|
|
|
|
Annex 1: Template questions
|
|
===========================
|
|
|
|
The following set of questions was used as a template to guide evaluation and
|
|
interaction with the surveyed projects:
|
|
|
|
1. Do you have any open documentation on how the governance model is set up?
|
|
|
|
2. How does the process look like in practice?
|
|
|
|
* Who are the key people?
|
|
* What "special statuses" can contributors have?
|
|
* How are they elected/how are the statuses assigned?
|
|
* How are regular decisions made?
|
|
* How are controversial decisions made?
|
|
* Is there a voting mechanism? how does it work? how often do votes actually happen?
|
|
* Is there a veto mechanism? how often was it actually used?
|
|
|
|
3. How do you like the process?
|
|
|
|
* Which parts work well?
|
|
* Which parts could work better?
|
|
* When it doesn't work well, how does it look like?
|
|
* What would you change if it were only up to you?
|
|
|
|
4. Related project work:
|
|
|
|
* How do you decide when a release happens and what goes into it?
|
|
* How do you decide who gets commit access?
|
|
* Where do you hold discussions? (GitHub, mailing lists, face-to-face meetings, and so on)
|
|
* Do you have a RFC/PEP-like process?
|
|
* Who has access to those discussion channels?
|
|
* How is this access granted/revoked?
|
|
* Who moderates those discussions?
|
|
* Do you (and how) censure participants and how?
|
|
|
|
5. Process evolution
|
|
|
|
* How did this process evolve historically?
|
|
* How can it be changed in the future?
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|