From 4a33866e32ef4f27bb02d12196690138a1acc75e Mon Sep 17 00:00:00 2001 From: Mariatta Date: Tue, 14 Aug 2018 05:15:35 -0700 Subject: [PATCH] PEP 581: Using GitHub Issues For CPython (#681) The first iteration. --- pep-0581.rst | 377 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 377 insertions(+) create mode 100644 pep-0581.rst diff --git a/pep-0581.rst b/pep-0581.rst new file mode 100644 index 000000000..836860ea1 --- /dev/null +++ b/pep-0581.rst @@ -0,0 +1,377 @@ +PEP: 581 +Title: Using GitHub Issues for CPython +Version: $Revision$ +Last-Modified: $Date$ +Author: Mariatta Wijaya +Discussions-To: ``#pep581`` stream in zulip +Status: Draft +Type: Process +Content-Type: text/x-rst +Created: 20-Jun-2018 + + +Abstract +======== + +This PEP outlines the steps required to migrate Python's issue tracker +from Roundup to using GitHub Issues. + + +Rationale +========= + +CPython's development moved to GitHub on February 2017. All other projects within +The PSF's organization are hosted on GitHub and are using GitHub issues. +CPython is still using Roundup as the issue tracker in bugs.python.org (bpo) [1]_. + +Why GitHub +---------- + +GitHub has a lot of nice features readily available out of the box, that are not +currently available in Roundup / bpo. + +- APIs we can use to build integrations and automations. There are various existing + integrations and applications available from GitHub Marketplace to help with + our workflow. New applications are easily installed and enabled. In addition, + we've had great success with building our own GitHub bots, like miss-islington [14]_, + bedevere [15]_, and the-knights-who-say-ni [16]_. + +- Ability to embed/drag and drop screenshots, debug log files into GitHub pull + requests and issues. + +- Administrators and core developers can edit issues comments and pull requests. + +- Ability to reply to issues and pull request conversations via email. + +- It supports two factor authentication. + +- It supports markdown and emoji. + +- It has a preview tab, so we can see how our comment will be rendered, before + actually posting. + +- It supports voting (reactions). + +- It supports permalink [2]_, allowing us to quote and copy paste + source code easily. + +- Core developers don't have to maintain the issue infrastructure/site, giving + us more time to focus on the development of Python. + +- We can automatically close issues when a PR has been merged [3]_. + +- Lowers the barrier to contribution. With more than 28 million users, an open + source contributor is more likely to already have an account, and familiar + with GitHub interface, making it easier to start contributing. + +- Email notifications contain metadata [4]_, integrated with GMail, and + allows you to systematically filter emails. + +- Provides additional privacy such as offering the user a choice to hide an + email address yet still allow communication with the user through @-mentions. + +Issues with bpo / Roundup +------------------------- + +- Less than five people maintain bpo. Some of them are core developers. + +- It is in Mercurial. Without any CI available, it puts heavy burden on existing + (few) maintainers in terms of reviewing, testing, and applying patches. + +- At its current state, it is not equipped to accept lots of contributions from + people who aren't already familiar with the code base. + +- The upstream Roundup is in Mercurial. There is an open discussion about + moving the source code of bpo to GitHub [5]_. If the source code of + bpo does move to GitHub, it will become difficult to update patches from + upstream. But as long as it is in Mercurial, it is be difficult to maintain + and onboard new contributors. + +- The user interface needs update and redesign. It will require UX/UI research + to keep it up to date with current web standards, including accessibility. + +- Email address is exposed. There is no choice to mask it. + +- There is no REST API available. There is an open issue in Roundup for adding + REST API [6]_. Last activity was in 2016. + +- It sends a number of unnecessary emails and notifications, and it is difficult, + if not impossible, to configure. An example is the nosy email, where we get + email notification whenever someone added themselves as "nosy". + An issue has been filed in upstream Roundup about this since 2012 with + little traction [7]_. + +- Creating an account has been a hassle. We've had reports when people have + trouble creating accounts or logging in. + +Why not GitLab +-------------- + +Had we migrated to GitLab instead of GitHub in 2017, this PEP would have been +titled "Using GitLab Issues for CPython". + +Why not other issue tracker +--------------------------- + +Using another issue tracker will require yet another learning curve, of having +to learn and get used to a different interface. We'll also need to learn and +figure out how to build the integrations with GitHub. + +By using GitHub issues, where the CPython source code is currently hosted, where +pull requests are taking place, we'll be providing consistent experience to +contributors and maintainers, and not having to jump from one interface to another. + +Why not focus on improving Roundup / bpo +---------------------------------------- + +GitHub has many features we like that are already available. We still need to +build out additional integrations and update our bots, but this is something +we already know how to do. + +In order to really improve Roundup / bpo, it needs to first migrate to GitHub, +add CI and bots. As I understand it, there is hesitation because upstream Roundup +is still in Mercurial. Someone more familiar with Roundup / bpo needs +to champion this effort. (I'm not volunteering, I'm sorry). + +I believe the effort of creating and maintaining GitHub integrations and bots +is much less than the effort needed to get Roundup up to speed and then to continue +maintaining it. + + +Migration Plan +============== + +Backup of GitHub data +--------------------- + +This effort has been started and being tracked as an issue in core-workflow +[8]_. We're using GitHub Migrations API [9]_ +to download GitHub data for CPython on daily basis. The archives will be +dropped in an S3 bucket. + +Thanks to Ernest W. Durbin III for working on this. + +Update the CLA host +------------------- + +At the moment the CLA is hosted within bpo. It needs to be updated such that +signing the CLA does not require a bpo account, and it should be hosted outside +of the bpo. + +The current CLA process itself is not ideal. Currently contributors to +devguide, peps, and core-workflow need to sign a CLA, and it requires a bpo +account. A bpo account should not be required for those projects. + +Currently the CLA process involves real human to manually check the records. +Question: Will it be possible to completely automate the CLA process, so +it does not require human intervention? + +Create labels for issue triage +------------------------------ + +In bpo, we currently have the following fields for each issues: + +Type: behavior, crash, compile error, resource usage, security, performance, enhancement. +Components: 2to3, Argument Clinic, asyncio, Build, Cross-build, ctypes, ... +Priority: release blocker, deferred blocker, critical, high, normal, low + +We will create the corresponding labels:: + + type-behavior, type-crash, type-compile error, type-resource usage, ... + + components-2to3, components-argument clinic, components-asyncio, ... + + priority-release blocker, priority-deferred blocker, priority-critical, ... + +In addition, we'll create ``needs triage`` label. + +Create issue templates +---------------------- + +We will create an issue template and add the following headers:: + + --- + Type: behavior | crash | compile error | resource usage (choose one) + Components: 2to3 | Argument Clinic | asyncio | Build | ... (can select more than one) + Priority: release blocker | deferred blocker | critical | ... + Needs backport to: 2.7 | 3.6 | 3.7 + --- + +The idea is to allow the issue creator to help us triage the issue. +The above values are pre-filled in the template. Issue creator will remove texts +that do not apply to their issue. + +Based on the above headers, bedevere-bot can apply the necessary labels to the +issue. If issue creator did not supply the above headers, the bot will apply +``needs triage`` label. At that point it will require a core developer to +properly label the issue. + +We can also take advantage of GitHub's multiple issue template feature. + +Updates to bedevere +------------------- + +Bedevere-bot will need to be updated to recognize the issue headers described above, +and apply the proper labels. + +Bedevere-bot can also copy over the labels to pull requests that correspond to +the issue. + +Update the devguide +------------------- + +Provide explanation in the devguide about new issue workflow and triage labels. + +Add a button in bpo to migrate the issue to GitHub +-------------------------------------------------- + +This will require actual update to the bpo. But I believe this effort needed +is much less than a complete overhaul. + +We will create a button in bpo, that will copy over the issue description +and associated comments into a GitHub issue. + +We should not be moving all open issues to GitHub. Issues with little or no +activity should just be closed. Issues with no decision made for years should +just be closed. + +If a core developer is still interested in the issue, they can indicate so in +the bpo issue, and later use the button to migrate it over to GitHub. + +Make bpo readonly +----------------- + +This should be the final step. Once we start using GitHub issues, make bpo +readonly, not shut it down. +Do not accept new registrations. Do not allow comments or issues to be created. + + +TBD and additional concerns +=========================== + +Expert index +------------ + +At the moment, there is a mechanism to add people in expert index automatically +added to the nosy list. We need to replicate this functionality. + +A GitHub account should not be a requirement +-------------------------------------------- + +Back when it was discussed about moving the CPython codebase from Mercurial +to GitHub [10]_ and [11]_, it was brought up that +we need to still allow uploading patches in bpo, and that a GitHub account should +not be a requirement in order to contribute to Python. + +If bpo is made readonly, we'll need to come up with a different solution to allow +folks to contribute when they don't own a GitHub account. + +One solution is to create a new "python-issues" mailing list, similar to +docs@python.org [12]_ mailing list, to allow people to submit their issues +there. + +Related to this, since the migration to GitHub in 2017, I recall one case +[13]_ where we had one contributor who submitted patch to Mercurial, and +refused to create a GitHub account. Because of this, our bot is unable to detect +whether the have signed CLA. Another person had volunteered to upload his +patch to GitHub. But we still require both people to sign the CLA. + +That particular situation was complicated. It took up five core developers time +to investigate and manually check the CLA, causing confusion. + +Trim off the "Components" list +------------------------------ + +Is the current "components" list still making sense and relevant? +Can we shorten the list? + +Priority list +------------- + +Is the current "priority" list useful? Nick Coghlan noted that perhaps only +``release blocker`` and ``deferred blocker`` are useful. + +Further questions and discussions +--------------------------------- + +There is a dedicated `#pep581 `_ +stream in python.zulipchat.com. + + +Acknowledgements +================ + +Thanks to Guido van Rossum, Brett Cannon, and Nick Coghlan who were consulted +in the early stage and research of this PEP. Their feedback, concerns, input, +and ideas have been valuable. + + +References +========== + +.. [1] bugs.python.org + (https://bugs.python.org/) + +.. [2] Getting permanent links to files + https://help.github.com/articles/getting-permanent-links-to-files/ + +.. [3] Closing issues using keywords + (https://help.github.com/articles/closing-issues-using-keywords/) + +.. [4] About GitHub email notifications + (https://help.github.com/articles/about-email-notifications/) + +.. [5] Consider whether or not to migrate bugs.python.org source code + to GitHub repo + https://github.com/python/bugs.python.org/issues/2 + +.. [6] Roundup issue 2550734 Expose roundup via a RESTful interface + (http://issues.roundup-tracker.org/issue2550734) + +.. [7] Roundup issue 2550742 Do not send email by default when adding + or removing oneself from the Nosy list + (http://issues.roundup-tracker.org/issue2550742) + +.. [8] Backup GitHub information + (https://github.com/python/core-workflow/issues/20) + +.. [9] GitHub Migrations API + (https://developer.github.com/v3/migrations/orgs/) + +.. [10] Python-committers email + (https://mail.python.org/pipermail/python-committers/2015-December/003642.html) + +.. [11] Python-committers email + (https://mail.python.org/pipermail/python-committers/2015-December/003645.html) + +.. [12] docs mailing list + (https://mail.python.org/mailman/listinfo/docs) + +.. [13] CPython GitHub Pull request 1505 + (https://github.com/python/cpython/pull/1505) + +.. [14] miss-islington + (https://github.com/python/miss-islington) + +.. [15] bedevere + (https://github.com/python/bedevere) + +.. [16] the-knights-who-say-ni + (https://github.com/python/the-knights-who-say-ni) + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: