python-peps/pep-0676.rst

236 lines
8.8 KiB
ReStructuredText
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

PEP: 676
Title: PEP Infrastructure Process
Author: Adam Turner <python@quite.org.uk>
Sponsor: Mariatta <mariatta@python.org>
Discussions-To: https://discuss.python.org/t/10774
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 01-Nov-2021
Post-History: 23-Sep-2021
Abstract
========
This PEP addresses the infrastructure around rendering PEP files from
reStructuredText_ files to HTML webpages. We aim to specify a self-contained
and maintainable solution for PEP readers, authors, and editors.
Motivation
==========
As of November 2021, Python Enhancement Proposals (PEPs) are rendered in a
multi-system, multi-stage process. A continuous integration (CI) task runs a
docutils_ script to render all PEP files individually. The CI task then uploads
a tar archive to a server, where it is retrieved and rendered into the
`python.org`_ website periodically.
This places a constraint on the `python.org`_ website to handle raw HTML
uploads and handle PEP rendering, and makes the appropriate place to raise
issues unclear in some cases [1]_.
This PEP provides a specification for self-contained rendering of PEPs. This
would:
* reduce the amount of distributed configuration for supporting PEPs
* enable quality-of-life improvements for those who read, write, and review
PEPs
* solve a number of outstanding issues, and lay the path for improvements
* save volunteer maintainers' time
We propose that PEPs are accessed through peps.python.org_ at the top-level
namespace (for example `peps.python.org/pep-0008/`_), and that all custom
tooling to support rendering PEPs is hosted in the `python/peps`_ repository.
Rationale
=========
Simplifying and Centralising Infrastructure
-------------------------------------------
As of November 2021, to locally render a PEP file, a PEP author or editor needs
to create a full local instance of the `python.org`_ website and run a number
of disparate scripts, following documentation_ that lives outside of the
`python/peps`_ repository.
The proposed implementation provides a single Makefile_ and a Python script to
render all PEP files, with options to target a web-server or local filesystem
environment.
Using a single repository to host all tooling will clarify where to raise
issues, reducing volunteer time spent in triage.
Simplified and centralised tooling may also reduce the barrier to entry to
further improvements, as the scope of the PEP rendering infrastructure is well
defined.
Quality-of-Life Improvements and Resolving Issues
-------------------------------------------------
There are several requests for additional features in reading PEPs, such as:
* syntax highlighting [2]_
* use of ``.. code-block::`` directives [2]_
* support for SVG images [3]_
* typographic quotation marks [4]_
* additional footer information [5]_
* intersphinx functionality [6]_
These are "easy wins" from this proposal, and would serve to improve the
quality-of-life for consumers of PEPs (including reviewers and writers). For
example, the reference implementation no longer requires a scheduled render
process to run, instead initiating rendering on every commit to the
`python/peps`_ repository.
Equally, there are a small number of broken items, for example list styles not
being respected or support for updating images being challenging with the
system [7]_. These would be solved by default in the proposal.
Commercial providers such as `Read the Docs`_ can additionally enhance this
experience, for example by providing rendered previews of changes in pull
requests.
Specification
=============
The proposed specification for rendering the PEP files to HTML is as per the
`reference implementation`_.
That the HTML files should be made available under peps.python.org_. The
rendered files may be hosted just as static files, and behind a content
delivery network (CDN).
The following redirect rules must be created from the `python.org`_ domain:
* /peps/ -> https://peps.python.org/
* /dev/peps/ -> https://peps.python.org/
* /peps/(.*)\.html -> https://peps.python.org/$1
* /dev/peps/(.*) -> https://peps.python.org/$1
.. code-block:: nginx
location ~ ^/dev/peps/?(.*)$ {
return 308 https://peps.python.org/$1/;
}
location ~ ^/peps/(.*)\.html$ {
return 308 https://peps.python.org/$1/;
}
location ^/(dev/)?peps(/.*)?$ {
return 308 https://peps.python.org/;
}
Redirects must be implemented to preserve `URL fragments`_ for backward
compatability purposes.
Backwards Compatibility
=======================
Due to server-side redirects to new canonical URLs, there are no backwards
compatability concerns. Links in previously published materials referring to
any old URL scheme will be guaranteed to work.
Security Implications
=====================
No security implications, and the main `python.org`_ website will no longer
take raw HTML uploads, closing a potential threat vector.
How to Teach This
=================
The new canonical URLs will be publicised in the documentation. However, this
is mainly a backend infrastructure change, and there should be minimal
end-user impact.
Reference Implementation
========================
The proposed implementation has been merged into the `python/peps`_ repository
in a series of pull requests [8]_. This automatically renders all PEPs on every
commit.
Rejected Ideas
==============
It would likely be possible to amend the current (as of November 2021)
rendering process to include a lot of the quality-of-life improvements and
issue mitigations mentioned above. However, we do not believe that this would
solve the distributed tooling issue.
It would be possible to use the output from the proposed rendering system and
import it into `python.org`_. We would argue however that this would be the
worst of both worlds, as a great deal of complexity is added, and none is
removed.
Open Issues
===========
None.
Acknowledgements
================
Thanks to Hugo van Kemenade, Pablo Galindo Salgado, and Éric Araujo for support
since April 2020.
Footnotes
=========
.. _documentation: https://pythondotorg.readthedocs.io/pep_generation.html
.. _docutils: https://docutils.sourceforge.io
.. _Makefile: https://www.gnu.org/software/make/manual/make.html#Introduction
.. _peps.python.org: https://peps.python.org/
.. _peps.python.org/pep-0008/: https://peps.python.org/pep-0008/
.. _python.org: https://www.python.org
.. _python/peps: https://github.com/python/peps
.. _Read the Docs: https://readthedocs.org
.. _reStructuredText: https://docutils.sourceforge.io/rst.html
.. _URL fragments: https://url.spec.whatwg.org/#concept-url-fragment
.. [1] For example,
`pythondotorg#1024 <https://github.com/python/pythondotorg/issues/1204>`__,
`pythondotorg#1038 <https://github.com/python/pythondotorg/issues/1038>`__,
`pythondotorg#1387 <https://github.com/python/pythondotorg/issues/1387>`__,
`pythondotorg#1388 <https://github.com/python/pythondotorg/issues/1388>`__,
`pythondotorg#1393 <https://github.com/python/pythondotorg/issues/1393>`__,
`pythondotorg#1564 <https://github.com/python/pythondotorg/issues/1564>`__,
`pythondotorg#1913 <https://github.com/python/pythondotorg/issues/1913>`__,
.. [2] Requested: `pythondotorg#1063 <https://github.com/python/pythondotorg/pull/1063>`__,
`pythondotorg#1206 <https://github.com/python/pythondotorg/issues/1206>`__,
`pythondotorg#1638 <https://github.com/python/pythondotorg/pull/1638>`__,
`peps#159 <https://github.com/python/peps/issues/159>`__,
`comment in peps#1571 <https://github.com/python/peps/pull/1571#discussion_r478701944>`__,
`peps#1577 <https://github.com/python/peps/pull/1577>`__,
.. [3] Requested: `peps#701 <https://github.com/python/peps/issues/701>`__
.. [4] Requested: `peps#165 <https://github.com/python/peps/issues/165>`__
.. [5] Requested: `pythondotorg#1564 <https://github.com/python/pythondotorg/issues/1564>`__
.. [6] Requested: `comment in peps#2 <https://github.com/python/peps/issues/2#issuecomment-339195595>`__
.. [7] As of November 2021, see
`peps#1387 <https://github.com/python/peps/issues/1387>`__,
`pythondotorg#824 <https://github.com/python/pythondotorg/issues/824>`__,
`pythondotorg#1556 <https://github.com/python/pythondotorg/pull/1556>`__,
.. [8] Implementation PRs:
`peps#1930 <https://github.com/python/peps/pull/1930>`__,
`peps#1931 <https://github.com/python/peps/pull/1931>`__,
`peps#1932 <https://github.com/python/peps/pull/1932>`__,
`peps#1933 <https://github.com/python/peps/pull/1933>`__,
`peps#1934 <https://github.com/python/peps/pull/1934>`__
Copyright
=========
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: