From 06ab672e31064f58c0efd54df4b1f9716f806f45 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sat, 2 Mar 2019 12:04:23 +0000 Subject: [PATCH] PEP 517: Add support for self-hosting backends and in-tree hooks (#901) --- pep-0517.txt | 101 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 3 deletions(-) diff --git a/pep-0517.txt b/pep-0517.txt index ea00b34c5..a41530576 100644 --- a/pep-0517.txt +++ b/pep-0517.txt @@ -166,6 +166,43 @@ Where the ``build-backend`` key exists, it takes precedence over Projects may still wish to include a ``setup.py`` for compatibility with tools that do not use this spec. +This PEP also defines a ``backend-path`` key for use in ``pyproject.toml``, see +the "In-Tree Build Backends" section below. This key would be used as follows:: + + [build-system] + # Defined by PEP 518: + requires = ["flit"] + # Defined by this PEP: + build-backend = "local_backend" + backend-path = ["backend"] + + +Build Requirements +================== + +This PEP places a number of additional requirements on the "build requirements" +section of ``pyproject.toml``. These are intended to ensure that projects do +not create impossible to satisfy conditions with their build requirements. + +- Project build requirements will define a directed graph of requirements + (project A needs B to build, B needs C and D, etc.) This graph MUST NOT + contain cycles. If (due to lack of co-ordination between projects, for + example) a cycle is present, front ends MAY refuse to build the project. +- Where build requirements are available as wheels, front ends SHOULD use these + where practical, to avoid deeply nested builds. However front ends MAY have + modes where they do not consider wheels when locating build requirements, and + so projects MUST NOT assume that publishing wheels is sufficient to break a + requirement cycle. +- Front ends SHOULD check explicitly for requirement cycles, and terminate + the build with an informative message if one is found. + +Note in particular that the requirement for no requirement cycles means that +backends wishing to self-host (i.e., building a wheel for a backend uses that +backend for the build) need to make special provision to avoid causing cycles. +Typically this will involve specifying themselves as an in-tree backend, and +avoiding external build dependencies (usually by vendoring them). + + ========================= Build backend interface ========================= @@ -503,6 +540,48 @@ package *authors*, while still allowing *end-users* to open up the hood and apply duct tape when necessary. +In-Tree Build Backends +====================== + +In certain circumstances, projects may wish to include the source code for the +build backend directly in the source tree, rather than referencing the backend +via the ``requires`` key. Two specific situations where this would be expected +are: + +- Backends themselves, which want to use their own features for building + themselves ("self-hosting backends") +- Project-specific backends, typically consisting of a custom wrapper around a + standard backend, where the wrapper is too project-specific to be worth + distributing independently ("in-tree backends") + +Projects can specify that their backend code is hosted in-tree by including the +``backend-path`` key in ``pyproject.toml``. This key contains a list of +directories, which the frontend will add to the start of ``sys.path`` when +loading the backend, and running the backend hooks. + +There are two restrictions on the content of the ``backend-path`` key: + +- Directories in ``backend-path`` are interpreted as relative to the project + root, and MUST refer to a location within the source tree (after relative + paths and symbolic links have been resolved). +- The backend code MUST be loaded from one of the directories specified in + ``backend-path`` (i.e., it is not permitted to specify ``backend-path`` and + *not* have in-tree backend code). + +The first restriction is to ensure that source trees remain self-contained, +and cannot refer to locations outside of the source tree. Frontends SHOULD +check this condition (typically by resolving the location to an absolute path +and resolving symbolic links, and then checking it against the project root), +and fail with an error message if it is violated. + +The ``backend-path`` feature is intended to support the implementation of +in-tree backends, and not to allow configuration of existing backends. The +second restriction above is specifically to ensure that this is how the feature +is used. Front ends MAY enforce this check, but are not required to. Doing so +would typically involve checking the backend's ``__file__`` attribute against +the locations in ``backend-path``. + + ====================== Source distributions ====================== @@ -599,9 +678,25 @@ automatically upgrade packages to the new format: reason, because we are quite tired of it. * Allowing the backend to be imported from files in the source tree would be more consistent with the way Python imports often work. However, not allowing - this prevents confusing errors from clashing module names. Projects which - need to provide their own backend could use a special proxy backend which - loads the hooks from the source tree, so the flexibility is not lost. + this prevents confusing errors from clashing module names. The initial + version of this PEP did not provide a means to allow backends to be + imported from files within the source tree, but the ``backend-path`` key + was added in the next revision to allow projects to opt into this behaviour + if needed. + + +=============================== + Summary of Changes to PEP 517 +=============================== + +The following changes were made to this PEP after the initial reference +implementation was released in pip 19.0. + +* Cycles in build requirements were explicitly prohibited. +* Support for in-tree backends and self-hosting of backends was added by + the introduction of the ``backend-path`` key in the ``[build-system]`` + table. + =================================== Appendix A: Comparison to PEP 516