PEP-662: Move to a wheel based approach (#1999)
Co-authored-by: Tomer <tomer.keren.dev@gmail.com>
This commit is contained in:
parent
335d61e924
commit
cab847fb1c
|
@ -517,6 +517,7 @@ pep-0659.rst @markshannon
|
|||
pep-0660.rst @pfmoore
|
||||
pep-0661.rst @taleinat
|
||||
pep-0662.rst @brettcannon
|
||||
pep-0662/pep-0662-editables.json @brettcannon
|
||||
# ...
|
||||
# pep-0666.txt
|
||||
# ...
|
||||
|
|
156
pep-0662.rst
156
pep-0662.rst
|
@ -26,8 +26,7 @@ This mode is usually called "development mode" or "editable installs".
|
|||
Currently, there is no standardized way to accomplish this, as it was explicitly
|
||||
left out of :pep:`517` due to the complexity of the actual observed behaviors.
|
||||
|
||||
At the moment, users can achieve this in a few ways, neither of them being a
|
||||
standard:
|
||||
At the moment, users to get this behaviour perform one of the following:
|
||||
|
||||
- For just Python code by adding the relevant source directories to
|
||||
``sys.path`` (configurable from the command line interface via the
|
||||
|
@ -58,9 +57,10 @@ installation is:
|
|||
The author of this PEP believes there's no one size fits all solution here,
|
||||
each method of achieving editable effect has its pros and cons. Therefore
|
||||
this PEP rejects option three as it's unlikely for the community to agree on a
|
||||
single solution. Therefore, question remains as to whether the frontend or the
|
||||
single solution. Furthermore, question remains as to whether the frontend or the
|
||||
build backend should own this responsibility. :pep:`660` proposes the build
|
||||
backend to own this, while the current PEP proposes the frontend.
|
||||
backend to own this, while the current PEP proposes primarily the frontend,
|
||||
but still allows the backend to take take control if it wants to do so.
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
@ -83,18 +83,21 @@ their users. In this proposal, the backend's role is to prepare the project for
|
|||
an editable installation, and then provide enough information to the frontend
|
||||
so that the frontend can manifest and enforce the editable installation.
|
||||
|
||||
The information the backend provides to the frontend is:
|
||||
The information the backend provides to the frontend is a wheel that follows
|
||||
the existing specification within :pep:`427`. The wheel metadata about the
|
||||
archive itself (``{distribution}-{version}.dist-info/WHEEL``) must also contain
|
||||
the key ``Editable`` with value of ``true``.
|
||||
|
||||
- the project metadata (as defined by :pep:`427` under ``.dist-info``),
|
||||
- the files to expose (formulated as a mapping of absolute source tree
|
||||
paths to relative target interpreter destination paths).
|
||||
However, instead of providing the project files within the wheel, it must
|
||||
provide an ``editable.json`` file (at the root level of the wheel) that defines
|
||||
the files to be exposed by the frontend. The content of this file is formulated
|
||||
as a mapping of absolute source tree paths to relative target interpreter
|
||||
destination paths within a scheme mapping.
|
||||
|
||||
We refer to this set of information as the virtual wheel. This virtual wheel
|
||||
should contain all information a wheel contains, however it's not zipped and
|
||||
its installation will not be done by copying the files. The frontend's role is
|
||||
to take the virtual wheel and install the project in editable mode. The way it
|
||||
achieves this is entirely up to the frontend and is considered implementation
|
||||
detail.
|
||||
A wheel that satisfies the previous two paragraphs is a virtual wheel. The
|
||||
frontend's role is to take the virtual wheel and install the project in
|
||||
editable mode. The way it achieves this is entirely up to the frontend and is
|
||||
considered implementation detail.
|
||||
|
||||
The editable installation mode implies that the source code of the project
|
||||
being installed is available in a local directory. Once the project is
|
||||
|
@ -116,9 +119,10 @@ installations, this may not always be possible and may be in tension with other
|
|||
user expectations. Depending on how a frontend implements the editable mode,
|
||||
some differences may be visible, such as the presence of additional files
|
||||
(compared to a typical installation), either in the source tree or the
|
||||
interpreter's installation path. Frontends should seek to minimize differences
|
||||
between the behavior of editable and standard installations and document known
|
||||
differences.
|
||||
interpreter's installation path.
|
||||
|
||||
Frontends should seek to minimize differences between the behavior of editable
|
||||
and standard installations and document known differences.
|
||||
|
||||
For reference, a non-editable installation works as follows:
|
||||
|
||||
|
@ -164,39 +168,52 @@ If not defined, the default implementation is equivalent to returning ``[]``.
|
|||
|
||||
.. code::
|
||||
|
||||
def build_editable(config_settings=None):
|
||||
def build_editable(self, wheel_directory, config_settings=None,
|
||||
metadata_directory=None):
|
||||
...
|
||||
|
||||
The function returns an object of type ``EditableInfo`` as defined below:
|
||||
Must build a .whl file, and place it in the specified ``wheel_directory``. It
|
||||
must return the basename (not the full path) of the ``.whl`` file it creates,
|
||||
as a unicode string. The wheel file must be of type virtual wheel as defined
|
||||
under the terminology section.
|
||||
|
||||
If the build frontend has previously called ``prepare_metadata_for_build_wheel``
|
||||
and depends on the wheel resulting from this call to have metadata
|
||||
matching this earlier call, then it should provide the path to the created
|
||||
``.dist-info`` directory as the ``metadata_directory`` argument. If this
|
||||
argument is provided, then ``build_editable`` MUST produce a wheel with identical
|
||||
metadata. The directory passed in by the build frontend MUST be
|
||||
identical to the directory created by ``prepare_metadata_for_build_wheel``,
|
||||
including any unrecognized files it created.
|
||||
|
||||
Backends which do not provide the ``prepare_metadata_for_build_wheel`` hook may
|
||||
either silently ignore the ``metadata_directory`` parameter to ``build_editable``,
|
||||
or else raise an exception when it is set to anything other than ``None``.
|
||||
|
||||
The source directory may be read-only, in such cases the backend may raise an
|
||||
error that the frontend can display to the user. The backend may store intermediate
|
||||
artifacts in cache locations or temporary directories. The presence or absence of
|
||||
any caches should not make a material difference to the final result of the build.
|
||||
|
||||
The content of the ``editable.json`` MUST pass against the following JSON schema:
|
||||
|
||||
.. include:: pep-0662/pep-0662-editable.json
|
||||
:code:
|
||||
|
||||
For example:
|
||||
|
||||
.. code::
|
||||
|
||||
from typing import Mapping, TypedDict
|
||||
|
||||
class SchemePaths(TypedDict, total=False):
|
||||
"""
|
||||
Files and folders that should be mapped:
|
||||
- key is the absolute source path
|
||||
- value is the relative path within the target interpreters prefix
|
||||
"""
|
||||
|
||||
purelib: Mapping[str, str]
|
||||
platlib: Mapping[str, str]
|
||||
headers: Mapping[str, str]
|
||||
scripts: Mapping[str, str]
|
||||
data: Mapping[str, str]
|
||||
|
||||
|
||||
class EditableInfo(TypedDict, total=True):
|
||||
version: int
|
||||
"""protocol version of the editable metadata, this PEP defines version 1"""
|
||||
|
||||
metadata_for_build_editable: str
|
||||
"""distribution information of the package as defined by PEP-491"""
|
||||
|
||||
paths: SchemePaths
|
||||
"""files to expose into the target interpreter"""
|
||||
|
||||
{
|
||||
"version": 1,
|
||||
"scheme": {
|
||||
"purelib": {"/src/tree/a.py": "tree/a.py"},
|
||||
"platlib": {},
|
||||
"data": {"/src/tree/py.typed": "tree/py.typed"},
|
||||
"headers": {},
|
||||
"scripts": {}
|
||||
}
|
||||
}
|
||||
|
||||
The scheme paths map from project source absolute paths to target directory
|
||||
relative paths. We allow backends to change the project layout from the project
|
||||
|
@ -210,32 +227,17 @@ Build frontend requirements
|
|||
---------------------------
|
||||
|
||||
The build frontend is responsible for setting up the environment for the build
|
||||
backend to generate the necessary information for an editable build. It's also
|
||||
responsible for communicating with the backend and receiving the
|
||||
``EditableInfo`` object. All recommendations from :pep:`517` for the build wheel
|
||||
hook applies here too.
|
||||
backend to generate the virtual wheel. All recommendations from :pep:`517` for
|
||||
the build wheel hook applies here too.
|
||||
|
||||
Frontend requirements
|
||||
---------------------
|
||||
|
||||
The frontend is responsible for ensuring the ``.dist-info`` folder is available
|
||||
at runtime within the target interpreter for the ``importlib.metadata`` and
|
||||
``importlib.resources`` modules.
|
||||
|
||||
The frontend must ensure that all installation requirements specified in the
|
||||
distribution information files are installed as part of the editable
|
||||
installation into the target interpreter. Additionally, the user might also
|
||||
select additional ``extras`` groups that also should be installed as part of the
|
||||
editable installation.
|
||||
|
||||
The frontend also must generate entrypoints, which may be for the console or the
|
||||
GUI. Those entrypoints are defined by the distribution information files, which
|
||||
are generated during the editable installation process.
|
||||
|
||||
The frontend is responsible for generating the ``RECORD`` file based on the
|
||||
object the build backend returns and their chosen editable implementation. For
|
||||
this reason, the uninstallation of editables should not require any special
|
||||
treatment.
|
||||
The frontend must install the virtual wheel exactly as defined within
|
||||
:pep:`427`. Furthermore is responsible for also installing the files defined
|
||||
witin the ``editable.json`` file. The manner in which it does is left up to
|
||||
the frontend, and is encouraged for the frontend to communicate with the user
|
||||
exactly the method choosen, and what limitations that solution will have.
|
||||
|
||||
The frontend must create a ``direct_url.json`` file in the ``.dist-info``
|
||||
directory of the installed distribution, in compliance with PEP 610. The ``url``
|
||||
|
@ -243,9 +245,8 @@ value must be a ``file://`` URL pointing to the project directory (i.e., the
|
|||
directory containing ``pyproject.toml``), and the ``dir_info`` value must be
|
||||
``{'editable': true}``.
|
||||
|
||||
The frontend must not rely on the ``prepare_metadata_for_build_wheel`` hook when
|
||||
installing in editable mode. It must instead invoke ``build_editable`` and use
|
||||
the ``.dist-info`` folder returned by that.
|
||||
The frontend can rely on the ``prepare_metadata_for_build_wheel`` hook when
|
||||
installing in editable mode.
|
||||
|
||||
If the frontend concludes it cannot achieve an editable installation with the
|
||||
information provided by the build backend it should fail and raise an error to
|
||||
|
@ -255,25 +256,27 @@ The frontend might implement one or more editable installation mechanisms and
|
|||
can leave it up to the user the choose one that its optimal to the use case
|
||||
of the user. For example, pip could add an editable mode flag, and allow the
|
||||
user to choose between ``pth`` files or symlinks (
|
||||
``pip install -e . --editable=pth`` vs ``pip install -e . --editable=symlink``).
|
||||
``pip install -e . --editable-mode=pth`` vs
|
||||
``pip install -e . --editable-mode=symlink``).
|
||||
|
||||
Example editable implementations
|
||||
--------------------------------
|
||||
|
||||
To show how this PEP might be used, we'll now present a few case studies. Note
|
||||
the offered solutions are purely for illustrating purpose.
|
||||
the offered solutions are purely for illustration purpose and are not normative
|
||||
for the frontend/backend.
|
||||
|
||||
Add the source tree as is to the interpreter
|
||||
''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
This is one of the simplest implementations, it will add the source tree as is
|
||||
into the interpreters scheme paths, the virtual wheel might look like:
|
||||
into the interpreters scheme paths, the ``editable.json`` within the virtual wheel
|
||||
might look like:
|
||||
|
||||
.. code::
|
||||
|
||||
{
|
||||
"metadata_for_build_editable": "<dir to dist-info>",
|
||||
{"scheme": "purelib": {"<project dir>": "<project dir>"}}
|
||||
{"version": 1, "scheme": {"purelib": {"<project dir>": "<project dir>"}}}
|
||||
}
|
||||
|
||||
The frontend then could either:
|
||||
|
@ -305,8 +308,7 @@ interpreter startup by adding a ``pth`` file.
|
|||
.. code::
|
||||
|
||||
{
|
||||
"metadata_for_build_editable": "<dir to dist-info>",
|
||||
{
|
||||
"version": 1,
|
||||
"scheme": {
|
||||
"purelib": {
|
||||
"<project dir>/.editable/_register_importer.pth": "<project dir>/_register_importer.pth".
|
||||
|
|
|
@ -0,0 +1,38 @@
|
|||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema",
|
||||
"$id": "http://pypa.io/editables.json",
|
||||
"type": "object",
|
||||
"title": "Virtual wheel editable schema.",
|
||||
"required": ["version", "scheme"],
|
||||
"properties": {
|
||||
"version": {
|
||||
"$id": "#/properties/version",
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 1,
|
||||
"title": "The version of the schema."
|
||||
},
|
||||
"scheme": {
|
||||
"$id": "#/properties/scheme",
|
||||
"type": "object",
|
||||
"title": "Files to expose.",
|
||||
"required": ["purelib", "platlib", "data", "headers", "scripts"],
|
||||
"properties": {
|
||||
"purelib": { "$ref": "#/$defs/mapping" },
|
||||
"platlib": { "$ref": "#/$defs/mapping" },
|
||||
"data": { "$ref": "#/$defs/mapping" },
|
||||
"headers": { "$ref": "#/$defs/mapping" },
|
||||
"scripts": { "$ref": "#/$defs/mapping" }
|
||||
},
|
||||
"additionalProperties": true
|
||||
}
|
||||
},
|
||||
"additionalProperties": true,
|
||||
"$defs": {
|
||||
"mapping": {
|
||||
"type": "object",
|
||||
"description": "A mapping of source to target paths. The source is absolute path, the destination is relative path.",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
Loading…
Reference in New Issue