PEP-662: Move to a wheel based approach (#1999)
Co-authored-by: Tomer <tomer.keren.dev@gmail.com>
This commit is contained in:
parent
335d61e924
commit
cab847fb1c
|
@ -517,6 +517,7 @@ pep-0659.rst @markshannon
|
||||||
pep-0660.rst @pfmoore
|
pep-0660.rst @pfmoore
|
||||||
pep-0661.rst @taleinat
|
pep-0661.rst @taleinat
|
||||||
pep-0662.rst @brettcannon
|
pep-0662.rst @brettcannon
|
||||||
|
pep-0662/pep-0662-editables.json @brettcannon
|
||||||
# ...
|
# ...
|
||||||
# pep-0666.txt
|
# pep-0666.txt
|
||||||
# ...
|
# ...
|
||||||
|
|
158
pep-0662.rst
158
pep-0662.rst
|
@ -26,8 +26,7 @@ This mode is usually called "development mode" or "editable installs".
|
||||||
Currently, there is no standardized way to accomplish this, as it was explicitly
|
Currently, there is no standardized way to accomplish this, as it was explicitly
|
||||||
left out of :pep:`517` due to the complexity of the actual observed behaviors.
|
left out of :pep:`517` due to the complexity of the actual observed behaviors.
|
||||||
|
|
||||||
At the moment, users can achieve this in a few ways, neither of them being a
|
At the moment, users to get this behaviour perform one of the following:
|
||||||
standard:
|
|
||||||
|
|
||||||
- For just Python code by adding the relevant source directories to
|
- For just Python code by adding the relevant source directories to
|
||||||
``sys.path`` (configurable from the command line interface via the
|
``sys.path`` (configurable from the command line interface via the
|
||||||
|
@ -58,9 +57,10 @@ installation is:
|
||||||
The author of this PEP believes there's no one size fits all solution here,
|
The author of this PEP believes there's no one size fits all solution here,
|
||||||
each method of achieving editable effect has its pros and cons. Therefore
|
each method of achieving editable effect has its pros and cons. Therefore
|
||||||
this PEP rejects option three as it's unlikely for the community to agree on a
|
this PEP rejects option three as it's unlikely for the community to agree on a
|
||||||
single solution. Therefore, question remains as to whether the frontend or the
|
single solution. Furthermore, question remains as to whether the frontend or the
|
||||||
build backend should own this responsibility. :pep:`660` proposes the build
|
build backend should own this responsibility. :pep:`660` proposes the build
|
||||||
backend to own this, while the current PEP proposes the frontend.
|
backend to own this, while the current PEP proposes primarily the frontend,
|
||||||
|
but still allows the backend to take take control if it wants to do so.
|
||||||
|
|
||||||
Rationale
|
Rationale
|
||||||
=========
|
=========
|
||||||
|
@ -83,18 +83,21 @@ their users. In this proposal, the backend's role is to prepare the project for
|
||||||
an editable installation, and then provide enough information to the frontend
|
an editable installation, and then provide enough information to the frontend
|
||||||
so that the frontend can manifest and enforce the editable installation.
|
so that the frontend can manifest and enforce the editable installation.
|
||||||
|
|
||||||
The information the backend provides to the frontend is:
|
The information the backend provides to the frontend is a wheel that follows
|
||||||
|
the existing specification within :pep:`427`. The wheel metadata about the
|
||||||
|
archive itself (``{distribution}-{version}.dist-info/WHEEL``) must also contain
|
||||||
|
the key ``Editable`` with value of ``true``.
|
||||||
|
|
||||||
- the project metadata (as defined by :pep:`427` under ``.dist-info``),
|
However, instead of providing the project files within the wheel, it must
|
||||||
- the files to expose (formulated as a mapping of absolute source tree
|
provide an ``editable.json`` file (at the root level of the wheel) that defines
|
||||||
paths to relative target interpreter destination paths).
|
the files to be exposed by the frontend. The content of this file is formulated
|
||||||
|
as a mapping of absolute source tree paths to relative target interpreter
|
||||||
|
destination paths within a scheme mapping.
|
||||||
|
|
||||||
We refer to this set of information as the virtual wheel. This virtual wheel
|
A wheel that satisfies the previous two paragraphs is a virtual wheel. The
|
||||||
should contain all information a wheel contains, however it's not zipped and
|
frontend's role is to take the virtual wheel and install the project in
|
||||||
its installation will not be done by copying the files. The frontend's role is
|
editable mode. The way it achieves this is entirely up to the frontend and is
|
||||||
to take the virtual wheel and install the project in editable mode. The way it
|
considered implementation detail.
|
||||||
achieves this is entirely up to the frontend and is considered implementation
|
|
||||||
detail.
|
|
||||||
|
|
||||||
The editable installation mode implies that the source code of the project
|
The editable installation mode implies that the source code of the project
|
||||||
being installed is available in a local directory. Once the project is
|
being installed is available in a local directory. Once the project is
|
||||||
|
@ -116,9 +119,10 @@ installations, this may not always be possible and may be in tension with other
|
||||||
user expectations. Depending on how a frontend implements the editable mode,
|
user expectations. Depending on how a frontend implements the editable mode,
|
||||||
some differences may be visible, such as the presence of additional files
|
some differences may be visible, such as the presence of additional files
|
||||||
(compared to a typical installation), either in the source tree or the
|
(compared to a typical installation), either in the source tree or the
|
||||||
interpreter's installation path. Frontends should seek to minimize differences
|
interpreter's installation path.
|
||||||
between the behavior of editable and standard installations and document known
|
|
||||||
differences.
|
Frontends should seek to minimize differences between the behavior of editable
|
||||||
|
and standard installations and document known differences.
|
||||||
|
|
||||||
For reference, a non-editable installation works as follows:
|
For reference, a non-editable installation works as follows:
|
||||||
|
|
||||||
|
@ -164,39 +168,52 @@ If not defined, the default implementation is equivalent to returning ``[]``.
|
||||||
|
|
||||||
.. code::
|
.. code::
|
||||||
|
|
||||||
def build_editable(config_settings=None):
|
def build_editable(self, wheel_directory, config_settings=None,
|
||||||
|
metadata_directory=None):
|
||||||
...
|
...
|
||||||
|
|
||||||
The function returns an object of type ``EditableInfo`` as defined below:
|
Must build a .whl file, and place it in the specified ``wheel_directory``. It
|
||||||
|
must return the basename (not the full path) of the ``.whl`` file it creates,
|
||||||
|
as a unicode string. The wheel file must be of type virtual wheel as defined
|
||||||
|
under the terminology section.
|
||||||
|
|
||||||
|
If the build frontend has previously called ``prepare_metadata_for_build_wheel``
|
||||||
|
and depends on the wheel resulting from this call to have metadata
|
||||||
|
matching this earlier call, then it should provide the path to the created
|
||||||
|
``.dist-info`` directory as the ``metadata_directory`` argument. If this
|
||||||
|
argument is provided, then ``build_editable`` MUST produce a wheel with identical
|
||||||
|
metadata. The directory passed in by the build frontend MUST be
|
||||||
|
identical to the directory created by ``prepare_metadata_for_build_wheel``,
|
||||||
|
including any unrecognized files it created.
|
||||||
|
|
||||||
|
Backends which do not provide the ``prepare_metadata_for_build_wheel`` hook may
|
||||||
|
either silently ignore the ``metadata_directory`` parameter to ``build_editable``,
|
||||||
|
or else raise an exception when it is set to anything other than ``None``.
|
||||||
|
|
||||||
|
The source directory may be read-only, in such cases the backend may raise an
|
||||||
|
error that the frontend can display to the user. The backend may store intermediate
|
||||||
|
artifacts in cache locations or temporary directories. The presence or absence of
|
||||||
|
any caches should not make a material difference to the final result of the build.
|
||||||
|
|
||||||
|
The content of the ``editable.json`` MUST pass against the following JSON schema:
|
||||||
|
|
||||||
|
.. include:: pep-0662/pep-0662-editable.json
|
||||||
|
:code:
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
.. code::
|
.. code::
|
||||||
|
|
||||||
from typing import Mapping, TypedDict
|
{
|
||||||
|
"version": 1,
|
||||||
class SchemePaths(TypedDict, total=False):
|
"scheme": {
|
||||||
"""
|
"purelib": {"/src/tree/a.py": "tree/a.py"},
|
||||||
Files and folders that should be mapped:
|
"platlib": {},
|
||||||
- key is the absolute source path
|
"data": {"/src/tree/py.typed": "tree/py.typed"},
|
||||||
- value is the relative path within the target interpreters prefix
|
"headers": {},
|
||||||
"""
|
"scripts": {}
|
||||||
|
}
|
||||||
purelib: Mapping[str, str]
|
}
|
||||||
platlib: Mapping[str, str]
|
|
||||||
headers: Mapping[str, str]
|
|
||||||
scripts: Mapping[str, str]
|
|
||||||
data: Mapping[str, str]
|
|
||||||
|
|
||||||
|
|
||||||
class EditableInfo(TypedDict, total=True):
|
|
||||||
version: int
|
|
||||||
"""protocol version of the editable metadata, this PEP defines version 1"""
|
|
||||||
|
|
||||||
metadata_for_build_editable: str
|
|
||||||
"""distribution information of the package as defined by PEP-491"""
|
|
||||||
|
|
||||||
paths: SchemePaths
|
|
||||||
"""files to expose into the target interpreter"""
|
|
||||||
|
|
||||||
|
|
||||||
The scheme paths map from project source absolute paths to target directory
|
The scheme paths map from project source absolute paths to target directory
|
||||||
relative paths. We allow backends to change the project layout from the project
|
relative paths. We allow backends to change the project layout from the project
|
||||||
|
@ -210,32 +227,17 @@ Build frontend requirements
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
The build frontend is responsible for setting up the environment for the build
|
The build frontend is responsible for setting up the environment for the build
|
||||||
backend to generate the necessary information for an editable build. It's also
|
backend to generate the virtual wheel. All recommendations from :pep:`517` for
|
||||||
responsible for communicating with the backend and receiving the
|
the build wheel hook applies here too.
|
||||||
``EditableInfo`` object. All recommendations from :pep:`517` for the build wheel
|
|
||||||
hook applies here too.
|
|
||||||
|
|
||||||
Frontend requirements
|
Frontend requirements
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
The frontend is responsible for ensuring the ``.dist-info`` folder is available
|
The frontend must install the virtual wheel exactly as defined within
|
||||||
at runtime within the target interpreter for the ``importlib.metadata`` and
|
:pep:`427`. Furthermore is responsible for also installing the files defined
|
||||||
``importlib.resources`` modules.
|
witin the ``editable.json`` file. The manner in which it does is left up to
|
||||||
|
the frontend, and is encouraged for the frontend to communicate with the user
|
||||||
The frontend must ensure that all installation requirements specified in the
|
exactly the method choosen, and what limitations that solution will have.
|
||||||
distribution information files are installed as part of the editable
|
|
||||||
installation into the target interpreter. Additionally, the user might also
|
|
||||||
select additional ``extras`` groups that also should be installed as part of the
|
|
||||||
editable installation.
|
|
||||||
|
|
||||||
The frontend also must generate entrypoints, which may be for the console or the
|
|
||||||
GUI. Those entrypoints are defined by the distribution information files, which
|
|
||||||
are generated during the editable installation process.
|
|
||||||
|
|
||||||
The frontend is responsible for generating the ``RECORD`` file based on the
|
|
||||||
object the build backend returns and their chosen editable implementation. For
|
|
||||||
this reason, the uninstallation of editables should not require any special
|
|
||||||
treatment.
|
|
||||||
|
|
||||||
The frontend must create a ``direct_url.json`` file in the ``.dist-info``
|
The frontend must create a ``direct_url.json`` file in the ``.dist-info``
|
||||||
directory of the installed distribution, in compliance with PEP 610. The ``url``
|
directory of the installed distribution, in compliance with PEP 610. The ``url``
|
||||||
|
@ -243,9 +245,8 @@ value must be a ``file://`` URL pointing to the project directory (i.e., the
|
||||||
directory containing ``pyproject.toml``), and the ``dir_info`` value must be
|
directory containing ``pyproject.toml``), and the ``dir_info`` value must be
|
||||||
``{'editable': true}``.
|
``{'editable': true}``.
|
||||||
|
|
||||||
The frontend must not rely on the ``prepare_metadata_for_build_wheel`` hook when
|
The frontend can rely on the ``prepare_metadata_for_build_wheel`` hook when
|
||||||
installing in editable mode. It must instead invoke ``build_editable`` and use
|
installing in editable mode.
|
||||||
the ``.dist-info`` folder returned by that.
|
|
||||||
|
|
||||||
If the frontend concludes it cannot achieve an editable installation with the
|
If the frontend concludes it cannot achieve an editable installation with the
|
||||||
information provided by the build backend it should fail and raise an error to
|
information provided by the build backend it should fail and raise an error to
|
||||||
|
@ -255,25 +256,27 @@ The frontend might implement one or more editable installation mechanisms and
|
||||||
can leave it up to the user the choose one that its optimal to the use case
|
can leave it up to the user the choose one that its optimal to the use case
|
||||||
of the user. For example, pip could add an editable mode flag, and allow the
|
of the user. For example, pip could add an editable mode flag, and allow the
|
||||||
user to choose between ``pth`` files or symlinks (
|
user to choose between ``pth`` files or symlinks (
|
||||||
``pip install -e . --editable=pth`` vs ``pip install -e . --editable=symlink``).
|
``pip install -e . --editable-mode=pth`` vs
|
||||||
|
``pip install -e . --editable-mode=symlink``).
|
||||||
|
|
||||||
Example editable implementations
|
Example editable implementations
|
||||||
--------------------------------
|
--------------------------------
|
||||||
|
|
||||||
To show how this PEP might be used, we'll now present a few case studies. Note
|
To show how this PEP might be used, we'll now present a few case studies. Note
|
||||||
the offered solutions are purely for illustrating purpose.
|
the offered solutions are purely for illustration purpose and are not normative
|
||||||
|
for the frontend/backend.
|
||||||
|
|
||||||
Add the source tree as is to the interpreter
|
Add the source tree as is to the interpreter
|
||||||
''''''''''''''''''''''''''''''''''''''''''''
|
''''''''''''''''''''''''''''''''''''''''''''
|
||||||
|
|
||||||
This is one of the simplest implementations, it will add the source tree as is
|
This is one of the simplest implementations, it will add the source tree as is
|
||||||
into the interpreters scheme paths, the virtual wheel might look like:
|
into the interpreters scheme paths, the ``editable.json`` within the virtual wheel
|
||||||
|
might look like:
|
||||||
|
|
||||||
.. code::
|
.. code::
|
||||||
|
|
||||||
{
|
{
|
||||||
"metadata_for_build_editable": "<dir to dist-info>",
|
{"version": 1, "scheme": {"purelib": {"<project dir>": "<project dir>"}}}
|
||||||
{"scheme": "purelib": {"<project dir>": "<project dir>"}}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
The frontend then could either:
|
The frontend then could either:
|
||||||
|
@ -305,9 +308,8 @@ interpreter startup by adding a ``pth`` file.
|
||||||
.. code::
|
.. code::
|
||||||
|
|
||||||
{
|
{
|
||||||
"metadata_for_build_editable": "<dir to dist-info>",
|
"version": 1,
|
||||||
{
|
"scheme": {
|
||||||
"scheme": {
|
|
||||||
"purelib": {
|
"purelib": {
|
||||||
"<project dir>/.editable/_register_importer.pth": "<project dir>/_register_importer.pth".
|
"<project dir>/.editable/_register_importer.pth": "<project dir>/_register_importer.pth".
|
||||||
"<project dir>/.editable/_editable_importer.py": "<project dir>/_editable_importer.py"
|
"<project dir>/.editable/_editable_importer.py": "<project dir>/_editable_importer.py"
|
||||||
|
|
|
@ -0,0 +1,38 @@
|
||||||
|
{
|
||||||
|
"$schema": "http://json-schema.org/draft-07/schema",
|
||||||
|
"$id": "http://pypa.io/editables.json",
|
||||||
|
"type": "object",
|
||||||
|
"title": "Virtual wheel editable schema.",
|
||||||
|
"required": ["version", "scheme"],
|
||||||
|
"properties": {
|
||||||
|
"version": {
|
||||||
|
"$id": "#/properties/version",
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 1,
|
||||||
|
"title": "The version of the schema."
|
||||||
|
},
|
||||||
|
"scheme": {
|
||||||
|
"$id": "#/properties/scheme",
|
||||||
|
"type": "object",
|
||||||
|
"title": "Files to expose.",
|
||||||
|
"required": ["purelib", "platlib", "data", "headers", "scripts"],
|
||||||
|
"properties": {
|
||||||
|
"purelib": { "$ref": "#/$defs/mapping" },
|
||||||
|
"platlib": { "$ref": "#/$defs/mapping" },
|
||||||
|
"data": { "$ref": "#/$defs/mapping" },
|
||||||
|
"headers": { "$ref": "#/$defs/mapping" },
|
||||||
|
"scripts": { "$ref": "#/$defs/mapping" }
|
||||||
|
},
|
||||||
|
"additionalProperties": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": true,
|
||||||
|
"$defs": {
|
||||||
|
"mapping": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "A mapping of source to target paths. The source is absolute path, the destination is relative path.",
|
||||||
|
"additionalProperties": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
Loading…
Reference in New Issue