iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP-662: Move to a wheel based approach (#1999) 

Co-authored-by: Tomer <tomer.keren.dev@gmail.com>
This commit is contained in:
Bernát Gábor 2021-06-24 21:38:37 +01:00 committed by GitHub
parent 335d61e924
commit cab847fb1c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 119 additions and 78 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.github/CODEOWNERS vendored
View File

@ -517,6 +517,7 @@ pep-0659.rst @markshannon
pep-0660.rst @pfmoore
pep-0661.rst @taleinat
pep-0662.rst @brettcannon
pep-0662/pep-0662-editables.json @brettcannon
# ...
# pep-0666.txt
# ...

158
pep-0662.rst
View File

@ -26,8 +26,7 @@ This mode is usually called "development mode" or "editable installs".
Currently, there is no standardized way to accomplish this, as it was explicitly
left out of :pep:`517` due to the complexity of the actual observed behaviors.
At the moment, users can achieve this in a few ways, neither of them being a
standard:
At the moment, users to get this behaviour perform one of the following:
- For just Python code by adding the relevant source directories to
``sys.path`` (configurable from the command line interface via the
@ -58,9 +57,10 @@ installation is:
The author of this PEP believes there's no one size fits all solution here,
each method of achieving editable effect has its pros and cons. Therefore
this PEP rejects option three as it's unlikely for the community to agree on a
single solution. Therefore, question remains as to whether the frontend or the
single solution. Furthermore, question remains as to whether the frontend or the
build backend should own this responsibility. :pep:`660` proposes the build
backend to own this, while the current PEP proposes the frontend.
backend to own this, while the current PEP proposes primarily the frontend,
but still allows the backend to take take control if it wants to do so.
Rationale
=========
@ -83,18 +83,21 @@ their users. In this proposal, the backend's role is to prepare the project for
an editable installation, and then provide enough information to the frontend
so that the frontend can manifest and enforce the editable installation.
The information the backend provides to the frontend is:
The information the backend provides to the frontend is a wheel that follows
the existing specification within :pep:`427`. The wheel metadata about the
archive itself (``{distribution}-{version}.dist-info/WHEEL``) must also contain
the key ``Editable`` with value of ``true``.
- the project metadata (as defined by :pep:`427` under ``.dist-info``),
- the files to expose (formulated as a mapping of absolute source tree
paths to relative target interpreter destination paths).
However, instead of providing the project files within the wheel, it must
provide an ``editable.json`` file (at the root level of the wheel) that defines
the files to be exposed by the frontend. The content of this file is formulated
as a mapping of absolute source tree paths to relative target interpreter
destination paths within a scheme mapping.
We refer to this set of information as the virtual wheel. This virtual wheel
should contain all information a wheel contains, however it's not zipped and
its installation will not be done by copying the files. The frontend's role is
to take the virtual wheel and install the project in editable mode. The way it
achieves this is entirely up to the frontend and is considered implementation
detail.
A wheel that satisfies the previous two paragraphs is a virtual wheel. The
frontend's role is to take the virtual wheel and install the project in
editable mode. The way it achieves this is entirely up to the frontend and is
considered implementation detail.
The editable installation mode implies that the source code of the project
being installed is available in a local directory. Once the project is
@ -116,9 +119,10 @@ installations, this may not always be possible and may be in tension with other
user expectations. Depending on how a frontend implements the editable mode,
some differences may be visible, such as the presence of additional files
(compared to a typical installation), either in the source tree or the
interpreter's installation path. Frontends should seek to minimize differences
between the behavior of editable and standard installations and document known
differences.
interpreter's installation path.
Frontends should seek to minimize differences between the behavior of editable
and standard installations and document known differences.
For reference, a non-editable installation works as follows:
@ -164,39 +168,52 @@ If not defined, the default implementation is equivalent to returning ``[]``.
.. code::
def build_editable(config_settings=None):
def build_editable(self, wheel_directory, config_settings=None,
metadata_directory=None):
...
The function returns an object of type ``EditableInfo`` as defined below:
Must build a .whl file, and place it in the specified ``wheel_directory``. It
must return the basename (not the full path) of the ``.whl`` file it creates,
as a unicode string. The wheel file must be of type virtual wheel as defined
under the terminology section.
If the build frontend has previously called ``prepare_metadata_for_build_wheel``
and depends on the wheel resulting from this call to have metadata
matching this earlier call, then it should provide the path to the created
``.dist-info`` directory as the ``metadata_directory`` argument. If this
argument is provided, then ``build_editable`` MUST produce a wheel with identical
metadata. The directory passed in by the build frontend MUST be
identical to the directory created by ``prepare_metadata_for_build_wheel``,
including any unrecognized files it created.
Backends which do not provide the ``prepare_metadata_for_build_wheel`` hook may
either silently ignore the ``metadata_directory`` parameter to ``build_editable``,
or else raise an exception when it is set to anything other than ``None``.
The source directory may be read-only, in such cases the backend may raise an
error that the frontend can display to the user. The backend may store intermediate
artifacts in cache locations or temporary directories. The presence or absence of
any caches should not make a material difference to the final result of the build.
The content of the ``editable.json`` MUST pass against the following JSON schema:
.. include:: pep-0662/pep-0662-editable.json
:code:
For example:
.. code::
from typing import Mapping, TypedDict
class SchemePaths(TypedDict, total=False):
"""
Files and folders that should be mapped:
- key is the absolute source path
- value is the relative path within the target interpreters prefix
"""
purelib: Mapping[str, str]
platlib: Mapping[str, str]
headers: Mapping[str, str]
scripts: Mapping[str, str]
data: Mapping[str, str]
class EditableInfo(TypedDict, total=True):
version: int
"""protocol version of the editable metadata, this PEP defines version 1"""
metadata_for_build_editable: str
"""distribution information of the package as defined by PEP-491"""
paths: SchemePaths
"""files to expose into the target interpreter"""
{
"version": 1,
"scheme": {
"purelib": {"/src/tree/a.py": "tree/a.py"},
"platlib": {},
"data": {"/src/tree/py.typed": "tree/py.typed"},
"headers": {},
"scripts": {}
}
}
The scheme paths map from project source absolute paths to target directory
relative paths. We allow backends to change the project layout from the project
@ -210,32 +227,17 @@ Build frontend requirements
---------------------------
The build frontend is responsible for setting up the environment for the build
backend to generate the necessary information for an editable build. It's also
responsible for communicating with the backend and receiving the
``EditableInfo`` object. All recommendations from :pep:`517` for the build wheel
hook applies here too.
backend to generate the virtual wheel. All recommendations from :pep:`517` for
the build wheel hook applies here too.
Frontend requirements
---------------------
The frontend is responsible for ensuring the ``.dist-info`` folder is available
at runtime within the target interpreter for the ``importlib.metadata`` and
``importlib.resources`` modules.
The frontend must ensure that all installation requirements specified in the
distribution information files are installed as part of the editable
installation into the target interpreter. Additionally, the user might also
select additional ``extras`` groups that also should be installed as part of the
editable installation.
The frontend also must generate entrypoints, which may be for the console or the
GUI. Those entrypoints are defined by the distribution information files, which
are generated during the editable installation process.
The frontend is responsible for generating the ``RECORD`` file based on the
object the build backend returns and their chosen editable implementation. For
this reason, the uninstallation of editables should not require any special
treatment.
The frontend must install the virtual wheel exactly as defined within
:pep:`427`. Furthermore is responsible for also installing the files defined
witin the ``editable.json`` file. The manner in which it does is left up to
the frontend, and is encouraged for the frontend to communicate with the user
exactly the method choosen, and what limitations that solution will have.
The frontend must create a ``direct_url.json`` file in the ``.dist-info``
directory of the installed distribution, in compliance with PEP 610. The ``url``
@ -243,9 +245,8 @@ value must be a ``file://`` URL pointing to the project directory (i.e., the
directory containing ``pyproject.toml``), and the ``dir_info`` value must be
``{'editable': true}``.
The frontend must not rely on the ``prepare_metadata_for_build_wheel`` hook when
installing in editable mode. It must instead invoke ``build_editable`` and use
the ``.dist-info`` folder returned by that.
The frontend can rely on the ``prepare_metadata_for_build_wheel`` hook when
installing in editable mode.
If the frontend concludes it cannot achieve an editable installation with the
information provided by the build backend it should fail and raise an error to
@ -255,25 +256,27 @@ The frontend might implement one or more editable installation mechanisms and
can leave it up to the user the choose one that its optimal to the use case
of the user. For example, pip could add an editable mode flag, and allow the
user to choose between ``pth`` files or symlinks (
``pip install -e . --editable=pth`` vs ``pip install -e . --editable=symlink``).
``pip install -e . --editable-mode=pth`` vs
``pip install -e . --editable-mode=symlink``).
Example editable implementations
--------------------------------
To show how this PEP might be used, we'll now present a few case studies. Note
the offered solutions are purely for illustrating purpose.
the offered solutions are purely for illustration purpose and are not normative
for the frontend/backend.
Add the source tree as is to the interpreter
''''''''''''''''''''''''''''''''''''''''''''
This is one of the simplest implementations, it will add the source tree as is
into the interpreters scheme paths, the virtual wheel might look like:
into the interpreters scheme paths, the ``editable.json`` within the virtual wheel
might look like:
.. code::
{
"metadata_for_build_editable": "<dir to dist-info>",
{"scheme": "purelib": {"<project dir>": "<project dir>"}}
{"version": 1, "scheme": {"purelib": {"<project dir>": "<project dir>"}}}
}
The frontend then could either:
@ -305,9 +308,8 @@ interpreter startup by adding a ``pth`` file.
.. code::
{
"metadata_for_build_editable": "<dir to dist-info>",
{
"scheme": {
"version": 1,
"scheme": {
"purelib": {
"<project dir>/.editable/_register_importer.pth": "<project dir>/_register_importer.pth".
"<project dir>/.editable/_editable_importer.py": "<project dir>/_editable_importer.py"

38
pep-0662/pep-0662-editable.json Normal file
View File

@ -0,0 +1,38 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "http://pypa.io/editables.json",
"type": "object",
"title": "Virtual wheel editable schema.",
"required": ["version", "scheme"],
"properties": {
"version": {
"$id": "#/properties/version",
"type": "integer",
"minimum": 1,
"maximum": 1,
"title": "The version of the schema."
},
"scheme": {
"$id": "#/properties/scheme",
"type": "object",
"title": "Files to expose.",
"required": ["purelib", "platlib", "data", "headers", "scripts"],
"properties": {
"purelib": { "$ref": "#/$defs/mapping" },
"platlib": { "$ref": "#/$defs/mapping" },
"data": { "$ref": "#/$defs/mapping" },
"headers": { "$ref": "#/$defs/mapping" },
"scripts": { "$ref": "#/$defs/mapping" }
},
"additionalProperties": true
}
},
"additionalProperties": true,
"$defs": {
"mapping": {
"type": "object",
"description": "A mapping of source to target paths. The source is absolute path, the destination is relative path.",
"additionalProperties": true
}
}
}