PEP 735: apply community feedback, including reversion of [project] table changes (#3990)
This commit is contained in:
Stephen Rosen
GitHub
parent
23530a8814
commit
79a41d1002
|
|
@ -51,6 +51,8 @@ which this PEP seeks to support:
|
|||
|
||||
* non-package projects, such as data science projects
|
||||
|
||||
Several motivating use cases are defined in detail in the :ref:`Use Cases Appendix <use_cases>`.
|
||||
|
||||
Limitations of ``requirements.txt`` files
|
||||
-----------------------------------------
|
||||
|
||||
|
|
@ -95,8 +97,11 @@ package specifiers which are published as part of a package's metadata, and
|
|||
which a user can request under that name, as in ``pip install 'foo[bar]'`` to
|
||||
install ``foo`` with the ``bar`` extra.
|
||||
|
||||
Because ``extras`` are package metadata, they are not usable when a project
|
||||
does not build a distribution (i.e., is not a package).
|
||||
Because ``extras`` are package metadata, they are not guaranteed to be
|
||||
statically defined and may require a build system to resolve.
|
||||
Furthermore, definition of a ``[project.optional-dependencies]`` indicates to
|
||||
many tools that a project is a package, and may drive tool behaviors such as
|
||||
validation of the ``[project]`` table.
|
||||
|
||||
For projects which are packages, ``extras`` are a common solution for defining
|
||||
development dependencies, but even under these circumstances they have
|
||||
|
|
@ -163,24 +168,6 @@ defined in greater detail in the :ref:`Use Cases Appendix <use_cases>`.
|
|||
be used as a location for locked dependency data)
|
||||
* Input data to an environment manager, such as tox, Nox, or Hatch
|
||||
* Configurable IDE discovery of test and linter requirements
|
||||
* Exposure of package dependencies for install, without the package itself
|
||||
|
||||
Support for Inclusion from ``project.dependencies``
|
||||
---------------------------------------------------
|
||||
|
||||
The ``project.dependencies`` and ``project.optional-dependencies`` tables are
|
||||
allowed to include Dependency Groups, requiring an update to the specification
|
||||
of these tables.
|
||||
|
||||
The drivers for such changes are that some usages are well solved by the
|
||||
addition of such support, and that failing to include support in the initial
|
||||
Dependency Group PEP, but adding such support later in a subsequent PEP,
|
||||
would make the support landscape significantly more difficult for tool
|
||||
maintainers.
|
||||
|
||||
Inclusion of a Dependency Group in ``project.dependencies`` or
|
||||
``project.optional-dependencies`` takes the form of a Dependency Group Include,
|
||||
defined in the specification section below.
|
||||
|
||||
Regarding Poetry and PDM Dependency Groups
|
||||
------------------------------------------
|
||||
|
|
@ -255,12 +242,7 @@ Strings in requirement lists must be valid
|
|||
`Dependency Specifiers <https://packaging.python.org/en/latest/specifications/dependency-specifiers/>`__,
|
||||
as defined in :pep:`508`.
|
||||
|
||||
Tables in requirement lists must be valid Dependency Object Specifiers,
|
||||
defined below.
|
||||
|
||||
The ``project`` table in ``pyproject.toml`` is modified such that
|
||||
``project.dependencies`` and the values of ``project.optional-dependencies``
|
||||
may contain Dependency Object Specifiers.
|
||||
Tables in requirement lists must be valid Dependency Object Specifiers.
|
||||
|
||||
Dependency Object Specifiers
|
||||
----------------------------
|
||||
|
|
@ -310,21 +292,6 @@ Includes, in which case those includes should be expanded as well. Dependency
|
|||
Group Includes MUST NOT include cycles, and tools SHOULD report an error if
|
||||
they detect a cycle.
|
||||
|
||||
``project`` Table Changes
|
||||
-------------------------
|
||||
|
||||
The ``[project]`` table, originally defined in :pep:`621` is extended in two ways.
|
||||
|
||||
``dependencies``
|
||||
|
||||
In addition to :pep:`508` strings, the array may contain Dependency Object
|
||||
Specifiers.
|
||||
|
||||
``optional-dependencies``
|
||||
|
||||
In addition to :pep:`508` strings, the array values in this table may contain
|
||||
Dependency Object Specifiers.
|
||||
|
||||
Example Dependency Groups Table
|
||||
-------------------------------
|
||||
|
||||
|
|
@ -404,11 +371,10 @@ Tools MAY choose to provide the same interfaces for installing Dependency
|
|||
Groups as they do for installing extras.
|
||||
|
||||
Note that this specification does not forbid having an extra whose name matches
|
||||
a Dependency Group. In such cases, tools must define their own semantics for
|
||||
precedence order or disambiguation.
|
||||
a Dependency Group.
|
||||
|
||||
Users are advised to avoid creating Dependency Groups whose names match extras.
|
||||
Tools SHOULD NOT treat such matching as an error.
|
||||
Tools MAY treat such matching as an error.
|
||||
|
||||
Validation and Compatibility
|
||||
----------------------------
|
||||
|
|
@ -482,7 +448,7 @@ The output is therefore valid ``requirements.txt`` data.
|
|||
|
||||
|
||||
def _resolve_dependency_group(
|
||||
dependency_groups: dict, group: str, past_groups: tuple[str] = ()
|
||||
dependency_groups: dict, group: str, past_groups: tuple[str, ...] = ()
|
||||
) -> list[str]:
|
||||
if group in past_groups:
|
||||
raise ValueError(f"Cyclic dependency group include: {group} -> {past_groups}")
|
||||
|
|
@ -542,16 +508,15 @@ At time of writing, the ``dependency-groups`` namespace within a
|
|||
reserved for use only by standards specified at packaging.python.org,
|
||||
there are no direct backwards compatibility concerns.
|
||||
|
||||
However, the introduction of the feature as a potential component of
|
||||
``project`` data has implications for a number of ecosystem tools.
|
||||
However, the introduction of the feature has implications for a
|
||||
number of ecosystem tools, especially those which attempt to support
|
||||
examination of data in ``setup.py`` and ``requirements.txt``.
|
||||
|
||||
Audit and Update Tools
|
||||
----------------------
|
||||
|
||||
A wide range of tools understand Python dependency data as expressed in
|
||||
``project.dependencies`` and ``project.optional-dependencies`` and may
|
||||
additionally support ``setup.cfg``, ``requirements.txt``, and even
|
||||
``setup.py``. (e.g., Dependabot, Tidelift, etc)
|
||||
``requirements.txt`` files. (e.g., Dependabot, Tidelift, etc)
|
||||
|
||||
Such tools inspect dependency data and, in some cases, offer tool-assisted or
|
||||
fully automated updates.
|
||||
|
|
@ -563,31 +528,6 @@ As a result, users of Dependency Groups would experience a degradation in their
|
|||
workflows and tool support at the time that they start using Dependency Groups.
|
||||
This is true of any new standard for where and how dependency data are encoded.
|
||||
|
||||
Repackaging
|
||||
-----------
|
||||
|
||||
Repackaging, and in particular tool-assisted repackaging such as Grayskull or
|
||||
PyInstaller, will need to contend with the change to package metadata
|
||||
definitions as well.
|
||||
|
||||
Because repackagers for alternate ecosystems such as conda and linux distros
|
||||
are often distinct persons vs the package publishers, this compatibility
|
||||
concern is more difficult to address than cases in which the package
|
||||
maintainers experience the impact. Package maintainers may be unaware of the
|
||||
impact of beginning to use Dependency Groups, and may unknowingly make changes
|
||||
which harm downstream repackaging workflows.
|
||||
|
||||
There are two primary ways in which this issue can be addressed:
|
||||
|
||||
* via education and "How to Teach This" -- users whose packages are repackaged
|
||||
should be made aware that using new standards may cause issues for downstream
|
||||
package consumers
|
||||
|
||||
* by driving more behavior through build-backends -- so long as dependency
|
||||
metadata are gathered via :pep:`517` interfaces, downstream repackagers can
|
||||
remain ignorant of which build system is being used and whether or not it
|
||||
supports Dependency Groups
|
||||
|
||||
Security Implications
|
||||
=====================
|
||||
|
||||
|
|
@ -632,22 +572,6 @@ defined dynamically. Requirements loaded from ``requirements.txt`` files and
|
|||
definitions of static lists prior to ``setup()`` invocation readily analogize
|
||||
with Dependency Groups.
|
||||
|
||||
Notes for Packages which are Repackaged
|
||||
---------------------------------------
|
||||
|
||||
A special note should be given to package maintainers whose packages are
|
||||
repackaged by linux distros, homebrew, conda, etc.
|
||||
|
||||
Use of Dependency Group Includes in core project metadata,
|
||||
``project.dependencies`` and ``project.optional-dependencies``, may break these
|
||||
consumers' uses of your package. Because they may be consuming and directly
|
||||
interact with the source for your repository, their toolchains may not support
|
||||
Dependency Groups at the same time that the package maintainers' tools are
|
||||
updated.
|
||||
|
||||
Ensure that repackaging consumers can contact you if there is an issue, and
|
||||
make sure to note transitions to use Dependency Groups in your changelogs.
|
||||
|
||||
Interfaces for Use of Dependency Groups
|
||||
---------------------------------------
|
||||
|
||||
|
|
@ -779,18 +703,92 @@ has impacts on high-level tools like IDEs and Dependabot, which cannot support
|
|||
deep integration with these Dependency Groups. (For example, at time of writing
|
||||
Dependabot will not flag dependencies which are pinned in ``tox.ini`` files.)
|
||||
|
||||
Deferred Ideas
|
||||
==============
|
||||
|
||||
Why not support Dependency Group Includes in ``[project.dependencies]`` or ``[project.optional-dependencies]``?
|
||||
---------------------------------------------------------------------------------------------------------------
|
||||
|
||||
Earlier drafts of this specification allowed Dependency Group Includes to be
|
||||
used in the ``[project]`` table.
|
||||
However, there were several issues raised during community feedback which led
|
||||
to its removal.
|
||||
|
||||
Only a small number of additional use cases would be addressed by the inclusion
|
||||
of Dependency Groups, and it increased the scope of the specification
|
||||
significantly. In particular, this inclusion would increase the number of parties
|
||||
impacted by the addition. Many readers of the ``[project]`` table, including build
|
||||
backends, SBOM generators, and dependency analyzers are implicated by a change to
|
||||
``[project]`` but may continue to operate as-is in the presence of a new (but
|
||||
unconnected) ``[dependency-groups]`` table.
|
||||
|
||||
Separately from the above concern, allowing inclusion of dependency groups from the
|
||||
``[project]`` table encourages package maintainers to move dependency metadata out
|
||||
of the current standard location.
|
||||
This complicates static ``pyproject.toml`` metadata and conflicts with the goal of
|
||||
:pep:`621` to store dependency metadata in a single location.
|
||||
|
||||
Finally, exclusion of ``[project]`` support from this PEP is not final. The
|
||||
use of includes from that table, or an inclusion syntax from
|
||||
``[dependency-groups]`` into ``[project]``, could be introduced by a future
|
||||
PEP and considered on its own merits.
|
||||
|
||||
Use Cases for Dependency Group Includes From ``[project]``
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Although deferred in this PEP, allowing includes from the ``[project]``
|
||||
table would address several use cases.
|
||||
|
||||
In particular, there are cases in which package developers would like to
|
||||
install only the dependencies of a package, without the package itself.
|
||||
|
||||
For example:
|
||||
|
||||
* Specify different environment variables or options when building dependencies
|
||||
vs when building the package itself
|
||||
|
||||
* Creating layered container images in which the dependencies are isolated from
|
||||
the package being installed
|
||||
|
||||
* Providing the dependencies to analysis environments (e.g., type checking)
|
||||
without having to build and install the package itself
|
||||
|
||||
For an example of the last case, consider the following sample
|
||||
``pyproject.toml``:
|
||||
|
||||
.. code-block:: toml
|
||||
|
||||
[project]
|
||||
dependencies = [{include = "runtime"}]
|
||||
[optional-dependencies]
|
||||
foo = [{include = "foo"}]
|
||||
[dependency-groups]
|
||||
runtime = ["a", "b"]
|
||||
foo = ["c", "d"]
|
||||
typing = ["mypy", {include = "runtime"}, {include = "foo"}]
|
||||
|
||||
In this case, a ``typing`` group can be defined, with all of the package's
|
||||
runtime dependencies, but without the package itself. This allows uses of the
|
||||
``typing`` Dependency Group to skip installation of the package -- not only is
|
||||
this more efficient, but it may reduce the requirements for testing systems.
|
||||
|
||||
Why not support Dependency Group Includes in ``[build-system.requires]``?
|
||||
-------------------------------------------------------------------------
|
||||
|
||||
Although it may be interesting to allow this in the future, incorporating the
|
||||
proposal into the build-system table reduces the ability of users to rely on
|
||||
this to bootstrap support. :pep:`517` frontends would be required to support
|
||||
Dependency Groups in order to achieve this, and the versions of these frontends
|
||||
are not easily controlled by packages.
|
||||
Given that we will not allow for ``[project]`` usage of Dependency Groups,
|
||||
``[build-system.requires]`` can be considered in comparison with
|
||||
``[project.dependencies]``.
|
||||
|
||||
By only defining build backend support, it is possible for packages to start
|
||||
leveraging the new syntax and capabilities without being concerned about
|
||||
controlling the environment in which the package is built and installed.
|
||||
There are fewer theoretical usages for build requirements specified in a group
|
||||
than package requirements. Additionally, the impact of such a change implicates
|
||||
:pep:`517` frontend, which would need to support Dependency Groups in order to
|
||||
prepare a build environment.
|
||||
|
||||
Compared with changes to ``[project.dependencies]`` and
|
||||
``[project.optional-dependencies]``, changing the behaviors of
|
||||
``[build-system.requires]`` is higher impact and has fewer potential uses.
|
||||
Therefore, given that this PEP declines to make changes to the ``[project]``
|
||||
table, changing ``[build-system]`` is also deferred.
|
||||
|
||||
.. _prior_art:
|
||||
|
||||
|
|
@ -1411,42 +1409,6 @@ for various purposes.
|
|||
This declaration allows the project author's knowledge of the appropriate tools
|
||||
for the project to be shared with all editors of that project.
|
||||
|
||||
Exposure of package dependencies without the package itself
|
||||
-----------------------------------------------------------
|
||||
|
||||
There are a variety of use-cases in which package developers would like to
|
||||
install only the dependencies of a package, without the package itself.
|
||||
|
||||
For example:
|
||||
|
||||
* Specify different environment variables or options when building dependencies
|
||||
vs when building the package itself
|
||||
|
||||
* Creating layered container images in which the dependencies are isolated from
|
||||
the package being installed
|
||||
|
||||
* Providing the dependencies to analysis environments (e.g., type checking)
|
||||
without having to build and install the package itself
|
||||
|
||||
For an example of the last case, consider the following sample
|
||||
``pyproject.toml``:
|
||||
|
||||
.. code-block:: toml
|
||||
|
||||
[project]
|
||||
dependencies = [{include = "runtime"}]
|
||||
[optional-dependencies]
|
||||
foo = [{include = "foo"}]
|
||||
[dependency-groups]
|
||||
runtime = ["a", "b"]
|
||||
foo = ["c", "d"]
|
||||
typing = ["mypy", {include = "runtime"}, {include = "foo"}]
|
||||
|
||||
In this case, a ``typing`` group can be defined, with all of the package's
|
||||
runtime dependencies, but without the package itself. This allows uses of the
|
||||
``typing`` Dependency Group to skip installation of the package -- not only is
|
||||
this more efficient, but it may reduce the requirements for testing systems.
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue