iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 665: clarifications based on feedback

This commit is contained in:
Brett Cannon 2021-07-30 16:50:34 -07:00
parent 9417cfacf1
commit ae53120c3f
1 changed files with 196 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

208
pep-0665.rst
View File

@ -75,6 +75,151 @@ of those communities include:
#. Rust_
Below, we identify various use-cases applicable to stakeholders in the
Python community and anyone who interacts with Python package
installers who are the ultimate consumers of a lock file.
---------
Providers
---------
Providers are the parties (organization, person, community, etc.) that
supply a service or software tool which interacts with Python
packaging. Two different types of providers are considered:
Platform/Infrastructure Providers
=================================
Platform providers (cloud environments, application hosting, etc.) and
infrastructure service providers need to support package installers
for their users to install Python dependencies. Most only support
``requirements.txt`` files and a smattering of other file formats for
listing a project's dependencies. Most providers do not want to maintain
support for more than one dependency specification format
because of the complexity it adds to their software or service and the
resources it takes to do so (e.g. not all platform providers have
the staffing to support pip-tools, Poetry, Pipenv, etc.).
This PEP would allow platform providers to declare support for this PEP
and thus only have to support one dependency specification format. What
this would mean is developers could use whatever toolchain they preferred
for development as long as they could emit a file that implemented this
PEP. This then allows developers to not have to align with what their
platform providers supports as long as everyone agrees to implementing
this PEP.
IDE Providers
=============
Integrated development environments may interact with Python package
installation and management. Most only support select few tools, and
users are required to find work arounds to install
their dependencies using other package installers. Similar to the
situation with PaaS & IaaS providers, IDE providers do not want to
maintain support for N different formats. Instead, tools would only
need to be able to read files which implement this PEP to perform various
actions (e.g. list all the dependencies of the open project, which ones
are missing, install dependencies, generate the lock file, etc.).
As an example, the Python extension for VS Code has to have custom support
for each installer tool people may use: pip, Poetry, Pipenv, etc. This is
not only tedious by having to track multiple projects and any changes they
make, but it also locks out newer tools whose popularity isn't great
enough to warrant inclusion in the extension.
----------
Developers
----------
Developers are teams, people, or communities that code and use Python
package installers and Python packages. Three different types of
developers are considered:
Developers using PaaS & IaaS providers
======================================
Most PaaS and IaaS providers only support one Python package
installer: ``requirements.txt. This dictates the installers that
developers can use while working with these providers, which might not
be optimal for their application or workflow.
Developers adopting this PEP would be able to use third party
platforms/infrastructure without having to
worry about which Python package installer they are required to use as
long as the provider also supports this PEP.
Developers using IDEs
=====================
Most IDEs only support pip or a few Python package installers.
Consequently, developers must use workarounds or hacky methods to
install their dependencies if they use an unsupported package
installer.
If the IDE uses/supports this PEP it would allow for
any the developer to use whatever tooling they wanted to generate
their lock file while the IDE can use whatever tooling it wants to
performs actions with/on the lock file.
Developers working with other developers
========================================
Developers want to be able to use the installer of their choice while
working with other developers, but currently have to synchronize their
installer choice for compatibility of dependency installation. If all
preferred installers instead implemented the specified interface, it
would allow for cross use of installers, allowing developers to choose
an installer regardless of their collaborator’s preference.
--------------------------------------------
Upgraders & Package Infrastructure Providers
--------------------------------------------
Package upgraders and package infrastructure in CI/CD such as
Dependabot_, PyUP_, etc. currently support a few formats. They work
by parsing and editing the dependency files with
relevant package information such as upgrades, downgrades, or new
hashes. Similar to Platform and IDE providers, most of these providers
do not want to support N different formats.
Currently, these services/bots have to implement support for each
format individually. Inevitably, the most popular
formats are supported first, and less popular tools are often never
supported. By implementing this specification, these services/bots can
support one format, allowing users to select the tool
of their choice to generate the file. This will allow for more innovation
in the space, as platforms and IDEs are no longer forced to prematurely
select a "winner" tool which generates a lock file.
---------------------
Open Source Community
---------------------
Specifying installer requirements and adopting this PEP will reduce
the friction between Python package installers and people's workflows.
Consequently, it will reduce the friction between Python package
installers and 3rd party infrastructure/technologies such as PaaS or
IDEs. Overall, it will allow for easier development, deployment and
maintenance of Python projects as Python package installation becomes
simpler and more interoperable.
Specifying a single file format can also increase the pace of innovation
around installers and the generation of dependency graphs. By
decoupling generating the dependency graph details from installation It
allows for each area to grow and innovate independently. It also allows
more flexibilty in tool selection on either end of the dependency graph
and installation ends of this process.
=========
Rationale
=========
@ -175,6 +320,9 @@ MUST only increment in future updates to the specification. What
consistitutes a version number increase is left to future PEPs or
standards changes.
Tools reading a lock file whose version they don't support MUST raise
an error.
``[tool]``
==========
@ -278,14 +426,14 @@ An optional key containing an array of strings following the
package depends on. See ``metadata.needs`` for full details.
``package.<name>.required-by``
``package.<name>.needed-by``
------------------------------
A key containing an array of package names which depend on this
package. The package names MUST match the package name as used in the
``package`` table.
The lack of a ``required-by`` key infers that the package is a
The lack of a ``needed-by`` key infers that the package is a
top-level package listed in ``metadata.needs``.
@ -322,7 +470,7 @@ A `wheel file`_ for the package version.
Supported keys in the table are:
- ``url``: a string of location of the wheel file (use the
``file://`` protocol for the local file system)
``file:`` protocol for the local file system)
- ``hash-algorithm``: a string of the algorithm used to generate the
hash value stored in ``hash-value``
- ``hash-value``: a string of the hash of the file contents
@ -347,7 +495,7 @@ keys may be compressed tags.
A `source distribution file`_ (sdist) for the package version.
- ``url``: a string of location of the sdist file (use the
``file://`` protocol for the local file system)
``file:`` protocol for the local file system)
- ``hash-algorithm``: a string of the algorithm used to generate the
hash value stored in ``hash-value``
- ``hash-value``: a string of the hash of the file contents
@ -359,7 +507,7 @@ A `source distribution file`_ (sdist) for the package version.
A Git_ version control repository for the package.
- ``url``: a string of location of the repository (use the
``file://`` protocol for the local file system)
``file:`` protocol for the local file system)
- ``commit``: a string of the commit of the repository which
represents the version of the package
@ -376,7 +524,7 @@ contents, there is no hash to verify.
A source tree which can be used to build a wheel.
- ``url``: a string of location of the source tree (use the
``file://`` protocol for the local file system)
``file:`` protocol for the local file system)
- ``mime-type``: (optional) a string representing the MIME type of the
URL
- ``hash-algorithm``: (optional for a local directory) a string of the
@ -419,7 +567,7 @@ Example
[[package.attrs]]
version = "21.2.0"
required-by = ["mousebender"]
needed-by = ["mousebender"]
[[package.attrs.code]]
type = "wheel"
@ -446,7 +594,7 @@ Example
[[package.packaging]]
version = "20.9"
needs = ["pyparsing>=2.0.2"]
required-by = ["mousebender"]
needed-by = ["mousebender"]
[[package.packaging.code]]
type = "git"
@ -455,7 +603,7 @@ Example
[[package.pyparsing]]
version = "2.4.7"
required-by = ["packaging"]
needed-by = ["packaging"]
[[package.pyparsing.code]]
type="wheel"
@ -678,12 +826,22 @@ any reason to disagree with that, the decision was to go with
-------------------------------------
Alternative Names for ``required-by``
Alternative Names for ``needed-by``
-------------------------------------
Other names that were considered were ``dependents``, ``depended-by``,
and ``supports``. In the end, ``required-by`` simply seemed like the
best fit.
, ``supports`` and ``required-by``. In the end, ``needed-by`` made
sense and tied into ``needs``.
--------------------------------------------------
Only Allowing a Single Code Location For a Project
--------------------------------------------------
While reproducibility is serviced better by only allowing a single
code location, it limits usability for situations where one wants to
support multiple platforms with a single lock file (which the community
has shown is desired).
-------------------------------------
@ -698,6 +856,32 @@ that was viewed as a security concern (Git commit IDs are hashes of
metadata and thus are viewed as immutable).
-----------------
Accepting PEP 650
-----------------
PEP 650 was an earlier attempt at trying to tackle this problem by
specifying an API for installers instead of standardizing on a lock file
format (ala PEP 517). The
`initial response <https://discuss.python.org/t/pep-650-specifying-installer-requirements-for-python-projects/6657/>`__
to PEP 650 could be considered mild/lukewarm. People seemed to be
consistently confused over which tools should provide what functionality
to implement the PEP. It also potentially incurred more overhead as
it would require executing Python APIs to perform any actions involving
packaging.
This PEP chose to standardize around an artifact instead of an API
(ala PEP 621). This would allow for more tool integrations as it
removes the need to specifically use Python to do things such as
create a lock file, update it, or even install packages listed in
a lock file. It also allows for easier introspection by forcing
dependency graph details to be written in a human-readable format.
It also allows for easier sharing of knowledge by standardizing what
people need to know more (e.g. tutorials become more portable between
tools when it comes to understanding the artifact they produce). It's
also simply the approach other language communities have taken and seem
to be happy with.
===========
Open Issues
===========