PEP 627: Recording installed projects
https://github.com/python/peps/pull/1527
This commit is contained in:
parent
69c061642a
commit
769d7789e5
|
@ -0,0 +1,246 @@
|
|||
PEP: 627
|
||||
Title: Recording installed projects
|
||||
Author: Petr Viktorin <encukou@gmail.com>
|
||||
Discussions-To: https://discuss.python.org/t/pep-627/4126
|
||||
Status: Draft
|
||||
Type: Informational
|
||||
Content-Type: text/x-rst
|
||||
Created: 15-Jul-2020
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP clarifies and updates :pep:`376` (Database of Installed Python
|
||||
Distributions), rewriting it as an interoperability standard.
|
||||
It moves the canonical location of the standard to the Python
|
||||
Packaging Authority (PyPA) standards repository, and sets up guidelines
|
||||
for changing it.
|
||||
|
||||
Two files in installed ``.dist-info`` directories are made optional:
|
||||
``RECORD`` (which PEP 376 lists as mandatory, but suggests it can be left out
|
||||
for "system packages"), and ``INSTALLER``.
|
||||
The ``REQUESTED`` file is removed from the specification.
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
||||
Python packaging is moving from relying on specific tools (Setuptools and pip)
|
||||
toward a ecosystem of tools and tool-agnostic interoperability standards.
|
||||
|
||||
PEP 376 is not written as an interoperability standard.
|
||||
It describes implementation details of specific tools and libraries,
|
||||
and is underspecified, leaving much room for implementation-defined behavior.
|
||||
|
||||
This is a proposal to “distill” the standard from PEP 376, clarify it,
|
||||
and rewrite it to be tool-agnostic.
|
||||
|
||||
The aim of this PEP is to have a better standard, not necessarily a perfect one.
|
||||
Some issues are left to later clarification.
|
||||
|
||||
|
||||
Rationale Change
|
||||
================
|
||||
|
||||
PEP 376's rationale focuses on two problems:
|
||||
|
||||
* There are too many ways to install projects and this makes interoperation difficult.
|
||||
* There is no API to get information on installed distributions.
|
||||
|
||||
The new document focuses only the on-disk format of information about
|
||||
installed projects.
|
||||
Providing API to install, uninstall or query this information is left to
|
||||
be implemented by tools.
|
||||
|
||||
|
||||
Standard and Changes Process
|
||||
============================
|
||||
|
||||
The canonical standard for *Recording installed projects* (previously known as
|
||||
*Database of Installed Python Distributions*) is the `documentation`_ at
|
||||
``packaging.python.org``.
|
||||
Any changes to the document (except trivial language or typography fixes) must
|
||||
be made through the PEP process.
|
||||
|
||||
The document is normative (with examples to aid understanding).
|
||||
PEPs that change it, such as this one, contain additional information that is
|
||||
expected to get out of date, such as rationales and compatibility
|
||||
considerations.
|
||||
|
||||
The proposed standard is submitted together with this PEP as a pull request to
|
||||
``packaging.python.org``.
|
||||
|
||||
.. _documentation: https://packaging.python.org/specifications/recording-installed-packages/
|
||||
|
||||
|
||||
Changes and their Rationale
|
||||
===========================
|
||||
|
||||
Renaming to "Recording installed projects"
|
||||
------------------------------------------
|
||||
|
||||
The standard is renamed from *Database of Installed Python Distributions*
|
||||
to *Recording installed projects*.
|
||||
|
||||
While putting files in known locations on disk may be thought of as
|
||||
a “database”, it's not what most people think about when they hear the term.
|
||||
The PyPA links to PEP 376 under the heading *Recording installed distributions*.
|
||||
|
||||
The PyPA glossary defines “Distribution” (or, “Distribution Package” to prevent
|
||||
confusion with e.g. Linux distributions) as “A versioned archive file […]”.
|
||||
Since there may be other ways to install Python code than from archive files,
|
||||
the document uses “installed project” rather than “installed distribution”.
|
||||
|
||||
|
||||
Removal of Implementation Details
|
||||
---------------------------------
|
||||
|
||||
All tool- and library-specific details are removed.
|
||||
The mechanisms of how a project is installed are also left out: the document
|
||||
focuses on the end state.
|
||||
One exception is a sketch of an uninstallation algorithm,
|
||||
which is given to better explain the purpose of the ``RECORD`` file.
|
||||
|
||||
References to ``.egg-info`` and ``.egg``,
|
||||
formats specific to ``setuptools`` and ``distutils``,
|
||||
are left out.
|
||||
|
||||
|
||||
Explicitly Allowing Additional Files
|
||||
------------------------------------
|
||||
|
||||
The ``.dist-info`` directory is allowed to contain files not specified in
|
||||
the spec.
|
||||
The current tools already do this.
|
||||
|
||||
A note in the specification mentions files in the ``.dist-info`` directory of *wheels*.
|
||||
Current tools copy these files to the installed ``.dist-info``—something
|
||||
to keep in mind for further standardization efforts.
|
||||
|
||||
|
||||
Clarifications in the ``RECORD`` File
|
||||
-------------------------------------
|
||||
|
||||
The CSV dialect is specified to be the default of Python's ``csv`` module.
|
||||
This resolves edge cases around handling double-quotes and line terminators
|
||||
in file names.
|
||||
|
||||
The “base” of relative paths in ``RECORD`` is specified relative to the
|
||||
``.dist-info`` directory, rather than tool-specific ``--install-lib`` and
|
||||
``--prefix`` options.
|
||||
|
||||
Both *hash* and *size* fields are now optional (for any file, not just
|
||||
``.pyc``, ``.pyo`` and ``RECORD``).
|
||||
This simplifies the spec, and makes it possible for tools to modify files
|
||||
(for example, rewrite shebangs) after ``RECORD`` is generated.
|
||||
|
||||
The new spec explicitly says that the ``RECORD`` file must now include *all*
|
||||
files of the installed project (the exception for ``.pyc`` files remains).
|
||||
Since tools use ``RECORD`` for uninstallation, incomplete file lists could
|
||||
introduce orphaned files to users' environments.
|
||||
On the other hand, this means that there is no way to record hashes of some
|
||||
any files if the full list of files is unknown.
|
||||
|
||||
A sketch of an uninstallation algorithm is included to clarify the file's
|
||||
primary purpose and contents.
|
||||
|
||||
On Windows, files in ``RECORD`` may be separated by either ``/`` or ``\``.
|
||||
PEP 376 was unclear on this: it mandates forward slashes in one place, but
|
||||
shows backslackes in a Windows-specific example.
|
||||
|
||||
|
||||
|
||||
Optional ``RECORD`` File
|
||||
------------------------
|
||||
|
||||
The ``RECORD`` file is made optional.
|
||||
Not all tools can easily generate a list of installed files in a
|
||||
Python-specific format.
|
||||
|
||||
Specifically, the ``RECORD`` file is unnecessary when projects are installed
|
||||
by a Linux system packaging system, which has its own ways to keep track of
|
||||
files, uninstall them or check their integrity.
|
||||
Having to keep a ``RECORD`` file in sync with the disk and the system package
|
||||
database would be unreasonably fragile, and no ``RECORD`` file is better
|
||||
than one that does not correspond to reality.
|
||||
|
||||
(Full disclosure: The author of this PEP is an RPM packager active in the Fedora Linux distro.)
|
||||
|
||||
|
||||
Optional ``INSTALLER`` File
|
||||
---------------------------
|
||||
|
||||
The ``INSTALLER`` file is also made optional, and specified to be used for
|
||||
informational purposes only.
|
||||
|
||||
This file was added to distinguish projects installed by the Python installer
|
||||
(``pip``) from ones installed by other package managers (e.g. ``dnf``).
|
||||
There were attempts to use this file to prevent ``pip`` from updating or
|
||||
uninstalling packages it didn't install.
|
||||
|
||||
Our goal is supporting interoperating tools, and basing any action on
|
||||
which tool happened to install a package runs counter to that goal.
|
||||
|
||||
Instead of relying on the installer name, tools should use feature detection.
|
||||
The current document offers a crude way of making a project untouchable by
|
||||
Python tooling: omitting ``RECORD`` file.
|
||||
|
||||
On the other hand, the installer name may be useful in hints to the user.
|
||||
|
||||
To align with this new purpose of the file, the new specification allows
|
||||
any ASCII string in ``INSTALLER``, rather than a lowercase identifier.
|
||||
It also suggests using the command-line command, if available.
|
||||
|
||||
|
||||
Removed ``REQUESTED`` File
|
||||
--------------------------
|
||||
|
||||
The ``REQUESTED`` file, which was never implemented in ``pip``, is removed
|
||||
from the specification.
|
||||
|
||||
The semantics of this file depend on all tools creating it when they should.
|
||||
Since today's most popular tool, ``pip``, does not create it, the file is
|
||||
not useful.
|
||||
(It might be useful in environments managed by a single tool; in that case,
|
||||
the tool is free to use ``REQUESTED`` as a private extension.)
|
||||
|
||||
(The author of this PEP considers ``REQUESTED`` an unnecessary attempt to emulate
|
||||
system package managers. Recent tools tend to support creating easily
|
||||
reproducible environments, rather than manage long-lived ones;
|
||||
deleting the environment and starting a new one is a surprissingly good
|
||||
solution to “dangling dependencies”.)
|
||||
|
||||
|
||||
Clarifications
|
||||
--------------
|
||||
|
||||
When possible, terms (such as ``name`` and ``version``) are qualified by
|
||||
references to existing specs.
|
||||
|
||||
|
||||
Deferred Ideas
|
||||
==============
|
||||
|
||||
To limit the scope of this PEP, some improvements are explicitly left to
|
||||
future PEPs:
|
||||
|
||||
* Encoding of the ``RECORD`` file
|
||||
* Limiting or namespacing files that can appear in ``.dist-info``
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document is placed in the public domain or under the
|
||||
CC0-1.0-Universal license, whichever is more permissive.
|
||||
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
Loading…
Reference in New Issue