197 lines
8.9 KiB
ReStructuredText
197 lines
8.9 KiB
ReStructuredText
PEP: 700
|
|
Title: Additional Fields for the Simple API for Package Indexes
|
|
Author: Paul Moore <p.f.moore@gmail.com>
|
|
PEP-Delegate: Donald Stufft <donald@stufft.io>
|
|
Discussions-To: https://discuss.python.org/t/pep-700-additional-fields-for-the-simple-api-for-package-indexes/20177
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Topic: Packaging
|
|
Content-Type: text/x-rst
|
|
Created: 21-Oct-2022
|
|
Post-History: `21-Oct-2022 <https://discuss.python.org/t/pep-700-additional-fields-for-the-simple-api-for-package-indexes/20177>`__
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
:pep:`691` defined a JSON form for the "Simple Repository API". This allowed
|
|
clients to more easily query the data that was previously only available in
|
|
HTML, as defined in :pep:`503`.
|
|
|
|
This proposal adds three fields to the JSON form, which allow it to be used in
|
|
place of PyPI's `JSON API <https://warehouse.pypa.io/api-reference/json.html>`__
|
|
in a number of situations.
|
|
|
|
- A field to allow retrieval of a list of all the published versions of a project.
|
|
- Fields containing the size and upload time for a project file.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
With the introduction of the JSON form of the simple API in :pep:`691`, the
|
|
simple API offers functionality that is almost as complete as the PyPI JSON API.
|
|
This PEP adds a number of fields which were previously only available through
|
|
the JSON API, in order to allow more clients which were previously Warehouse
|
|
specific to support arbitrary standards-compliant indexes.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
This specification defines version 1.1 of the simple repository API. For the
|
|
HTML version of the API, there is no change from version 1.0. For the JSON
|
|
version of the API, the following changes are made:
|
|
|
|
- The ``api-version`` must specify version 1.1 or later.
|
|
- A new ``versions`` key is added at the top level.
|
|
- Two new "file information" keys, ``size`` and ``upload-time``, are added to
|
|
the ``files`` data.
|
|
- Keys (at any level) named with a leading underscore are reserved as private
|
|
for index server use. No future standard will assign a meaning to any such key.
|
|
|
|
All new fields are mandatory (although ``upload-time`` may be ``None`` if the
|
|
server does not store that information).
|
|
|
|
Versions
|
|
--------
|
|
|
|
An additional key, ``versions`` MUST be present at the top level, in addition to
|
|
the keys ``name``, ``files`` and ``meta`` defined in :pep:`691`. This key MUST
|
|
contain a list of version strings specifying all of the project versions uploaded
|
|
for this project. The value is logically a set, and as such may not contain
|
|
duplicates, and the order of the values is not significant.
|
|
|
|
All of the files listed in the ``files`` key MUST be associated with one of the
|
|
versions in the ``versions`` key. The ``versions`` key MAY contain versions with
|
|
no associated files (to represent versions with no files uploaded, if the server
|
|
has such a concept).
|
|
|
|
Note that because servers may hold "legacy" data from before the adoption of
|
|
:pep:`440`, version strings currently cannot be required to be valid :pep:`440`
|
|
versions, and therefore cannot be assumed to be orderable using the :pep:`440`
|
|
rules. However, servers SHOULD use normalised :pep:`440` versions where
|
|
possible.
|
|
|
|
|
|
Additional file information
|
|
---------------------------
|
|
|
|
The elements of the ``files`` key MUST contain two additional sub-keys, ``size``
|
|
and ``upload-time``. The ``size`` field MUST contain an integer which is the
|
|
file size in bytes, and the ``upload-time`` field MUST contain either ``null``
|
|
(see below) or a valid ISO 8601 date/time string, in the format
|
|
``yyyy-mm-ddThh:mm:ss.ffffffZ``, which represents the time the file was uploaded
|
|
to the index. As indicated by the ``Z`` suffix, the upload time MUST use the UTC
|
|
timezone. The fractional seconds part of the timestamp (the ``.ffffff`` part) is
|
|
optional, and if present may contain up to 6 digits of precision.
|
|
|
|
If a server does not record upload time information for a file, it MAY return
|
|
a value for ``upload-time`` of ``None`` (``null`` in JSON). However, it MUST
|
|
NOT omit the key.
|
|
|
|
|
|
FAQ
|
|
===
|
|
|
|
Why not add this data to the HTML API as well?
|
|
----------------------------------------------
|
|
|
|
It would be possible to add the data to the HTML API, but the vast majority of
|
|
consumers for this data are likely to be currently getting it from the PyPI JSON
|
|
API, and so will already be expecting to parse JSON. Traditional consumers of
|
|
the HTML API have never needed this data previously.
|
|
|
|
Does this imply that the HTML API is obsolete?
|
|
----------------------------------------------
|
|
|
|
No. The FAQ of :pep:`691` was clear that the HTML API is not being deprecated,
|
|
and this PEP does not change that position. However, clients wishing to access
|
|
the new data introduced by this PEP will need to use the JSON API to get it. And
|
|
indexes wanting to provide it will need to serve the JSON format.
|
|
|
|
Is the simple API replacing the Warehouse JSON and XML-RPC APIs?
|
|
----------------------------------------------------------------
|
|
|
|
Where possible, clients should prefer the simple API over the JSON or XML-RPC
|
|
APIs, as the former is standardised and can be assumed to be available from any
|
|
index, whereas the latter are exclusive to the Warehouse project.
|
|
|
|
However, while this PEP brings the simple API closer to being able to replace
|
|
the JSON API, there is no formal policy that the simple API will replicate all
|
|
of the functionality covered by the existing Warehouse APIs. Proposed additions
|
|
to the simple API will still be considered on their individual merits, and the
|
|
requirement that the API should be simple and fast for the primary use case of
|
|
locating files for a project will remain the overriding consideration.
|
|
|
|
Why not allow other date formats?
|
|
---------------------------------
|
|
|
|
The ISO 8601 standard is complex, and there seems little value in requiring
|
|
clients to deal with that. The standard library ``datetime`` module provides
|
|
methods to parse ISO 8601 strings, but it is possible that users may want to
|
|
access index data *without* using Python (for example, piping the output of
|
|
``curl`` into ``jq``). Having a single, well-defined format makes this easier,
|
|
and doesn't have any significant disadvantages.
|
|
|
|
What if file sizes are too big for a JSON number?
|
|
-------------------------------------------------
|
|
|
|
The JSON standard does not specify how numbers are to be interpreted. Python can
|
|
read and write arbitrary-length integers in a JSON file, so this should not be
|
|
an issue for code written in Python. Non-Python implementations may need to take
|
|
care to handle large integers correctly, but this is not expected to be a
|
|
significant problem.
|
|
|
|
Why is it not possible to serve some of the new information, but not all?
|
|
-------------------------------------------------------------------------
|
|
|
|
The versioning scheme for the simple API does not allow servers to "mix and
|
|
match" what information they provide, except by making fields optional, and
|
|
handling optional fields is an inconvenience for clients using the API. The
|
|
PEP takes the view that ease of use for clients should be the priority (as
|
|
there will always be more clients than servers).
|
|
|
|
The main problem for servers is likely to be the upload time, as some servers
|
|
may not retain this information. For this situation, a server is allowed to
|
|
return a null value for the upload time, indicating that the value is not known.
|
|
|
|
Version and file size information do not have a "not known" option, as they
|
|
should not need them. Versions can be determined from the file names if
|
|
necessary, and file size information is needed to serve the files, so it should
|
|
always be possible for the server to determine it.
|
|
|
|
Why not require PEP 440 versions?
|
|
---------------------------------
|
|
|
|
At the time this PEP was written, PyPI still contains (and serves) projects and
|
|
files with "legacy" versions. Requiring :pep:`440` versions would make it
|
|
impossible for PyPI to follow this specification while still serving the
|
|
existing content.
|
|
|
|
Ideally, at some future time, the simple index API will be updated to require
|
|
:pep:`440` versions, at which time this specification should be updated to
|
|
reflect that. However, that change will need to be co-ordinated with existing
|
|
index providers including PyPI, to desupport and remove non-conforming projects
|
|
and/or files.
|
|
|
|
Why not provide a "latest version" value?
|
|
-----------------------------------------
|
|
|
|
For :pep:`440` versions, this is easy enough for the client to do (using the
|
|
``packaging`` library, ``latest = max(Version(s) for s in proj["versions"])``).
|
|
For non-standard versions, there is no well-defined ordering, and clients will
|
|
need to decide on what rule is appropriate for their needs. Requiring the server
|
|
to supply a latest version value takes the choice away from the client.
|
|
|
|
Servers with an explicit concept of which version is the "latest", which cannot
|
|
be calculated from data available to the client, can provide a non-standard,
|
|
underscore-prefixed key to convey that information to the client if they wish.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|