diff --git a/pep-0700.rst b/pep-0700.rst index 962f44799..8175c7c00 100644 --- a/pep-0700.rst +++ b/pep-0700.rst @@ -18,51 +18,78 @@ Abstract clients to more easily query the data that was previously only available in HTML, as defined in :pep:`503`. -This proposal adds three *optional* additional fields to the JSON form, which -allow it to be used in place of PyPI's `JSON API `__ +This proposal adds three fields to the JSON form, which allow it to be used in +place of PyPI's `JSON API `__ in a number of situations. - A field to allow retrieval of a list of all the published versions of a project. - Fields containing the size and upload time for a project file. +Rationale +========= + +With the introduction of the JSON form of the simple API in :pep:`691`, the +simple API offers functionality that is almost as complete as the PyPI JSON API. +This PEP adds a number of fields which were previously only available through +the JSON API, in order to allow more clients which were previously Warehouse +specific to support arbitrary standards-compliant indexes. + + Specification ============= -The proposal adds three new fields to the JSON form of the "project detail" -pages in the simple index, located at the URL ``//``. These fields are -*not* available in the HTML form of the page. +This specification defines version 1.1 of the simple repository API. For the +HTML version of the API, there is no change from version 1.0. For the JSON +version of the API, the following changes are made: -An index MUST provide all of the specified information, for all projects and -files, if it provides any of it. In other words, while indexes may choose to -only support the base :pep:`691`, if they choose to support this PEP they must -do so completely. +- The ``api-version`` must specify version 1.1 or later. +- A new ``versions`` key is added at the top level. +- Two new "file information" keys, ``size`` and ``upload-time``, are added to + the ``files`` data. +- Keys (at any level) named with a leading underscore are reserved as private + for index server use. No future standard will assign a meaning to any such key. -In addition, as a technical point, this PEP reserves all field names in the JSON -form beginning with an underscore as available for private use by indexes - no -future PEP will assign a standardised meaning to such names. +All new fields are mandatory (although ``upload-time`` may be ``None`` if the +server does not store that information). Versions -------- -An additional key, ``versions`` MAY be present at the top level, in addition to -the keys ``name``, ``files`` and ``meta`` defined in :pep:`691`. This key, if -present, MUST contain a list of version strings conforming to :pep:`440`, which -list all of the project versions uploaded for this project. +An additional key, ``versions`` MUST be present at the top level, in addition to +the keys ``name``, ``files`` and ``meta`` defined in :pep:`691`. This key MUST +contain a list of version strings specifying all of the project versions uploaded +for this project. The value is logically a set, and as such may not contain +duplicates, and the order of the values is not significant. All of the files listed in the ``files`` key MUST be associated with one of the -versions in the ``versions`` key, if present. The ``versions`` key MAY contain -versions with no associated files. +versions in the ``versions`` key. The ``versions`` key MAY contain versions with +no associated files (to represent versions with no files uploaded, if the server +has such a concept). + +Note that because servers may hold "legacy" data from before the adoption of +:pep:`440`, version strings currently cannot be required to be valid :pep:`440` +versions, and therefore cannot be assumed to be orderable using the :pep:`440` +rules. However, servers SHOULD use normalised :pep:`440` versions where +possible. + Additional file information --------------------------- -The elements of the ``files`` key MAY contain two additional sub-keys, ``size`` -and ``upload_time``. The ``size`` field MUST contain an integer which is the -file size in bytes, and the ``upload_time`` field MUST contain a valid ISO 8601 -date/time string, in the format ``yyyy-mm-ddThh:mm:ss.ffffffZ``, which -represents the time the file was uploaded to the index. As indicated by the -``Z`` suffix, the upload time MUST use the UTC timezone. +The elements of the ``files`` key MUST contain two additional sub-keys, ``size`` +and ``upload-time``. The ``size`` field MUST contain an integer which is the +file size in bytes, and the ``upload-time`` field MUST contain either ``null`` +(see below) or a valid ISO 8601 date/time string, in the format +``yyyy-mm-ddThh:mm:ss.ffffffZ``, which represents the time the file was uploaded +to the index. As indicated by the ``Z`` suffix, the upload time MUST use the UTC +timezone. The fractional seconds part of the timestamp (the ``.ffffff`` part) is +optional, and if present may contain up to 6 digits of precision. + +If a server does not record upload time information for a file, it MAY return +a value for ``upload-time`` of ``None`` (``null`` in JSON). However, it MUST +NOT omit the key. + FAQ === @@ -83,6 +110,20 @@ and this PEP does not change that position. However, clients wishing to access the new data introduced by this PEP will need to use the JSON API to get it. And indexes wanting to provide it will need to serve the JSON format. +Is the simple API replacing the Warehouse JSON and XML-RPC APIs? +---------------------------------------------------------------- + +Where possible, clients should prefer the simple API over the JSON or XML-RPC +APIs, as the former is standardised and can be assumed to be available from any +index, whereas the latter are exclusive to the Warehouse project. + +However, while this PEP brings the simple API closer to being able to replace +the JSON API, there is no formal policy that the simple API will replicate all +of the functionality covered by the existing Warehouse APIs. Proposed additions +to the simple API will still be considered on their individual merits, and the +requirement that the API should be simple and fast for the primary use case of +locating files for a project will remain the overriding consideration. + Why not allow other date formats? --------------------------------- @@ -102,6 +143,51 @@ an issue for code written in Python. Non-Python implementations may need to take care to handle large integers correctly, but this is not expected to be a significant problem. +Why is it not possible to serve some of the new information, but not all? +------------------------------------------------------------------------- + +The versioning scheme for the simple API does not allow servers to "mix and +match" what information they provide, except by making fields optional, and +handling optional fields is an inconvenience for clients using the API. The +PEP takes the view that ease of use for clients should be the priority (as +there will always be more clients than servers). + +The main problem for servers is likely to be the upload time, as some servers +may not retain this information. For this situation, a server is allowed to +return a null value for the upload time, indicating that the value is not known. + +Version and file size information do not have a "not known" option, as they +should not need them. Versions can be determined from the file names if +necessary, and file size information is needed to serve the files, so it should +always be possible for the server to determine it. + +Why not require PEP 440 versions? +--------------------------------- + +At the time this PEP was written, PyPI still contains (and serves) projects and +files with "legacy" versions. Requiring :pep:`440` versions would make it +impossible for PyPI to follow this specification while still serving the +existing content. + +Ideally, at some future time, the simple index API will be updated to require +:pep:`440` versions, at which time this specification should be updated to +reflect that. However, that change will need to be co-ordinated with existing +index providers including PyPI, to desupport and remove non-conforming projects +and/or files. + +Why not provide a "latest version" value? +----------------------------------------- + +For :pep:`440` versions, this is easy enough for the client to do (using the +``packaging`` library, ``latest = max(Version(s) for s in proj["versions"])``). +For non-standard versions, there is no well-defined ordering, and clients will +need to decide on what rule is appropriate for their needs. Requiring the server +to supply a latest version value takes the choice away from the client. + +Servers with an explicit concept of which version is the "latest", which cannot +be calculated from data available to the client, can provide a non-standard, +underscore-prefixed key to convey that information to the client if they wish. + Copyright =========