python-peps/peps/pep-0503.rst

PEP: 503
Title: Simple Repository API
Version: $Revision$
Last-Modified: $Date$
Author: Donald Stufft <donald@stufft.io>
BDFL-Delegate: Donald Stufft <donald@stufft.io>
Discussions-To: distutils-sig@python.org
Status: Final
Type: Standards Track
Topic: Packaging
Content-Type: text/x-rst
Created: 04-Sep-2015
Post-History: 04-Sep-2015
Resolution: https://mail.python.org/pipermail/distutils-sig/2015-September/026899.html


Abstract
========

There are many implementations of a Python package repository and many tools
that consume them. Of these, the canonical implementation that defines what
the "simple" repository API looks like is the implementation that powers
PyPI. This document will specify that API, documenting what the correct
behavior for any implementation of the simple repository API.


Specification
=============

A repository that implements the simple API is defined by its base URL, this is
the top level URL that all additional URLs are below. The API is named the
"simple" repository due to the fact that PyPI's base URL is
``https://pypi.org/simple/``.

.. note:: All subsequent URLs in this document will be relative to this base
          URL (so given PyPI's URL, a URL of ``/foo/`` would be
          ``https://pypi.org/simple/foo/``.


Within a repository, the root URL (``/`` for this PEP which represents the base
URL) **MUST** be a valid HTML5 page with a single anchor element per project in
the repository. The text of the anchor tag **MUST** be the name of
the project and the href attribute **MUST** link to the URL for that particular
project. As an example::

   <!DOCTYPE html>
   <html>
     <body>
       <a href="/frob/">frob</a>
       <a href="/spamspamspam/">spamspamspam</a>
     </body>
   </html>

Below the root URL is another URL for each individual project contained within
a repository. The format of this URL is ``/<project>/`` where the ``<project>``
is replaced by the normalized name for that project, so a project named
"HolyGrail" would have a URL like ``/holygrail/``. This URL must respond with
a valid HTML5 page with a single anchor element per file for the project. The
href attribute **MUST** be a URL that links to the location of the file for
download, and the text of the anchor tag **MUST** match the final path
component (the filename) of the URL. The URL **SHOULD** include a hash in the
form of a URL fragment with the following syntax: ``#<hashname>=<hashvalue>``,
where ``<hashname>`` is the lowercase name of the hash function (such as
``sha256``) and ``<hashvalue>`` is the hex encoded digest.

In addition to the above, the following constraints are placed on the API:

* All URLs which respond with an HTML5 page **MUST** end with a ``/`` and the
  repository **SHOULD** redirect the URLs without a ``/`` to add a ``/`` to the
  end.

* URLs may be either absolute or relative as long as they point to the correct
  location.

* There are no constraints on where the files must be hosted relative to the
  repository.

* There may be any other HTML elements on the API pages as long as the required
  anchor elements exist.

* Repositories **MAY** redirect unnormalized URLs to the canonical normalized
  URL (e.g. ``/Foobar/`` may redirect to ``/foobar/``), however clients
  **MUST NOT** rely on this redirection and **MUST** request the normalized
  URL.

* Repositories **SHOULD** choose a hash function from one of the ones
  guaranteed to be available via the ``hashlib`` module in the Python standard
  library (currently ``md5``, ``sha1``, ``sha224``, ``sha256``, ``sha384``,
  ``sha512``). The current recommendation is to use ``sha256``.

* If there is a GPG signature for a particular distribution file it **MUST**
  live alongside that file with the same name with a ``.asc`` appended to it.
  So if the file ``/packages/HolyGrail-1.0.tar.gz`` existed and had an
  associated signature, the signature would be located at
  ``/packages/HolyGrail-1.0.tar.gz.asc``.

* A repository **MAY** include a ``data-gpg-sig`` attribute on a file link with
  a value of either ``true`` or ``false`` to indicate whether or not there is a
  GPG signature. Repositories that do this **SHOULD** include it on every link.

* A repository **MAY** include a ``data-requires-python`` attribute on a file
  link. This exposes the *Requires-Python* metadata field, specified in :pep:`345`,
  for the corresponding release. Where this is present, installer tools
  **SHOULD** ignore the download when installing to a Python version that
  doesn't satisfy the requirement. For example::

      <a href="..." data-requires-python="&gt;=3">...</a>

  In the attribute value, < and > have to be HTML encoded as ``&lt;`` and
  ``&gt;``, respectively.

Normalized Names
----------------

This PEP references the concept of a "normalized" project name. As per :pep:`426`
the only valid characters in a name are the ASCII alphabet, ASCII numbers,
``.``, ``-``, and ``_``. The name should be lowercased with all runs of the
characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This
can be implemented in Python with the ``re`` module::

   import re

   def normalize(name):
       return re.sub(r"[-_.]+", "-", name).lower()

Changes
-------

* The optional ``data-requires-python`` attribute was added in July 2016.


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								PEP: 503
 								Title: Simple Repository API
 								Version: $Revision$
 								Last-Modified: $Date$
 								Author: Donald Stufft <donald@stufft.io>
 								BDFL-Delegate: Donald Stufft <donald@stufft.io>
 								Discussions-To: distutils-sig@python.org
-												Multiple PEPs: Move Packaging PEPs to Standards Track and mark as Final (#2657)

These all introduce or update existing Python Packaging Specifications,
and should follow the PyPA Specification processes. These are expected
to be Standards Track PEPs and are not informational.

This change also ensures that they get listed under the correct section
on the Packaging topic index.
											
										
										
											2022-06-21 17:47:58 -04:00
+								Status: Final
 								Type: Standards Track
-												Several PEPs: Add `Topic: Packaging` (#2636)


											
										
										
											2022-06-14 17:22:20 -04:00
+								Topic: Packaging
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								Content-Type: text/x-rst
 								Created: 04-Sep-2015
 								Post-History: 04-Sep-2015
-												Accept the PEP

											
										
										
											2015-09-24 12:05:15 -04:00
+								Resolution: https://mail.python.org/pipermail/distutils-sig/2015-September/026899.html
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								Abstract
 								========
 								There are many implementations of a Python package repository and many tools
-												PEP 503: Fix typos, add clarification, and add the missing GPG signatures

											
										
										
											2015-09-05 12:11:45 -04:00
+								that consume them. Of these, the canonical implementation that defines what
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								the "simple" repository API looks like is the implementation that powers
 								PyPI. This document will specify that API, documenting what the correct
 								behavior for any implementation of the simple repository API.
 								Specification
 								=============
-												PEP 503: Update PyPI's base URL for Warehouse (#1036)


											
										
										
											2019-05-09 17:27:53 -04:00
+								A repository that implements the simple API is defined by its base URL, this is
 								the top level URL that all additional URLs are below. The API is named the
 								"simple" repository due to the fact that PyPI's base URL is
 								``https://pypi.org/simple/``.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								.. note:: All subsequent URLs in this document will be relative to this base
-												PEP 503: Reword to match pip’s URL lookup behavior (#962)

* Reword PEP 503 to match pip's URL lookup behavior

Clarify that the "filename" displayed in the anchor tag refers to the
final path component of the URL (instead of, say, the filename parameter
in the HTTP Content-Disposition header).

This matches pip's behavior: The URL's final path component is used to
determine the artifact's name (for selection), and its filename after
download.

* Replace "an URL" with "a URL"

											
										
										
											2019-10-11 07:15:25 -04:00
+								          URL (so given PyPI's URL, a URL of ``/foo/`` would be
-												PEP 503: Update PyPI's base URL for Warehouse (#1036)


											
										
										
											2019-05-09 17:27:53 -04:00
+								          ``https://pypi.org/simple/foo/``.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
-												Clarify that the CDATA for a repo index should match the project name, not the normalized name (#1324)


											
										
										
											2020-03-03 13:19:37 -05:00
+								Within a repository, the root URL (``/`` for this PEP which represents the base
 								URL) **MUST** be a valid HTML5 page with a single anchor element per project in
 								the repository. The text of the anchor tag **MUST** be the name of
 								the project and the href attribute **MUST** link to the URL for that particular
 								project. As an example::
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								   <!DOCTYPE html>
 								   <html>
 								     <body>
 								       <a href="/frob/">frob</a>
 								       <a href="/spamspamspam/">spamspamspam</a>
 								     </body>
 								   </html>
 								Below the root URL is another URL for each individual project contained within
 								a repository. The format of this URL is ``/<project>/`` where the ``<project>``
 								is replaced by the normalized name for that project, so a project named
-												PEP 503: Reword to match pip’s URL lookup behavior (#962)

* Reword PEP 503 to match pip's URL lookup behavior

Clarify that the "filename" displayed in the anchor tag refers to the
final path component of the URL (instead of, say, the filename parameter
in the HTTP Content-Disposition header).

This matches pip's behavior: The URL's final path component is used to
determine the artifact's name (for selection), and its filename after
download.

* Replace "an URL" with "a URL"

											
										
										
											2019-10-11 07:15:25 -04:00
+								"HolyGrail" would have a URL like ``/holygrail/``. This URL must respond with
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								a valid HTML5 page with a single anchor element per file for the project. The
-												PEP 503: Reword to match pip’s URL lookup behavior (#962)

* Reword PEP 503 to match pip's URL lookup behavior

Clarify that the "filename" displayed in the anchor tag refers to the
final path component of the URL (instead of, say, the filename parameter
in the HTTP Content-Disposition header).

This matches pip's behavior: The URL's final path component is used to
determine the artifact's name (for selection), and its filename after
download.

* Replace "an URL" with "a URL"

											
										
										
											2019-10-11 07:15:25 -04:00
+								href attribute **MUST** be a URL that links to the location of the file for
 								download, and the text of the anchor tag **MUST** match the final path
 								component (the filename) of the URL. The URL **SHOULD** include a hash in the
 								form of a URL fragment with the following syntax: ``#<hashname>=<hashvalue>``,
 								where ``<hashname>`` is the lowercase name of the hash function (such as
 								``sha256``) and ``<hashvalue>`` is the hex encoded digest.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								In addition to the above, the following constraints are placed on the API:
-												PEP 503: Fix typos, add clarification, and add the missing GPG signatures

											
										
										
											2015-09-05 12:11:45 -04:00
+								* All URLs which respond with an HTML5 page **MUST** end with a ``/`` and the
 								  repository **SHOULD** redirect the URLs without a ``/`` to add a ``/`` to the
 								  end.
 								* URLs may be either absolute or relative as long as they point to the correct
 								  location.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
-												PEP 503: Update PyPI's base URL for Warehouse (#1036)


											
										
										
											2019-05-09 17:27:53 -04:00
+								* There are no constraints on where the files must be hosted relative to the
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								  repository.
 								* There may be any other HTML elements on the API pages as long as the required
 								  anchor elements exist.
-												Fix various typos, spelling and grammar errors

Errors detected using Topy (https://github.com/intgr/topy), all changes
verified by hand.

											
										
										
											2016-07-11 11:14:08 -04:00
+								* Repositories **MAY** redirect unnormalized URLs to the canonical normalized
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								  URL (e.g. ``/Foobar/`` may redirect to ``/foobar/``), however clients
 								  **MUST NOT** rely on this redirection and **MUST** request the normalized
 								  URL.
 								* Repositories **SHOULD** choose a hash function from one of the ones
-												PEP 503: Fix typos, add clarification, and add the missing GPG signatures

											
										
										
											2015-09-05 12:11:45 -04:00
+								  guaranteed to be available via the ``hashlib`` module in the Python standard
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								  library (currently ``md5``, ``sha1``, ``sha224``, ``sha256``, ``sha384``,
 								  ``sha512``). The current recommendation is to use ``sha256``.
-												PEP 503: Fix typos, add clarification, and add the missing GPG signatures

											
										
										
											2015-09-05 12:11:45 -04:00
+								* If there is a GPG signature for a particular distribution file it **MUST**
 								  live alongside that file with the same name with a ``.asc`` appended to it.
 								  So if the file ``/packages/HolyGrail-1.0.tar.gz`` existed and had an
 								  associated signature, the signature would be located at
 								  ``/packages/HolyGrail-1.0.tar.gz.asc``.
-												Add the data-gpg-sig attribute to file links

											
										
										
											2015-09-24 12:01:20 -04:00
+								* A repository **MAY** include a ``data-gpg-sig`` attribute on a file link with
 								  a value of either ``true`` or ``false`` to indicate whether or not there is a
 								  GPG signature. Repositories that do this **SHOULD** include it on every link.
-												Add data-requires-python attribute to file links (PEP 503) (#56)


											
										
										
											2016-07-22 12:44:43 -04:00
+								* A repository **MAY** include a ``data-requires-python`` attribute on a file
-												Several PEPs: Use explicit `:pep:` and `:rfc:` roles (#2209)


											
										
										
											2022-01-21 06:03:51 -05:00
+								  link. This exposes the *Requires-Python* metadata field, specified in :pep:`345`,
-												Add data-requires-python attribute to file links (PEP 503) (#56)


											
										
										
											2016-07-22 12:44:43 -04:00
+								  for the corresponding release. Where this is present, installer tools
 								  **SHOULD** ignore the download when installing to a Python version that
 								  doesn't satisfy the requirement. For example::
 								      <a href="..." data-requires-python="&gt;=3">...</a>
 								  In the attribute value, < and > have to be HTML encoded as ``&lt;`` and
 								  ``&gt;``, respectively.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								Normalized Names
 								----------------
-												Several PEPs: Use explicit `:pep:` and `:rfc:` roles (#2209)


											
										
										
											2022-01-21 06:03:51 -05:00
+								This PEP references the concept of a "normalized" project name. As per :pep:`426`
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
+								the only valid characters in a name are the ASCII alphabet, ASCII numbers,
 								``.``, ``-``, and ``_``. The name should be lowercased with all runs of the
 								characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This
 								can be implemented in Python with the ``re`` module::
 								   import re
 								   def normalize(name):
 								       return re.sub(r"[-_.]+", "-", name).lower()
-												Add data-requires-python attribute to file links (PEP 503) (#56)


											
										
										
											2016-07-22 12:44:43 -04:00
+								Changes
 								-------
 								* The optional ``data-requires-python`` attribute was added in July 2016.
-												Add PEP 503 - Simple Repository API

											
										
										
											2015-09-04 21:11:56 -04:00
 								Copyright
 								=========
 								This document has been placed in the public domain.
 								..
 								   Local Variables:
 								   mode: indented-text
 								   indent-tabs-mode: nil
 								   sentence-end-double-space: t
 								   fill-column: 70
 								   coding: utf-8
 								   End: