diff --git a/pep-0685.rst b/pep-0685.rst index 21cbaaf78..922d405d5 100644 --- a/pep-0685.rst +++ b/pep-0685.rst @@ -37,7 +37,7 @@ something tool authors can agree to following. The issue of there being no standard was brought forward by an `initial discussion `__ noting that the extra ``adhoc-ssl`` was not considered equal to the name -``adhoc_ssl`` by pip. +``adhoc_ssl`` by pip 22. Rationale @@ -51,57 +51,70 @@ This collapses any run of the substitution character down to a single character, e.g. ``---`` gets collapsed down to ``-``. This does not produce a valid Python identifier as specified by the -core metadata specification for extra names. +core metadata 2.2 specification for extra names. -`Setuptools does normalization `__ +`Setuptools 60 does normalization `__ via:: re.sub('[^A-Za-z0-9.-]+', '_', name).lower() The use of an underscore/``_`` differs from PEP 503's use of a hyphen/``-``. -Runs of characters, unlike PEP 503, do **not** get collapsed, +Runs of ``_``, unlike PEP 503, do **not** get collapsed, e.g. ``___`` stays the same. -For pip, its -"extra normalisation behaviour is quite convoluted and erratic", +For pip 22, its +"extra normalisation behaviour is quite convoluted and erratic" [pip-erratic]_, and so its use is not considered. +.. [pip-erratic] https://discuss.python.org/t/what-extras-names-are-treated-as-equal-and-why/7614/10? + Specification ============= When comparing extra names, tools MUST normalize the names being compared -using the equivalent semantics of:: +using the semantics outlined in `PEP 503 for names `__:: - re.sub('[^A-Za-z0-9.-]+', '_', name).lower() + re.sub(r"[-_.]+", "-", name).lower() -This normalizes any extra name previously allowed by :pep:`508` in a -fashion consistent with setuptools. +The `core metadata`_ specification will be updated such that the allowed +names for `Provides-Extra`_ matches what :pep:`508` specifies for names. +As this is a superset of what is currently allowed by the core metadata 2.2 +specification, +it allows for a *loosening* of the naming requirements. +It will also bring extra naming in line with that of the Name_ field. For tools writing `core metadata`_, they MUST write out extra names in their normalized form. -This applies to the ``Provides-Extra`` and ``Provides-Dist`` fields, -both when specifying extras for a distribution as well as the -``extra`` marker. -This will also help enforce the current requirement from the core -metadata specification that extra names be valid Python identifiers. +This applies to the `Provides-Extra`_ field and the `extra marker`_ +when used in the `Requires-Dist`_ field. Tools generating metadata MUST raise an error if a user specified two or more extra names which would normalize to the same name. +Tools SHOULD warn users when an invalid extra name is read. Backwards Compatibility ======================= -Older distributions which contain conflicting names when normalized -will no longer have all of their extra names made available to users -as independent options, but instead as a single extra. -It is hoped that relying on setuptools' algorithm for normalization -will minimize the breakage from this. +Moving to :pep:`503` normalization and :pep:`508` name acceptance, it +allows for all preexisting, valid names to continue to be valid. -As distributions make new releases using tools which implement this PEP, -the backwards-compatibility issues will become less of a concern. +Based on research looking at a collection of wheels on PyPI [pypi-results]_, +the risk of extra name clashes is limited to 73 clashes when considering +even invalid names, +while *only* looking at valid names leads to only 3 clashes: + +1. dev-test: dev_test, dev-test, dev.test +2. dev-lint: dev-lint, dev.lint, dev_lint +3. apache-beam: apache-beam, apache.beam + +By requiring tools writing core metadata to only record the normalized name, +the issue of preexisting, invalid extra names should be diminished over +time. + +.. [pypi-results] https://discuss.python.org/t/pep-685-comparison-of-extra-names-for-optional-distribution-dependencies/14141/17?u=brettcannon Security Implications @@ -110,9 +123,10 @@ Security Implications It is possible that for a distribution that has conflicting extra names, a tool ends up installing distributions that somehow weaken the security of the system. -This is only hypothetical and if it were to occur, it would probably be -more of a security concern for the distributions specifying such extras names -rather than the distribution that pulled them in together. +This is only hypothetical and if it were to occur, +it would probably be more of a security concern for the distributions +specifying such extras names rather than the distribution that pulled +them in together. How to Teach This @@ -160,6 +174,8 @@ CC0-1.0-Universal license, whichever is more permissive. .. _core metadata: https://packaging.python.org/en/latest/specifications/core-metadata/ +.. _extra marker: https://peps.python.org/pep-0508/#extras +.. _Name: https://packaging.python.org/en/latest/specifications/core-metadata/#name .. _packaging project: https://packaging.pypa.io .. _Provides-Extra: https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use - +.. _Requires-Dist: https://packaging.python.org/en/latest/specifications/core-metadata/#requires-dist-multiple-use