diff --git a/peps/pep-0752.rst b/peps/pep-0752.rst index f73b31b1e..ab29d0f5e 100644 --- a/peps/pep-0752.rst +++ b/peps/pep-0752.rst @@ -1,5 +1,5 @@ PEP: 752 -Title: Package repository namespaces +Title: Implicit namespaces for package repositories Author: Ofek Lev Sponsor: Barry Warsaw PEP-Delegate: Dustin Ingram @@ -75,9 +75,6 @@ because typos would have to be in the prefix itself which is Rationale ========= -Tolerance for Disruption ------------------------- - Other package ecosystems have generally solved this problem by taking one of two approaches: either minimizing or maximizing backwards compatibility. @@ -111,34 +108,9 @@ __ https://learn.microsoft.com/en-us/nuget/nuget-org/id-prefix-reservation __ https://devblogs.microsoft.com/nuget/Package-identity-and-trust/ This PEP specifies the NuGet approach of authorized reservation across a flat -namespace for the following reasons: - -* Causing churn for the community is a hard blocker. -* The NPM approach has the potential to cause confusion for users if we allow - unscoped names. Our community has chosen to normalize separator characters - and so ``@aws/s3`` would likely be confused with ``@aws-s3``. - -Approval Process ----------------- - -PyPI has been understaffed, receiving the first `dedicated specialist`__ in -July 2024. Due to lack of resources, user support has been lacking for -`package name claims `__, -`organization requests `__, -`storage limit increases `__, -and even `account recovery `__. - -__ https://pyfound.blogspot.com/2024/07/announcing-our-new-pypi-support.html - -The `default policy `_ of only allowing -`corporate organizations `_ to reserve namespaces (except in -specific scenarios) provides the following benefits: - -* PyPI would have a constant source of funding for support specialists, - infrastructure maintenance and new features. -* Although each application would require independent review, less human - feedback would be required because the process to approve a paid organization - already bestows a certain amount of trust. +namespace. Any solution that requires new package syntax must be built atop the +existing flat namespace and therefore implicit namespaces acquired via a +reservation mechanism would be a prerequisite to such explicit namespaces. Terminology =========== @@ -149,23 +121,37 @@ interpreted as described in :rfc:`2119`. Organization `Organizations `_ are entities that own projects and have various - users associated with them. Only organizations have access to the namespace - application form. -Corporate Organization - `Corporate organizations `_ are organizations that have paid - for special functionality on PyPI. -Shared Namespace - A `shared `_ namespace allows for uploads from any - project owner. -Private Namespace - A private namespace only allows uploads from an owner of the namespace. + users associated with them. +Grant + A grant is a reservation of a namespace for a package repository. +Open Namespace + An `open `_ namespace allows for uploads from any project + owner. +Restricted Namespace + A restricted namespace only allows uploads from an owner of the namespace. +Parent Namespace + A namespace's parent refers to the namespace without the trailing + hyphenated component e.g. the parent of ``foo-bar`` is ``foo``. +Child Namespace + A namespace's child refers to the namespace with an additional trailing + hyphenated component e.g. the child of ``foo`` is ``foo-bar``. Specification ============= -`Organizations `_ (NOT regular users) MAY reserve one or more -namespaces. Such reservations neither confer ownership nor grant special -privileges to existing projects. +.. _orgs: + +Organizations +------------- + +Any package repository that allows for the creation of projects (e.g. +non-mirrors) MAY offer the concept of `organizations`__. Organizations +are entities that own projects and have various users associated with them. + +__ https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ + +Organizations MAY reserve one or more namespaces. Such reservations neither +confer ownership nor grant special privileges to existing projects. .. _naming: @@ -173,15 +159,13 @@ Naming ------ A namespace MUST be a `valid`__ project name and `normalized`__ internally e.g. -``foo.bar`` would become ``foo-bar``. The user facing namespace (e.g. in UI -tooltips) MUST preserve the original pre-normalized text as defined during -reservation. +``foo.bar`` would become ``foo-bar``. __ https://packaging.python.org/en/latest/specifications/name-normalization/#name-format __ https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization -Grant Semantics ---------------- +Semantics +--------- A namespace grant bestows ownership over the following: @@ -194,56 +178,16 @@ A namespace grant bestows ownership over the following: Package name matching acts upon the `normalized `_ namespace. Namespaces are per-package repository and SHALL NOT be shared between -repositories. Each repository is responsible for managing namespaces the way -they want, and repositories are not expected to replicate namespaces from one -repository to another. +repositories. For example, if PyPI has a namespace ``microsoft`` that is owned +by the company Microsoft, packages starting with ``microsoft-`` that come from +other non-PyPI mirror repositories do not confer the same level of trust. -Grant Types ------------ - -There are two types of grants. - -.. _root-grant: - -Root Grant -'''''''''' - -Only `organizations `_ have the ability to submit requests for namespace -grants. An organization gets a root grant for every accepted request. This -grant may produce any number of `child grants `_. - -Root grants MUST NOT overlap. For example, if there is an existing root grant -for ``foo-bar`` then an organization cannot `apply `_ for -``foo``. An overlap is determined by comparing the `normalized `_ -proposed namespace with the normalized namespace of every existing root grant. -Every comparison must append a hyphen to the end of the proposed and existing -namespace. An overlap is detected when any existing namespace starts with the -proposed namespace. - -.. _child-grant: - -Child Grant -''''''''''' - -A child grant is created by the owner of a `root grant `_. The -child namespace MUST be prefixed by the root grant namespace followed by a -hyphen. For example, ``google-cloud`` would be a valid child of the root -namespace ``google``. - -Child grants cannot have their own child grants. - -.. _grant-ownership: - -Grant Ownership ---------------- - -The owner of a grant may allow any number of other organizations to use the -grant. The grants behave as if they were owned by the organization. The owner -may revoke this permission at any time. - -The owner may transfer ownership to another organization. If the organization -is a corporate organization, the target for transfer must also be. Settings for -permitted organizations are transferred as well. +Grants MUST NOT overlap. For example, if there is an existing grant +for ``foo-bar`` then a new grant for ``foo`` would be forbidden. An overlap is +determined by comparing the `normalized `_ proposed namespace with the +normalized namespace of every existing root grant. Every comparison must append +a hyphen to the end of the proposed and existing namespace. An overlap is +detected when any existing namespace starts with the proposed namespace. .. _uploads: @@ -259,180 +203,84 @@ If the following criteria are all true for a given upload: Then the upload MUST fail with a 403 HTTP status code. -.. _user-interface: +.. _open-namespaces: -User Interface --------------- - -Every project's page -(`example `__) -that matches an active namespace grant MUST indicate what the prefix is (NuGet -currently does not do this). This value MUST match the ``namespace.name`` key -in the `API `_. - -Clicking on the namespace MUST take the user to a page that has more -information such as the current owners and the time at which ownership was -granted. This page SHOULD enumerate every project that matches the namespace -and display a count of the total number of projects. - -Any indicator used in the following scenarios for projects that match an active -namespace grant MUST be distinct from one another: - -1. Projects that are tied to a `grant owner `_ MAY have an - indicator. -2. Projects that are not tied to a `grant owner `_ and the - matching grant is `shared `_ MUST have an indicator. -3. Projects that are not tied to a `grant owner `_ and the - matching grant is private MUST have an indicator. This situation arises when - the project existed before the grant was created. - -Repositories MUST NOT offer a way to view all active namespace grants. This is -to prevent leaking potentially private information such as an upcoming product. - -.. _shared-namespaces: - -Shared Namespaces +Open Namespaces ----------------- The owner of a grant may choose to allow others the ability to release new projects with the associated namespace. Doing so MUST allow -`uploads `_ for new projects matching the namespace from any user -but such projects MUST have a distinct `visual indicator `_. +`uploads `_ for new projects matching the namespace from any user. -It is possible for the `owner `_ of a namespace to both make -it shared and allow other organizations to use the grant. In this case, the -permitted organizations have no special permissions and are equivalent to a -shared grant without ownership. - -When a `child grant `_ is created, its shared status SHALL be -inherited from the `root grant `_. Owners of child grants MAY -make them shared at any time. If a grant is shared, it MUST NOT be made private -unless the owner of the grant is the owner of every project that matches the -namespace. - -Root grants given to `community projects `_ SHOULD -only be shared but is ultimately up to the reviewer of the application. +It is possible for the owner of a namespace to both make it open and allow +other organizations to use the grant. In this case, the authorized +organizations have no special permissions and are equivalent to an open grant +without ownership. .. _repository-metadata: Repository Metadata ------------------- -To allow installers and other tooling insight into this project-level metadata -of a namespaced project, the :pep:`JSON API <691>` version will be incremented -and support new keys for the project endpoint. +The :pep:`JSON API <691>` version will be incremented from ``1.0`` to ``1.1``. +The following API changes MUST be implemented by repositories that support +this PEP. Repositories that do not support this PEP MUST NOT implement these +changes so that consumers of the API are able to determine whether the +repository supports this PEP. -The ``owner`` key SHOULD be added and refer to the owner of the project, -whether an organization or a user. +.. _project-detail: -The ``namespace`` key MAY be added and MUST be ``null`` if the project does not -match an active namespace grant. If the project does match a namespace grant, -the value MUST be a mapping with the following keys: +Project Detail +'''''''''''''' -* ``name``: This is the associated `normalized `_ namespace e.g. +The :pep:`project detail <691#project-detail>` response will be modified as +follows. + +The ``namespace`` key MUST be ``null`` if the project does not match an active +namespace grant. If the project does match a namespace grant, the value MUST be +a mapping with the following keys: + +* ``prefix``: This is the associated `normalized `_ namespace e.g. ``foo-bar``. If the owner of the project owns multiple matching grants then this MUST be the namespace with the most number of characters. For example, if the project name matched both ``foo-bar`` and ``foo-bar-baz`` then this key would be the latter. -* ``owners``: This is an array of organizations that - `own `_ the grant. This is useful for tools that wish to - make a distinction between official and community packages by checking if - the array contains the project ``owner``. -* ``shared``: This is a boolean indicating whether the namespace is - `shared `_. +* ``authorized``: This is a boolean and will be true if the project owner + is an organization and is one of the current owners of the grant. This is + useful for tools that wish to make a distinction between official and + community packages. +* ``open``: This is a boolean indicating whether the namespace is + `open `_. -The presence of the ``namespace`` key indicates support for this PEP. +Namespace Detail +'''''''''''''''' + +The format of this URL is ``/namespace/`` where ```` is +the `normalized `_ namespace. For example, the URL for the namespace +``foo.bar`` would be ``/namespace/foo-bar``. + +The response will be a mapping with the following keys: + +* ``prefix``: This is the `normalized `_ version of the namespace e.g. + ``foo-bar``. +* ``owner``: This is the organization that is responsible for the namespace. +* ``open``: This is a boolean indicating whether the namespace is + `open `_. +* ``parent``: This is the parent namespace if it exists. For example, if the + namespace is ``foo-bar`` and there is an active grant for ``foo``, then this + would be ``"foo"``. If there is no parent then this key will be ``null``. +* ``children``: This is an array of any child namespaces. For example, if the + namespace is ``foo`` and there are active grants for ``foo-bar`` and + ``foo-bar-baz`` then this would be ``["foo-bar", "foo-bar-baz"]``. Grant Removal ------------- -If a grant is shared with other organizations, the owner organization MUST -initiate a transfer as a prerequisite for organization deletion. +When a reserved namespace becomes unclaimed, repositories MUST set the +``namespace`` key to ``null`` in the `API `_. -If a grant is not shared, the owner may unclaim the namespace in either of the -following circumstances: - -* The organization manually removes themselves as the owner. -* The organization is deleted. - -When a reserved namespace becomes unclaimed, repositories: - -1. MUST remove the `visual indicator `_ -2. MUST remove the ``namespace`` key in the `API `_ - -Grant Applications ------------------- - -.. _grant-application: - -Submission -'''''''''' - -Only `organizations `_ have access to the page for submitting grant -applications. Reviews of `corporate organizations `_ applications -MUST be prioritized. - -.. _grant-approval-criteria: - -Approval Criteria -''''''''''''''''' - -1. The namespace MUST NOT be something common like ``tool`` or ``apps``. -2. The namespace SHOULD be greater than three characters. -3. The namespace SHOULD properly and clearly identify the reservation owner. -4. The organization SHOULD be actively using the namespace. -5. There SHOULD be evidence that *not* reserving the namespace may cause - ambiguity, confusion, or other harm to the community. - -Organizations that are not `corporate organizations `_ MUST -represent one of the following: - -* Large, popular open-source projects with many packages [2]_ -* Universities that actively publish packages -* Government organizations that actively publish packages -* NPOs/NGOs that actively publish packages like - `Our World in Data `__ - -Recommendations -=============== - -Visual Indicators ------------------ - -For projects that match an active namespace grant, the namespace is always -shown in `the UI `_. It is recommended that this should -stand out as a pill or label. - -There are three types of visual indicators for each type of project: - -1. There should be no indicator for projects that are tied to a - `grant owner `_ and users should solely rely on the - always-present namespace. If an indicator is chosen, it should not be - a check mark or similar because it may mistakingly convey that there are - associated security guarantees inherent to the use of the package. - Additionally, some social media platforms use a check mark for verified - users which may cause confusion. -2. Projects that are not tied to a `grant owner `_ and are - part of a `shared `_ namespace should never have an - indicator that conveys mistrust or danger. A good choice might be the - `users `_ icon from Font Awesome or the - `groups`__ icon from Google Fonts. -3. Projects that are not tied to a `grant owner `_ and are - part of a private grant (i.e. existed before the grant) should have an - indicator that conveys inauthenticity or lack of trust. A good choice might - be a warning sign (⚠). - -__ https://fonts.google.com/icons?selected=Material+Symbols+Outlined:groups - -Grant Applications ------------------- - -Generally speaking, reviewers should be more tolerant of corporate -organizations that apply for grants for which they are not yet using. - -For example, while it's reasonable to grant a namespace to a startup or an -existing company with a new product line, it's not as reasonable to grant a -namespace to a community project that doesn't have many users. +Namespaces that were previously claimed but are now not SHOULD be eligible for +claiming again by any organization. Backwards Compatibility ======================= @@ -446,10 +294,6 @@ __ https://github.com/python/typeshed/issues/2491#issuecomment-578456045 Security Implications ===================== -* Although users will no longer see the visual indicator when a namespace - becomes unclaimed, external consumers of metadata may have difficulty - scraping the user facing - `enumeration `_ of grants to verify current ownership. * There is an opportunity to build on top of :pep:`740` and :pep:`480` so that one could prove cryptographically that a specific release came from an owner of the associated namespace. This PEP makes no effort to describe how this @@ -458,13 +302,10 @@ Security Implications How to Teach This ================= -For organizations, we will document how to reserve namespaces, what the -benefits are and pricing. - -For consumers of packages we will document the indicator on release pages, how -metadata is exposed in the `API `_ and potentially in -future note tooling that supports utilizing namespaces to provide extra -security guarantees during installation. +For consumers of packages we will document how metadata is exposed in the +`API `_ and potentially in future note tooling that +supports utilizing namespaces to provide extra security guarantees during +installation. Reference Implementation ======================== @@ -498,43 +339,12 @@ shuffling, an acquisition, or any other reason. Whenever this happens every project they own would in effect be renamed which would cause unnecessary confusion for users, frequently. -Users have come to expect that package names may be typed without worry of -conflicting shell syntax and any namespace solution would pose challenges: - -* Copying NPM's syntax (e.g. ``@foo/bar``) would alienate a large number of - Windows users because the ``@`` character is considered special in - `PowerShell`__. -* Starting names with a ``/`` would conflict with the common installer - capability of accepting paths without URI ``file://`` syntax. -* Starting names with a ``//`` like Bazel - `target patterns `__ - would be confusing to users because the current normalization standard - eliminates consecutive separator characters. - -__ https://learn.microsoft.com/en-us/powershell/scripting/lang-spec/chapter-07?view=powershell-7.4#717--operator - Finally, the disruption to the community would be massive because it would require an update from every package manager, security scanner, IDE, etc. New packages released with the scoping would be incompatible with older tools and would cause confusion for users along with frustration from maintainers having to triage such complaints. -Allow Private Namespaces for Community Projects ------------------------------------------------ - -This PEP enforces that the discretionary namespace grants for community -projects are `shared `_. This is almost always desired by -such projects and prevents the following situations: - -* A perceived reduction in openness of community projects, for example if a - project was taken over by a business entity there may be a desire for it to - prevent the creation of new packages matching the namespace. -* When an existing community project with plugins (such as MkDocs) chooses to - reserve a namespace, future plugins that are officially adopted would have to - change their name. This would cause a massive disruption to users and reset - usage statistics. The workaround is to have a new package that is advertised - which would depend on the real package but this is suboptimal. - Open Issues =========== @@ -594,9 +404,6 @@ __ https://github.com/open-telemetry/opentelemetry-python-contrib __ https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/plugins.html __ https://airflow.apache.org/docs/apache-airflow-providers/index.html -.. _orgs: https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ -.. _corp-orgs: https://docs.pypi.org/organization-accounts/pricing-and-payments/#corporate-organizations - Copyright =========