diff --git a/pep-0727.rst b/pep-0727.rst index da9b018e2..d8f900415 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -2,22 +2,22 @@ PEP: 727 Title: Documentation Metadata in Typing Author: Sebastián Ramírez Sponsor: Jelle Zijlstra -Discussions-To: https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566 +Discussions-To: https://discuss.python.org/t/32566 Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 28-Aug-2023 Python-Version: 3.13 -Post-History: `30-Aug-2023 `__ +Post-History: `30-Aug-2023 `__ Abstract ======== This document proposes a way to complement docstrings to add additional documentation -to Python symbols using type annotations with ``Annotated`` (in class attributes, -function and method parameters, return values, and variables). +to Python symbols using type annotations with :py:class:`~typing.Annotated` +(in class attributes, function and method parameters, return values, and variables). Motivation @@ -60,7 +60,7 @@ documentation in some other way (e.g. an API, a CLI, etc). Some of these previous formats tried to account for the lack of type annotations in older Python versions by including typing information in the docstrings, but now that information doesn't need to be in docstrings as there is now an official -syntax for type annotations. +:pep:`syntax for type annotations <484>`. Rationale @@ -84,79 +84,56 @@ like to adopt it. Specification ============= +The main proposal is to introduce a new function, ``typing.doc()``, +to be used when documenting Python objects. +This function MUST only be used within :py:class:`~typing.Annotated` annotations. +The function takes a single string argument, ``documentation``, +and returns an instance of ``typing.DocInfo``, +which stores the input string unchanged. -``typing.doc`` --------------- +Any tool processing ``typing.DocInfo`` objects SHOULD interpret the string as +a docstring, and therefore SHOULD normalize whitespace +as if ``inspect.cleandoc()`` were used. -The main proposal is to have a new function ``doc()`` in the ``typing`` module. -Even though this is not strictly related to the type annotations, it's expected -to go in ``Annotated`` type annotations, and to interact with type annotations. +The string passed to ``typing.doc()`` SHOULD be of the form that would be a valid docstring. +This means that `f-strings`__ and string operations SHOULD NOT be used. +As this cannot be enforced by the Python runtime, +tools SHOULD NOT rely on this behaviour, +and SHOULD exit with an error if such a prohibited string is encountered. -There's also the particular benefit that it could be implemented in the -``typing_extensions`` package to have support for older versions of Python and -early adopters of this proposal. +__ https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals -This ``doc()`` function would receive one single parameter ``documentation`` with -a documentation string. +Examples +-------- -This string could be a multi-line string, in which case, when extracted by tools, -should be interpreted cleaning up indentation as if using ``inspect.cleandoc()``, -the same procedure used for docstrings. +Class attributes may be documented: -This string could probably contain markup, like Markdown or reST. As that could -be highly debated, that decision is left for a future proposal, to focus here -on the main functionality. +.. code:: python -This specification targets static analysis tools and editors, and as such, the -value passed to ``doc()`` should allow static evaluation and analysis. If a -developer passes as the value something that requires runtime execution -(e.g. a function call) the behavior of static analysis tools is unspecified -and they could omit it from their process and results. For static analysis -tools to be conformant with this specification they need only to support -statically accessible values. + from typing import Annotated, doc -An example documenting the attributes of a class, or in this case, the keys -of a ``TypedDict``, could look like this: + class User: + first_name: Annotated[str, doc("The user's first name")] + last_name: Annotated[str, doc("The user's last name")] -.. code-block:: + ... - from typing import Annotated, TypedDict, NotRequired, doc +As can function or method parameters: +.. code:: python - class User(TypedDict): - firstname: Annotated[str, doc("The user's first name")] - lastname: Annotated[str, doc("The user's last name")] + from typing import Annotated, doc + def create_user( + first_name: Annotated[str, doc("The user's first name")], + last_name: Annotated[str, doc("The user's last name")], + cursor: DatabaseConnection | None = None, + ) -> Annotated[User, doc("The created user after saving in the database")]: + """Create a new user in the system. -An example documenting the parameters of a function could look like this: - -.. code-block:: - - from typing import Annotated, doc - - - def create_user( - lastname: Annotated[str, doc("The **last name** of the newly created user")], - firstname: Annotated[str | None, doc("The user's **first name**")] = None, - ) -> Annotated[User, doc("The created user after saving in the database")]: - """ - Create a new user in the system, it needs the database connection to be already - initialized. - """ - pass - - -The return of the ``doc()`` function is an instance of a class that can be checked -and used at runtime, defined similar to: - -.. code-block:: - - class DocInfo: - def __init__(self, documentation: str): - self.documentation = documentation - -...where the attribute ``documentation`` contains the same value string passed to -the function ``doc()``. + It needs the database connection to be already initialized. + """ + pass Additional Scenarios @@ -171,34 +148,30 @@ but implementers are not required to support them. Type Alias ----------- +'''''''''' When creating a type alias, like: -.. code-block:: +.. code:: python - Username = Annotated[str, doc("The name of a user in the system")] + Username = Annotated[str, doc("The name of a user in the system")] -...the documentation would be considered to be carried by the parameter annotated +The documentation would be considered to be carried by the parameter annotated with ``Username``. So, in a function like: -.. code-block:: +.. code:: python - def hi( - to: Username, - ) -> None: ... + def hi(to: Username) -> None: ... -...it would be equivalent to: +It would be equivalent to: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("The name of a user in the system")], - ) -> None: ... + def hi(to: Annotated[str, doc("The name of a user in the system")]) -> None: ... Nevertheless, implementers would not be required to support type aliases outside of the final type annotation to be conformant with this specification, as it @@ -206,17 +179,17 @@ could require more complex dereferencing logic. Annotating Type Parameters --------------------------- +'''''''''''''''''''''''''' When annotating type parameters, as in: -.. code-block:: +.. code:: python - def hi( - to: list[Annotated[str, doc("The name of a user in a list")]], - ) -> None: ... + def hi( + to: list[Annotated[str, doc("The name of a user in a list")]], + ) -> None: ... -...the documentation in ``doc()`` would refer to what it is annotating, in this +The documentation in ``doc()`` would refer to what it is annotating, in this case, each item in the list, not the list itself. There are currently no practical use cases for documenting type parameters, @@ -225,17 +198,17 @@ conformant, but it's included for completeness. Annotating Unions ------------------ +''''''''''''''''' If used in one of the parameters of a union, as in: -.. code-block:: +.. code:: python - def hi( - to: str | Annotated[list[str], doc("List of user names")], - ) -> None: ... + def hi( + to: str | Annotated[list[str], doc("List of user names")], + ) -> None: ... -...again, the documentation in ``doc()`` would refer to what it is annotating, +Again, the documentation in ``doc()`` would refer to what it is annotating, in this case, this documents the list itself, not its items. In particular, the documentation would not refer to a single string passed as a @@ -247,7 +220,7 @@ included for completeness. Nested ``Annotated`` --------------------- +'''''''''''''''''''' Continuing with the same idea above, if ``Annotated`` was used nested and used multiple times in the same parameter, ``doc()`` would refer to the type it @@ -255,14 +228,15 @@ is annotating. So, in an example like: -.. code-block:: +.. code:: python - def hi( - to: Annotated[ - Annotated[str, doc("A user name")] | Annotated[list, doc("A list of user names")], - doc("Who to say hi to"), - ], - ) -> None: ... + def hi( + to: Annotated[ + Annotated[str, doc("A user name")] + | Annotated[list, doc("A list of user names")], + doc("Who to say hi to"), + ], + ) -> None: ... The documentation for the whole parameter ``to`` would be considered to be @@ -281,16 +255,16 @@ of the parameter passed is of one type or another, but they are not required to Duplication ------------ +''''''''''' If ``doc()`` is used multiple times in a single ``Annotated``, it would be considered invalid usage from the developer, for example: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("A user name"), doc("The current user name")], - ) -> None: ... + def hi( + to: Annotated[str, doc("A user name"), doc("The current user name")], + ) -> None: ... Implementers can consider this invalid and are not required to support this to be @@ -302,46 +276,68 @@ can opt to support one of the ``doc()`` declarations. In that case, the suggestion would be to support the last one, just because this would support overriding, for example, in: -.. code-block:: +.. code:: python - User = Annotated[str, doc("A user name")] + User = Annotated[str, doc("A user name")] - CurrentUser = Annotated[User, doc("The current user name")] + CurrentUser = Annotated[User, doc("The current user name")] Internally, in Python, ``CurrentUser`` here is equivalent to: -.. code-block:: +.. code:: python - CurrentUser = Annotated[str, doc("A user name"), doc("The current user name")] + CurrentUser = Annotated[str, + doc("A user name"), + doc("The current user name")] For an implementation that supports the last ``doc()`` appearance, the above example would be equivalent to: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("The current user name")], - ) -> None: ... + def hi(to: Annotated[str, doc("The current user name")]) -> None: ... -Early Adopters and Older Python Versions -======================================== +.. you need to fill these in: -For older versions of Python and early adopters of this proposal, ``doc()`` and -``DocInfo`` can be imported from the ``typing_extensions`` package. + Backwards Compatibility + ======================= -.. code-block:: - - from typing import Annotated - - from typing_extensions import doc + [Describe potential impact and severity on pre-existing code.] - def hi( - to: Annotated[str, doc("The current user name")], - ) -> None: ... + Security Implications + ===================== + + [How could a malicious user take advantage of this new feature?] + + + How to Teach This + ================= + + [How to teach users, new and experienced, how to apply the PEP to their work.] + + +Reference Implementation +======================== + +``typing.doc`` and ``typing.DocInfo`` are implemented as follows: + +.. code:: python + + def doc(documentation: str, /) -> DocInfo: + return DocInfo(documentation) + + class DocInfo: + def __init__(self, documentation: str, /): + self.documentation = documentation + + +These have been implemented in the `typing_extensions`__ package. + +__ https://pypi.org/project/typing-extensions/ Rejected Ideas @@ -407,8 +403,8 @@ to be used by those that are willing to take the extra verbosity in exchange for the benefits. -Doc is not Typing ------------------ +Documentation is not Typing +--------------------------- It could also be argued that documentation is not really part of typing, or that it should live in a different module. Or that this information should not be part