diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a90a99d75..e65be54cc 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -624,6 +624,7 @@ peps/pep-0742.rst @JelleZijlstra peps/pep-0743.rst @vstinner peps/pep-0744.rst @brandtbucher peps/pep-0745.rst @hugovk +peps/pep-0746.rst @jellezijlstra # ... # peps/pep-0754.rst # ... diff --git a/peps/pep-0746.rst b/peps/pep-0746.rst new file mode 100644 index 000000000..cfcb290db --- /dev/null +++ b/peps/pep-0746.rst @@ -0,0 +1,149 @@ +PEP: 746 +Title: Type checking Annotated metadata +Author: Adrian Garcia Badaracco +Sponsor: Jelle Zijlstra +Discussions-To: https://discuss.python.org/t/pep-746-typedmetadata-for-type-checking-of-pep-593-annotated/53834 +Status: Draft +Type: Standards Track +Topic: Typing +Created: 20-May-2024 +Python-Version: 3.14 +Post-History: 20-May-2024 + +Abstract +======== + +This PEP proposes a mechanism for type checking metadata that uses +the :py:data:`typing.Annotated` type. Metadata objects that implement +the new ``__supports_type__`` protocol will be type checked by static +type checkers to ensure that the metadata is valid for the given type. + +Motivation +========== + +:pep:`593` introduced ``Annotated`` as a way to attach runtime metadata to types. +In general, the metadata is not meant for static type checkers, but even so, +it is often useful to be able to check that the metadata makes sense for the given +type. + +Take the first example in :pep:`593`, which uses ``Annotated`` to attach +serialization information to a field:: + + class Student(struct2.Packed): + name: Annotated[str, struct2.ctype("<10s")] + +Here, the ``struct2.ctype("<10s")`` metadata is meant to be used by a serialization +library to serialize the field. Such libraries can only serialize a subset of types: +it would not make sense to write, for example, ``Annotated[list[str], struct2.ctype("<10s")]``. +Yet the type system provides no way to enforce this. The metadata are completely +ignored by type checkers. + +This use case comes up in libraries like :pypi:`pydantic`, which use +``Annotated`` to attach validation and conversion information to fields. + +Specification +============= + +This PEP introduces a new ``__supports_type__`` protocol that both static and +runtime type checkers can use to understand if a metadata object in +``Annotated`` is valid for the given type. Objects that implement this protocol +must have a method named ``__supports_type__`` that takes a single positional argument and +returns ``bool``:: + + class Int64: + def __supports_type__(self, obj: int) -> bool: + return isinstance(obj, int) + +When a static type checker encounters a type expression of the form ``Annotated[T, M1, M2, ...]``, +it should enforce that for each metadata element in ``M1, M2, ...``, one of the following is true: + +* The metadata element evaluates to a type that does not have a ``__supports_type__`` method; or +* The metadata element evaluates to an object ``M`` that has a ``__supports_type__`` method, and + a call to ``M.__supports_type__(T)`` type checks without errors (i.e., ``T`` is assignable to the + parameter type of the ``__supports_type__`` method), and that call does not + evaluate to ``Literal[False]``. + +The body of the ``__supports_type__`` method is not used to check the validity of the metadata +and static type checkers can ignore it. However, tools that use the annotation at +runtime may call the method to check that a particular value is valid. + +For example, to support a generic ``Gt`` metadata, one might write:: + + from typing import Protocol + + class SupportsGt[T](Protocol): + def __gt__(self, __other: T) -> bool: + ... + + class Gt[T]: + def __init__(self, value: T) -> None: + self.value = value + + def __supports_type__(self, obj: SupportsGt[T]) -> bool: + return obj > self.value + + x1: Annotated[int, Gt(0)] = 1 # OK + x2: Annotated[str, Gt(0)] = 0 # type checker error: str is not assignable to SupportsGt[int] + x3: Annotated[int, Gt(1)] = 0 # OK for static type checkers; runtime type checkers may flag this + +Implementations may be generic and may use overloads that return ``Literal[True]`` or ``Literal[False]`` +to indicate if the metadata is valid for the given type. + +Backwards Compatibility +======================= + +Metadata that does not implement the protocol will be considered valid for all types, +so no breaking changes are introduced for existing code. The new checks only apply +to metadata objects that explicitly implement the protocol specified by this PEP. + +Security Implications +===================== + +None. + +How to Teach This +================= + +This protocol is intended mostly for libraries that provide ``Annotated`` metadata; +end users of those libraries are unlikely to need to implement the protocol themselves. +The protocol should be mentioned in the documentation for :py:data:`typing.Annotated` and +in the typing specification. + +Reference Implementation +======================== + +None yet. + +Rejected ideas +============== + +Introducing a type variable instead of a generic class +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We considered using a special type variable, ``AnnotatedT = TypeVar("AnnotatedT")``, +to represent the type ``T`` of the inner type in ``Annotated``; metadata would be +type checked against this type variable. However, this would require using the old +type variable syntax (before :pep:`695`), which is now a discouraged feature. +In addition, this would use type variables in an unusual way that does not fit well +with the rest of the type system. + +Introducing a new type to ``typing.py`` that all metadata objects should subclass +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A previous version of this PEP suggested adding a new generic base class, ``TypedMetadata[U]``, +that metadata objects would subclass. If a metadata object is a subclass of ``TypedMetadata[U]``, +then type checkers would check that the annotation's base type is assignable to ``U``. +However, this mechanism does not integrate as well with the rest of the language; Python +does not generally use marker base classes. In addition, it provides less flexibility than +the current proposal: it would not allow overloads, and it would require metadata objects +to add a new base class, which may make their runtime implementation more complex. + +Acknowledgments +=============== + +We thank Eric Traut for suggesting the idea of using a protocol. + +Copyright +========= + +This document has been placed in the public domain.