158 lines
6.3 KiB
ReStructuredText
158 lines
6.3 KiB
ReStructuredText
PEP: 746
|
|
Title: Type checking Annotated metadata
|
|
Author: Adrian Garcia Badaracco <adrian@adriangb.com>
|
|
Sponsor: Jelle Zijlstra <jelle.zijlstra@gmail.com>
|
|
Discussions-To: https://discuss.python.org/t/pep-746-typedmetadata-for-type-checking-of-pep-593-annotated/53834
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Topic: Typing
|
|
Created: 20-May-2024
|
|
Python-Version: 3.14
|
|
Post-History: 20-May-2024
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP proposes a mechanism for type checking metadata that uses
|
|
the :py:data:`typing.Annotated` type. Metadata objects that implement
|
|
the new ``__supports_type__`` protocol will be type checked by static
|
|
type checkers to ensure that the metadata is valid for the given type.
|
|
|
|
Motivation
|
|
==========
|
|
|
|
:pep:`593` introduced ``Annotated`` as a way to attach runtime metadata to types.
|
|
In general, the metadata is not meant for static type checkers, but even so,
|
|
it is often useful to be able to check that the metadata makes sense for the given
|
|
type.
|
|
|
|
Take the first example in :pep:`593`, which uses ``Annotated`` to attach
|
|
serialization information to a field::
|
|
|
|
class Student(struct2.Packed):
|
|
name: Annotated[str, struct2.ctype("<10s")]
|
|
|
|
Here, the ``struct2.ctype("<10s")`` metadata is meant to be used by a serialization
|
|
library to serialize the field. Such libraries can only serialize a subset of types:
|
|
it would not make sense to write, for example, ``Annotated[list[str], struct2.ctype("<10s")]``.
|
|
Yet the type system provides no way to enforce this. The metadata are completely
|
|
ignored by type checkers.
|
|
|
|
This use case comes up in libraries like :pypi:`pydantic` and :pypi:`msgspec`, which use
|
|
``Annotated`` to attach validation and conversion information to fields or :pypi:`fastapi`,
|
|
which uses ``Annotated`` to mark parameters as extracted from headers, query strings or
|
|
dependency injection.
|
|
|
|
Specification
|
|
=============
|
|
This PEP introduces a protocol that can be used by static and runtime type checkers to validate
|
|
the consistency between ``Annotated`` metadata and a given type.
|
|
Objects that implement this protocol have an attribute called ``__supports_type__``
|
|
that specifies whether the metadata is valid for a given type::
|
|
|
|
class Int64:
|
|
__supports_type__: int
|
|
|
|
The attribute may also be marked as a ``ClassVar`` to avoid interaction with dataclasses::
|
|
|
|
from dataclasses import dataclass
|
|
from typing import ClassVar
|
|
|
|
@dataclass
|
|
class Gt:
|
|
value: int
|
|
__supports_type__: ClassVar[int]
|
|
|
|
When a static type checker encounters a type expression of the form ``Annotated[T, M1, M2, ...]``,
|
|
it should enforce that for each metadata element in ``M1, M2, ...``, one of the following is true:
|
|
|
|
* The metadata element evaluates to an object that does not have a ``__supports_type__`` attribute; or
|
|
* The metadata element evaluates to an object ``M`` that has a ``__supports_type__`` attribute;
|
|
and ``T`` is assignable to the type of ``M.__supports_type__``.
|
|
|
|
To support generic ``Gt`` metadata, one might write::
|
|
|
|
from typing import Protocol
|
|
|
|
class SupportsGt[T](Protocol):
|
|
def __gt__(self, __other: T) -> bool:
|
|
...
|
|
|
|
class Gt[T]:
|
|
__supports_type__: ClassVar[SupportsGt[T]]
|
|
|
|
def __init__(self, value: T) -> None:
|
|
self.value = value
|
|
|
|
x1: Annotated[int, Gt(0)] = 1 # OK
|
|
x2: Annotated[str, Gt(0)] = 0 # type checker error: str is not assignable to SupportsGt[int]
|
|
x3: Annotated[int, Gt(1)] = 0 # OK for static type checkers; runtime type checkers may flag this
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
Metadata that does not implement the protocol will be considered valid for all types,
|
|
so no breaking changes are introduced for existing code. The new checks only apply
|
|
to metadata objects that explicitly implement the protocol specified by this PEP.
|
|
|
|
Security Implications
|
|
=====================
|
|
|
|
None.
|
|
|
|
How to Teach This
|
|
=================
|
|
|
|
This protocol is intended mostly for libraries that provide ``Annotated`` metadata;
|
|
end users of those libraries are unlikely to need to implement the protocol themselves.
|
|
The protocol should be mentioned in the documentation for :py:data:`typing.Annotated` and
|
|
in the typing specification.
|
|
|
|
Reference Implementation
|
|
========================
|
|
|
|
None yet.
|
|
|
|
Rejected ideas
|
|
==============
|
|
|
|
Introducing a type variable instead of a generic class
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
We considered using a special type variable, ``AnnotatedT = TypeVar("AnnotatedT")``,
|
|
to represent the type ``T`` of the inner type in ``Annotated``; metadata would be
|
|
type checked against this type variable. However, this would require using the old
|
|
type variable syntax (before :pep:`695`), which is now a discouraged feature.
|
|
In addition, this would use type variables in an unusual way that does not fit well
|
|
with the rest of the type system.
|
|
|
|
Introducing a new type to ``typing.py`` that all metadata objects should subclass
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
A previous version of this PEP suggested adding a new generic base class, ``TypedMetadata[U]``,
|
|
that metadata objects would subclass. If a metadata object is a subclass of ``TypedMetadata[U]``,
|
|
then type checkers would check that the annotation's base type is assignable to ``U``.
|
|
However, this mechanism does not integrate as well with the rest of the language; Python
|
|
does not generally use marker base classes. In addition, it provides less flexibility than
|
|
the current proposal: it would not allow overloads, and it would require metadata objects
|
|
to add a new base class, which may make their runtime implementation more complex.
|
|
|
|
Using a method instead of an attribute for ``__supports_type__``
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
We considered using a method instead of an attribute for the protocol, so that this method can be used
|
|
at runtime to check the validity of the metadata and to support overloads or returning boolean literals.
|
|
However, using a method adds boilerplate to the implementation and the value of the runtime use cases or
|
|
more complex scenarios involving overloads and returning boolean literals was not clear.
|
|
|
|
Acknowledgments
|
|
===============
|
|
|
|
We thank Eric Traut for suggesting the idea of using a protocol and implementing provisional support in Pyright.
|
|
Thank you to Jelle Zijlstra for sponsoring this PEP.
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|