162 lines
6.7 KiB
ReStructuredText
162 lines
6.7 KiB
ReStructuredText
PEP: 746
|
|
Title: Type checking Annotated metadata
|
|
Author: Adrian Garcia Badaracco <adrian@adriangb.com>
|
|
Sponsor: Jelle Zijlstra <jelle.zijlstra@gmail.com>
|
|
Discussions-To: https://discuss.python.org/t/pep-746-typedmetadata-for-type-checking-of-pep-593-annotated/53834
|
|
Status: Draft
|
|
Type: Standards Track
|
|
Topic: Typing
|
|
Created: 20-May-2024
|
|
Python-Version: 3.14
|
|
Post-History: 20-May-2024
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP proposes a mechanism for type checking metadata that uses
|
|
the :py:data:`typing.Annotated` type. Metadata objects that implement
|
|
the new ``__supports_type__`` protocol will be type checked by static
|
|
type checkers to ensure that the metadata is valid for the given type.
|
|
|
|
Motivation
|
|
==========
|
|
|
|
:pep:`593` introduced ``Annotated`` as a way to attach runtime metadata to types.
|
|
In general, the metadata is not meant for static type checkers, but even so,
|
|
it is often useful to be able to check that the metadata makes sense for the given
|
|
type.
|
|
|
|
Take the first example in :pep:`593`, which uses ``Annotated`` to attach
|
|
serialization information to a field::
|
|
|
|
class Student(struct2.Packed):
|
|
name: Annotated[str, struct2.ctype("<10s")]
|
|
|
|
Here, the ``struct2.ctype("<10s")`` metadata is meant to be used by a serialization
|
|
library to serialize the field. Such libraries can only serialize a subset of types:
|
|
it would not make sense to write, for example, ``Annotated[list[str], struct2.ctype("<10s")]``.
|
|
Yet the type system provides no way to enforce this. The metadata are completely
|
|
ignored by type checkers.
|
|
|
|
This use case comes up in libraries like :pypi:`pydantic`, which use
|
|
``Annotated`` to attach validation and conversion information to fields.
|
|
|
|
Specification
|
|
=============
|
|
This PEP introduces a protocol that can be used by static and runtime type checkers to validate
|
|
the consistency between ``Annotated`` metadata and a given type.
|
|
Objects that implement this protocol have a method named ``__supports_type__``
|
|
that takes a single positional argument and returns ``bool``::
|
|
|
|
class Int64:
|
|
def __supports_type__(self, obj: int) -> bool:
|
|
return isinstance(obj, int)
|
|
|
|
The protocol being introduced would be defined as follows if it were to be defined in code form::
|
|
|
|
from typing import Protocol
|
|
|
|
class SupportsType[T](Protocol):
|
|
def __supports_type__(self, obj: T, /) -> bool:
|
|
...
|
|
|
|
When a static type checker encounters a type expression of the form ``Annotated[T, M1, M2, ...]``,
|
|
it should enforce that for each metadata element in ``M1, M2, ...``, one of the following is true:
|
|
|
|
* The metadata element evaluates to an object that does not have a ``__supports_type__`` attribute; or
|
|
* The metadata element evaluates to an object ``M`` that implements the ``SupportsType`` protocol;
|
|
and, with ``T`` instantiated to a value ``v``, a call to ``M.__supports_type__(v)`` type checks without errors;
|
|
and that call does not evaluate to ``Literal[False]``.
|
|
|
|
The body of the ``__supports_type__`` method is not used to check the validity of the metadata
|
|
and static type checkers can ignore it. However, tools that use the annotation at
|
|
runtime may call the method to check that a particular value is valid.
|
|
|
|
For example, to support a generic ``Gt`` metadata, one might write::
|
|
|
|
from typing import Protocol
|
|
|
|
class SupportsGt[T](Protocol):
|
|
def __gt__(self, __other: T) -> bool:
|
|
...
|
|
|
|
class Gt[T]:
|
|
def __init__(self, value: T) -> None:
|
|
self.value = value
|
|
|
|
def __supports_type__(self, obj: SupportsGt[T], /) -> bool:
|
|
return obj > self.value
|
|
|
|
x1: Annotated[int, Gt(0)] = 1 # OK
|
|
x2: Annotated[str, Gt(0)] = 0 # type checker error: str is not assignable to SupportsGt[int]
|
|
x3: Annotated[int, Gt(1)] = 0 # OK for static type checkers; runtime type checkers may flag this
|
|
|
|
Implementations may be generic and may use overloads that return ``Literal[True]`` or ``Literal[False]``
|
|
to indicate if the metadata is valid for the given type.
|
|
|
|
Implementations may raise a NotImplementedError if they cannot determine if the metadata is valid for the given type.
|
|
Tools calling ``__supports_type__`` at runtime should catch this exception and treat it as if ``__supports_type__``
|
|
was not present; they should not take this as an indication that the metadata is invalid for the type.
|
|
|
|
Tools that use the metadata at runtime may choose to ignore the implementation of ``__supports_type__``; this PEP does not
|
|
specify how the method should be used at runtime, only that it may be available for use.
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
|
|
Metadata that does not implement the protocol will be considered valid for all types,
|
|
so no breaking changes are introduced for existing code. The new checks only apply
|
|
to metadata objects that explicitly implement the protocol specified by this PEP.
|
|
|
|
Security Implications
|
|
=====================
|
|
|
|
None.
|
|
|
|
How to Teach This
|
|
=================
|
|
|
|
This protocol is intended mostly for libraries that provide ``Annotated`` metadata;
|
|
end users of those libraries are unlikely to need to implement the protocol themselves.
|
|
The protocol should be mentioned in the documentation for :py:data:`typing.Annotated` and
|
|
in the typing specification.
|
|
|
|
Reference Implementation
|
|
========================
|
|
|
|
None yet.
|
|
|
|
Rejected ideas
|
|
==============
|
|
|
|
Introducing a type variable instead of a generic class
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
We considered using a special type variable, ``AnnotatedT = TypeVar("AnnotatedT")``,
|
|
to represent the type ``T`` of the inner type in ``Annotated``; metadata would be
|
|
type checked against this type variable. However, this would require using the old
|
|
type variable syntax (before :pep:`695`), which is now a discouraged feature.
|
|
In addition, this would use type variables in an unusual way that does not fit well
|
|
with the rest of the type system.
|
|
|
|
Introducing a new type to ``typing.py`` that all metadata objects should subclass
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
A previous version of this PEP suggested adding a new generic base class, ``TypedMetadata[U]``,
|
|
that metadata objects would subclass. If a metadata object is a subclass of ``TypedMetadata[U]``,
|
|
then type checkers would check that the annotation's base type is assignable to ``U``.
|
|
However, this mechanism does not integrate as well with the rest of the language; Python
|
|
does not generally use marker base classes. In addition, it provides less flexibility than
|
|
the current proposal: it would not allow overloads, and it would require metadata objects
|
|
to add a new base class, which may make their runtime implementation more complex.
|
|
|
|
Acknowledgments
|
|
===============
|
|
|
|
We thank Eric Traut for suggesting the idea of using a protocol.
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|