1065 lines
36 KiB
ReStructuredText
1065 lines
36 KiB
ReStructuredText
PEP: 747
|
||
Title: TypeExpr: Type Hint for a Type Expression
|
||
Author: David Foster <david at dafoster.net>
|
||
Sponsor: Jelle Zijlstra <jelle.zijlstra at gmail.com>
|
||
Discussions-To: https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Topic: Typing
|
||
Created: 27-May-2024
|
||
Python-Version: 3.14
|
||
Post-History: `19-Apr-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435>`__, `04-May-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/7/>`__, `17-Jun-2024 <https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984>`__
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
:pep:`484` defines the notation ``type[C]`` where ``C`` is a class, to
|
||
refer to a class object that is a subtype of ``C``. It explicitly does
|
||
not allow ``type[C]`` to refer to arbitrary
|
||
:ref:`type expression <typing:type-expression>` objects such
|
||
as the runtime object ``str | None``, even if ``C`` is an unbounded
|
||
``TypeVar``. [#type_c]_ In cases where that restriction is unwanted, this
|
||
PEP proposes a new notation ``TypeExpr[T]`` where ``T`` is a type, to
|
||
refer to a either a class object or some other type expression object
|
||
that is a subtype of ``T``, allowing any kind of type to be referenced.
|
||
|
||
This PEP makes no Python grammar changes. Correct usage of
|
||
``TypeExpr[]`` is intended to be enforced only by static and runtime
|
||
type checkers and need not be enforced by Python itself at runtime.
|
||
|
||
|
||
.. _motivation:
|
||
|
||
Motivation
|
||
==========
|
||
|
||
The introduction of ``TypeExpr`` allows new kinds of metaprogramming
|
||
functions that operate on type expressions to be type-annotated and
|
||
understood by type checkers.
|
||
|
||
For example, here is a function that checks whether a value is
|
||
assignable to a variable of a particular type, and if so returns the
|
||
original value:
|
||
|
||
::
|
||
|
||
def trycast[T](typx: TypeExpr[T], value: object) -> T | None: ...
|
||
|
||
The use of ``TypeExpr[]`` and the type variable ``T`` enables the return
|
||
type of this function to be influenced by a ``typx`` value passed at
|
||
runtime, which is quite powerful.
|
||
|
||
Here is another function that checks whether a value is assignable to a
|
||
variable of a particular type, and if so returns ``True`` (as a special
|
||
``TypeIs[]`` bool [#TypeIsPep]_):
|
||
|
||
::
|
||
|
||
def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...
|
||
|
||
The use of ``TypeExpr[]`` and ``TypeIs[]`` together enables type
|
||
checkers to narrow the return type appropriately depending on what type
|
||
expression is passed in:
|
||
|
||
::
|
||
|
||
request_json: object = ...
|
||
if isassignable(request_json, MyTypedDict):
|
||
assert_type(request_json, MyTypedDict) # type is narrowed!
|
||
|
||
That ``isassignable`` function enables a kind of enhanced ``isinstance``
|
||
check which is useful for `checking whether a value decoded from JSON
|
||
conforms to a particular structure`_ of nested ``TypedDict``\ s,
|
||
lists, unions, ``Literal``\ s, and other types. This kind
|
||
of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` but could
|
||
not be implemented at the time without a notation similar to
|
||
``TypeExpr[]``.
|
||
|
||
.. _checking whether a value decoded from JSON conforms to a particular structure: https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/
|
||
|
||
|
||
Why can’t ``type[]`` be used?
|
||
-----------------------------
|
||
|
||
One might think you could define the example functions above to take a
|
||
``type[C]`` - which is syntax that already exists - rather than a
|
||
``TypeExpr[T]``. However if you were to do that then certain type
|
||
expressions like ``str | None`` - which are not class objects and
|
||
therefore not ``type``\ s at runtime - would be rejected:
|
||
|
||
::
|
||
|
||
# NOTE: Uses a type[C] parameter rather than a TypeExpr[T]
|
||
def trycast_type[C](typ: type[C], value: object) -> T | None: ...
|
||
|
||
trycast_type(str, 'hi') # ok; str is a type
|
||
trycast_type(Optional[str], 'hi') # ERROR; Optional[str] is not a type
|
||
trycast_type(str | int, 'hi') # ERROR; (str | int) is not a type
|
||
trycast_type(MyTypedDict, dict(value='hi')) # questionable; accepted by mypy 1.9.0
|
||
|
||
To solve that problem, ``type[]`` could be widened to include the
|
||
additional values allowed by ``TypeExpr``. However doing so would lose
|
||
``type[]``\ ’s current ability to spell a class object which always
|
||
supports instantiation and ``isinstance`` checks, unlike arbitrary type
|
||
expression objects. Therefore ``TypeExpr`` is proposed as new notation
|
||
instead.
|
||
|
||
For a longer explanation of why we don’t just widen ``type[T]`` to
|
||
accept all type expressions, see
|
||
:ref:`widen_type_C_to_support_all_type_expressions`.
|
||
|
||
|
||
.. _runtime_type_checkers_using_typeexpr:
|
||
|
||
Common kinds of functions that would benefit from TypeExpr
|
||
----------------------------------------------------------
|
||
|
||
`A survey of various Python libraries`_ revealed a few kinds of commonly
|
||
defined functions which would benefit from ``TypeExpr[]``:
|
||
|
||
.. _A survey of various Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886
|
||
|
||
- Assignability checkers:
|
||
|
||
- Returns whether a value is assignable to a type expression. If so
|
||
then also narrows the type of the value to match the type
|
||
expression.
|
||
- Pattern 1:
|
||
``def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]``
|
||
- Pattern 2:
|
||
``def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]``
|
||
- Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_,
|
||
typeguard.\ `check_type`_, xdsl.\ `isa`_
|
||
|
||
.. _is_bearable: https://github.com/beartype/beartype/issues/255
|
||
.. _isassignable: https://github.com/davidfstr/trycast?tab=readme-ov-file#isassignable-api
|
||
.. _check_type: https://typeguard.readthedocs.io/en/latest/api.html#typeguard.check_type
|
||
.. _isa: https://github.com/xdslproject/xdsl/blob/ac12c9ab0d64618475efb98d1d197bdd79f593c3/xdsl/utils/hints.py#L23
|
||
|
||
- Converters:
|
||
|
||
- If a value is assignable to (or coercible to) a type expression,
|
||
a *converter* returns the value narrowed to (or coerced to) that type
|
||
expression. Otherwise, it raises an exception.
|
||
- Pattern 1:
|
||
``def convert[T](value: object, typx: TypeExpr[T]) -> T``
|
||
|
||
- Examples: cattrs.BaseConverter.\ `structure`_, trycast.\ `checkcast`_,
|
||
typedload.\ `load`_
|
||
|
||
- Pattern 2:
|
||
|
||
::
|
||
|
||
class Converter[T]:
|
||
def __init__(self, typx: TypeExpr[T]) -> None: ...
|
||
def convert(self, value: object) -> T: ...
|
||
|
||
- Examples: pydantic.\ `TypeAdapter(T).validate_python`_,
|
||
mashumaro.\ `JSONDecoder(T).decode`_
|
||
|
||
.. _structure: https://github.com/python-attrs/cattrs/blob/5f5c11627a7f67a23d6212bc7df9f96243c62dc5/src/cattrs/converters.py#L332-L334
|
||
.. _checkcast: https://github.com/davidfstr/trycast#checkcast-api
|
||
.. _load: https://ltworf.github.io/typedload/
|
||
.. _TypeAdapter(T).validate_python: https://stackoverflow.com/a/61021183/604063
|
||
.. _JSONDecoder(T).decode: https://github.com/Fatal1ty/mashumaro?tab=readme-ov-file#usage-example
|
||
|
||
- Typed field definitions:
|
||
|
||
- Pattern:
|
||
|
||
::
|
||
|
||
class Field[T]:
|
||
value_type: TypeExpr[T]
|
||
|
||
- Examples: attrs.\ `make_class`_,
|
||
dataclasses.\ `make_dataclass`_ [#DataclassInitVar]_, `openapify`_
|
||
|
||
.. _make_class: https://www.attrs.org/en/stable/api.html#attrs.make_class
|
||
.. _make_dataclass: https://github.com/python/typeshed/issues/11653
|
||
.. _openapify: https://github.com/Fatal1ty/openapify/blob/c8d968c7c9c8fd7d4888bd2ddbe18ffd1469f3ca/openapify/core/models.py#L16
|
||
|
||
The survey also identified some introspection functions that take
|
||
annotation expressions as input using plain ``object``\ s which would
|
||
*not* gain functionality by marking those inputs as ``TypeExpr[]``:
|
||
|
||
- General introspection operations:
|
||
|
||
- Pattern: ``def get_annotation_info(maybe_annx: object) -> object``
|
||
- Examples: typing.{`get_origin`_, `get_args`_},
|
||
`typing_inspect`_.{is_*_type, get_origin, get_parameters}
|
||
|
||
.. _get_origin: https://docs.python.org/3/library/typing.html#typing.get_origin
|
||
.. _get_args: https://docs.python.org/3/library/typing.html#typing.get_args
|
||
.. _typing_inspect: https://github.com/ilevkivskyi/typing_inspect?tab=readme-ov-file#readme
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
Before this PEP existed there were already a few definitions in use to describe
|
||
different kinds of type annotations:
|
||
|
||
.. code-block:: text
|
||
|
||
+----------------------------------+
|
||
| +------------------------------+ |
|
||
| | +-------------------------+ | |
|
||
| | | +---------------------+ | | |
|
||
| | | | Class object | | | | = type[C]
|
||
| | | +---------------------+ | | |
|
||
| | | Type expression object | | | = TypeExpr[T] <-- new!
|
||
| | +-------------------------+ | |
|
||
| | Annotation expression object | |
|
||
| +------------------------------+ |
|
||
| Object | = object
|
||
+----------------------------------+
|
||
|
||
- :ref:`Class objects <typing:type-brackets>`,
|
||
spelled as ``type[C]``, support ``isinstance`` checks and are callable.
|
||
|
||
- Examples: ``int``, ``str``, ``MyClass``
|
||
|
||
- :ref:`Type expressions <typing:type-expression>`
|
||
include any type annotation which describes a type.
|
||
|
||
- Examples: ``list[int]``, ``MyTypedDict``, ``int | str``,
|
||
``Literal['square']``, any class object
|
||
|
||
- :ref:`Annotation expressions <typing:annotation-expression>`
|
||
include any type annotation, including those only valid in specific contexts.
|
||
|
||
- Examples: ``Final[int]``, ``Required[str]``, ``ClassVar[str]``,
|
||
any type expression
|
||
|
||
``TypeExpr`` aligns with an existing definition from the above list -
|
||
*type expression* - to avoid introducing yet another subset of type annotations
|
||
that users of Python typing need to think about.
|
||
|
||
``TypeExpr`` aligns with *type expression* specifically
|
||
because a type expression is already used to parameterize type variables,
|
||
which are used in combination with ``TypeIs`` and ``TypeGuard`` to enable
|
||
the compelling examples mentioned in :ref:`Motivation <motivation>`.
|
||
|
||
``TypeExpr`` does not align with *annotation expression* for reasons given in
|
||
:ref:`Rejected Ideas » Accept arbitrary annotation expressions <accept_arbitrary_annotation_expressions>`.
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>`
|
||
such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
|
||
A ``TypeExpr`` type is written as
|
||
``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
|
||
written without brackets as just ``TypeExpr``, which is treated the same as
|
||
to ``TypeExpr[Any]``.
|
||
|
||
|
||
Using TypeExprs
|
||
---------------
|
||
|
||
A ``TypeExpr`` is a new kind of type expression, usable in any context where a
|
||
type expression is valid, as a function parameter type, a return type,
|
||
or a variable type:
|
||
|
||
::
|
||
|
||
def is_union_type(typx: TypeExpr) -> bool: ... # parameter type
|
||
|
||
::
|
||
|
||
def union_of[S, T](s: TypeExpr[S], t: TypeExpr[T]) \
|
||
-> TypeExpr[S | T]: ... # return type
|
||
|
||
::
|
||
|
||
STR_TYPE: TypeExpr = str # variable type
|
||
|
||
Note however that an *unannotated* variable assigned a type expression literal
|
||
will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
|
||
484 :pep:`reserves that syntax for defining type aliases <484#type-aliases>`:
|
||
|
||
- No:
|
||
|
||
::
|
||
|
||
STR_TYPE = str # OOPS; treated as a type alias!
|
||
|
||
If you want a type checker to recognize a type expression literal in a bare
|
||
assignment you’ll need to explicitly declare the assignment-target as
|
||
having ``TypeExpr`` type:
|
||
|
||
- Yes:
|
||
|
||
::
|
||
|
||
STR_TYPE: TypeExpr = str
|
||
|
||
- Yes:
|
||
|
||
::
|
||
|
||
STR_TYPE: TypeExpr
|
||
STR_TYPE = str
|
||
|
||
- Okay, but discouraged:
|
||
|
||
::
|
||
|
||
STR_TYPE = str # type: TypeExpr # the type comment is significant
|
||
|
||
``TypeExpr`` values can be passed around and assigned just like normal
|
||
values:
|
||
|
||
::
|
||
|
||
def swap1[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
|
||
t1_new: TypeExpr[T] = t2 # assigns a TypeExpr value to a new annotated variable
|
||
t2_new: TypeExpr[S] = t1
|
||
return (t1_new, t2_new)
|
||
|
||
def swap2[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
|
||
t1_new = t2 # assigns a TypeExpr value to a new unannotated variable
|
||
t2_new = t1
|
||
assert_type(t1_new, TypeExpr[T])
|
||
assert_type(t2_new, TypeExpr[S])
|
||
return (t1_new, t2_new)
|
||
|
||
# NOTE: A more straightforward implementation would use isinstance()
|
||
def ensure_int(value: object) -> None:
|
||
value_type: TypeExpr = type(value) # assigns a type (a subtype of TypeExpr)
|
||
assert value_type == int
|
||
|
||
|
||
TypeExpr Values
|
||
---------------
|
||
|
||
A variable of type ``TypeExpr[T]`` where ``T`` is a type, can hold any
|
||
**type expression object** - the result of evaluating a
|
||
:ref:`type expression <typing:type-expression>`
|
||
at runtime - which is a subtype of ``T``.
|
||
|
||
Incomplete expressions like a bare ``Optional`` or ``Union`` which do
|
||
not spell a type are not ``TypeExpr`` values.
|
||
|
||
``TypeExpr[...]`` is itself a ``TypeExpr`` value:
|
||
|
||
::
|
||
|
||
OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK
|
||
assert isassignable(int | None, OPTIONAL_INT_TYPE)
|
||
|
||
.. _non_universal_typeexpr:
|
||
|
||
``TypeExpr[]`` values include *all* type expressions including some
|
||
**non-universal type expressions** which are not valid in all annotation contexts.
|
||
In particular:
|
||
|
||
- ``Self`` (valid only in some contexts)
|
||
- ``TypeGuard[...]`` (valid only in some contexts)
|
||
- ``TypeIs[...]`` (valid only in some contexts)
|
||
|
||
|
||
Explicit TypeExpr Values
|
||
''''''''''''''''''''''''
|
||
|
||
The syntax ``TypeExpr(T)`` (with parentheses) can be used to
|
||
spell a ``TypeExpr[T]`` value explicitly:
|
||
|
||
::
|
||
|
||
NONE = TypeExpr(None)
|
||
INT1 = TypeExpr('int') # stringified type expression
|
||
INT2 = TypeExpr(int)
|
||
|
||
At runtime the ``TypeExpr(...)`` callable returns its single argument unchanged.
|
||
|
||
|
||
.. _implicit_typeexpr_values:
|
||
|
||
Implicit TypeExpr Values
|
||
''''''''''''''''''''''''
|
||
|
||
Historically static type checkers have only needed to recognize
|
||
*type expressions* in contexts where a type expression was expected.
|
||
Now *type expression objects* must also be recognized in contexts where a
|
||
value expression is expected.
|
||
|
||
Static type checkers already recognize **class objects** (``type[C]``):
|
||
|
||
- As a value expression, ``C`` has type ``type[C]``,
|
||
for each of the following values of C:
|
||
|
||
- ``name`` (where ``name`` must refer to a valid in-scope class, type alias, or TypeVar)
|
||
- ``name '[' ... ']'``
|
||
- ``<type> '[' ... ']'``
|
||
|
||
The following **unparameterized type expressions** can be recognized unambiguously:
|
||
|
||
- As a value expression, ``X`` has type ``TypeExpr[X]``,
|
||
for each of the following values of X:
|
||
|
||
- ``<Any>``
|
||
- ``<Self>``
|
||
- ``<LiteralString>``
|
||
- ``<NoReturn>``
|
||
- ``<Never>``
|
||
|
||
**None**: The type expression ``None`` (``NoneType``) is ambiguous with the value ``None``,
|
||
so must use the explicit ``TypeExpr(...)`` syntax:
|
||
|
||
- As a value expression, ``TypeExpr(None)`` has type ``TypeExpr[None]``.
|
||
- As a value expression, ``None`` continues to have type ``None``.
|
||
|
||
The following **parameterized type expressions** can be recognized unambiguously:
|
||
|
||
- As a value expression, ``X`` has type ``TypeExpr[X]``,
|
||
for each of the following values of X:
|
||
|
||
- ``<Literal> '[' ... ']'``
|
||
- ``<Optional> '[' ... ']'``
|
||
- ``<Union> '[' ... ']'``
|
||
- ``<Callable> '[' ... ']'``
|
||
- ``<tuple> '[' ... ']'``
|
||
- ``<TypeGuard> '[' ... ']'``
|
||
- ``<TypeIs> '[' ... ']'``
|
||
|
||
.. _recognizing_annotated:
|
||
|
||
**Annotated**: The type expression ``Annotated[...]`` is ambiguous with
|
||
the annotation expression ``Annotated[...]``,
|
||
so must be disambiguated based on its argument type:
|
||
|
||
- As a value expression, ``Annotated[x, ...]`` has type ``type[C]``
|
||
if ``x`` has type ``type[C]``.
|
||
- As a value expression, ``Annotated[x, ...]`` has type ``TypeExpr[T]``
|
||
if ``x`` has type ``TypeExpr[T]``.
|
||
- As a value expression, ``Annotated[x, ...]`` has type ``object``
|
||
if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.
|
||
|
||
**Union**: The type expression ``T1 | T2`` is ambiguous with
|
||
the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more,
|
||
so must use the explicit ``TypeExpr(...)`` syntax:
|
||
|
||
- Yes:
|
||
|
||
::
|
||
|
||
if isassignable(value, TypeExpr(int | str)): ...
|
||
|
||
- No:
|
||
|
||
::
|
||
|
||
if isassignable(value, int | str): ...
|
||
|
||
Future PEPs may make it possible to recognize the value expression ``T1 | T2`` directly as an
|
||
implicit TypeExpr value and avoid the need to use the explicit ``TypeExpr(...)`` syntax,
|
||
but that work is :ref:`deferred for now <recognize_uniontype_as_implicit_typeexpr_value>`.
|
||
|
||
The **stringified type expression** ``"T"`` is ambiguous with both
|
||
the stringified annotation expression ``"T"``
|
||
and the string literal ``"T"``,
|
||
so must use the explicit ``TypeExpr(...)`` syntax:
|
||
|
||
- As a value expression, ``TypeExpr("T")`` has type ``TypeExpr[T]``,
|
||
where ``T`` is a valid type expression
|
||
- As a value expression, ``"T"`` continues to have type ``Literal["T"]``.
|
||
|
||
No other kinds of type expressions currently exist.
|
||
|
||
New kinds of type expressions that are introduced should define how they
|
||
will be recognized in a value expression context.
|
||
|
||
|
||
Literal[] TypeExprs
|
||
'''''''''''''''''''
|
||
|
||
A value of ``Literal[...]`` type is *not* considered assignable to
|
||
a ``TypeExpr`` variable even if all of its members spell valid types because
|
||
dynamic values are not allowed in type expressions:
|
||
|
||
::
|
||
|
||
STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
|
||
STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr
|
||
|
||
However ``Literal[...]`` itself is still a ``TypeExpr``:
|
||
|
||
::
|
||
|
||
DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right'] # OK
|
||
|
||
|
||
Static vs. Runtime Representations of TypeExprs
|
||
'''''''''''''''''''''''''''''''''''''''''''''''
|
||
|
||
A ``TypeExpr`` value appearing statically in a source file may be normalized
|
||
to a different representation at runtime. For example string-based
|
||
forward references are normalized at runtime to be ``ForwardRef`` instances
|
||
in some contexts: [#forward_ref_normalization]_
|
||
|
||
::
|
||
|
||
>>> IntTree = list[typing.Union[int, 'IntTree']]
|
||
>>> IntTree
|
||
list[typing.Union[int, ForwardRef('IntTree')]]
|
||
|
||
The runtime representations of ``TypeExpr``\ s are considered implementation
|
||
details that may change over time and therefore static type checkers are
|
||
not required to recognize them:
|
||
|
||
::
|
||
|
||
INT_TREE: TypeExpr = ForwardRef('IntTree') # ERROR: Runtime-only form
|
||
|
||
Runtime type checkers that wish to assign a runtime-only representation
|
||
of a type expression to a ``TypeExpr[]`` variable must use ``cast()`` to
|
||
avoid errors from static type checkers:
|
||
|
||
::
|
||
|
||
INT_TREE = cast(TypeExpr, ForwardRef('IntTree')) # OK
|
||
|
||
|
||
Subtyping
|
||
---------
|
||
|
||
Whether a ``TypeExpr`` value can be assigned from one variable to another is
|
||
determined by the following rules:
|
||
|
||
Relationship with type
|
||
''''''''''''''''''''''
|
||
|
||
``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:
|
||
|
||
- ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
|
||
subtype of ``T2``.
|
||
- ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
|
||
of ``C2``.
|
||
|
||
An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr``
|
||
but not the other way around:
|
||
|
||
- ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
|
||
other way around.)
|
||
|
||
Relationship with object
|
||
''''''''''''''''''''''''
|
||
|
||
``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:
|
||
|
||
- ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.
|
||
|
||
``TypeExpr[T]``, where ``T`` is a type variable, is assumed to have all
|
||
the attributes and methods of ``object`` and is not callable.
|
||
|
||
|
||
Interactions with isinstance() and issubclass()
|
||
-----------------------------------------------
|
||
|
||
The ``TypeExpr`` special form cannot be used as the ``type`` argument to
|
||
``isinstance``:
|
||
|
||
::
|
||
|
||
>>> isinstance(str, TypeExpr)
|
||
TypeError: typing.TypeExpr cannot be used with isinstance()
|
||
|
||
>>> isinstance(str, TypeExpr[str])
|
||
TypeError: isinstance() argument 2 cannot be a parameterized generic
|
||
|
||
The ``TypeExpr`` special form cannot be used as any argument to
|
||
``issubclass``:
|
||
|
||
::
|
||
|
||
>>> issubclass(TypeExpr, object)
|
||
TypeError: issubclass() arg 1 must be a class
|
||
|
||
>>> issubclass(object, TypeExpr)
|
||
TypeError: typing.TypeExpr cannot be used with issubclass()
|
||
|
||
|
||
Affected signatures in the standard library
|
||
-------------------------------------------
|
||
|
||
Changed signatures
|
||
''''''''''''''''''
|
||
|
||
The following signatures related to type expressions introduce
|
||
``TypeExpr`` where previously ``object`` or ``Any`` existed:
|
||
|
||
- ``typing.cast``
|
||
- ``typing.assert_type``
|
||
|
||
Unchanged signatures
|
||
''''''''''''''''''''
|
||
|
||
The following signatures related to annotation expressions continue to
|
||
use ``object`` and remain unchanged:
|
||
|
||
- ``typing.get_origin``
|
||
- ``typing.get_args``
|
||
|
||
The following signatures related to class objects continue to use
|
||
``type`` and remain unchanged:
|
||
|
||
- ``builtins.isinstance``
|
||
- ``builtins.issubclass``
|
||
- ``builtins.type``
|
||
|
||
``typing.get_type_hints(..., include_extras=False)`` nearly returns only type
|
||
expressions in Python 3.12, stripping out most type qualifiers
|
||
(``Required, NotRequired, ReadOnly, Annotated``) but currently preserves a
|
||
few type qualifiers which are only allowed in annotation expressions
|
||
(``ClassVar, Final, InitVar, Unpack``). It may be desirable to alter the
|
||
behavior of this function in the future to also strip out those
|
||
qualifiers and actually return type expressions, although this PEP does
|
||
not propose those changes now:
|
||
|
||
- ``typing.get_type_hints(..., include_extras=False)``
|
||
|
||
- Almost returns only type expressions, but not quite
|
||
|
||
- ``typing.get_type_hints(..., include_extras=True)``
|
||
|
||
- Returns annotation expressions
|
||
|
||
|
||
Backwards Compatibility
|
||
=======================
|
||
|
||
The rules for recognizing type expression objects
|
||
in a value expression context were not previously defined, so static type checkers
|
||
`varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
|
||
to such objects. Existing programs manipulating type expression objects
|
||
were already limited in manipulating them as plain ``object`` values,
|
||
and such programs should not break with
|
||
:ref:`the newly-defined rules <implicit_typeexpr_values>`.
|
||
|
||
|
||
How to Teach This
|
||
=================
|
||
|
||
Normally when using type annotations in Python you're concerned with defining
|
||
the shape of values allowed to be passed to a function parameter, returned
|
||
by a function, or stored in a variable:
|
||
|
||
.. code-block:: text
|
||
|
||
parameter type return type
|
||
| |
|
||
v v
|
||
def plus(n1: int, n2: int) -> int:
|
||
sum: int = n1 + n2
|
||
^
|
||
|
|
||
variable type
|
||
|
||
return sum
|
||
|
||
However type annotations themselves are valid values in Python and can be
|
||
assigned to variables and manipulated like any other data in a program:
|
||
|
||
.. code-block:: text
|
||
|
||
a variable a type
|
||
| |
|
||
v v
|
||
MAYBE_INT_TYPE: TypeExpr = int | None
|
||
^
|
||
|
|
||
the type of a type
|
||
|
||
``TypeExpr[]`` is how you spell the type of a variable containing a
|
||
type annotation object describing a type.
|
||
|
||
``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only
|
||
spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``.
|
||
``TypeExpr[]`` by contrast can additionally spell more complex types,
|
||
including those with brackets (like ``list[int]``) or pipes (like ``int | None``),
|
||
and including special types like ``Any``, ``LiteralString``, or ``Never``.
|
||
|
||
A ``TypeExpr`` variable (``maybe_float: TypeExpr``) looks similar to
|
||
a ``TypeAlias`` definition (``MaybeFloat: TypeAlias``), but ``TypeExpr``
|
||
can only be used where a dynamic value is expected:
|
||
|
||
- No:
|
||
|
||
::
|
||
|
||
maybe_float: TypeExpr = float | None
|
||
def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeExpr value in a type annotation
|
||
|
||
- Okay, but discouraged in Python 3.12+:
|
||
|
||
::
|
||
|
||
MaybeFloat: TypeAlias = float | None
|
||
def sqrt(n: float) -> MaybeFloat: ...
|
||
|
||
- Yes:
|
||
|
||
::
|
||
|
||
type MaybeFloat = float | None
|
||
def sqrt(n: float) -> MaybeFloat: ...
|
||
|
||
It is uncommon for a programmer to define their *own* function which accepts
|
||
a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common
|
||
for a programmer to pass a literal type expression to an *existing* function
|
||
accepting a ``TypeExpr`` input which was imported from a runtime type checker
|
||
library.
|
||
|
||
For example the ``isassignable`` function from the ``trycast`` library
|
||
can be used like Python's built-in ``isinstance`` function to check whether
|
||
a value matches the shape of a particular type.
|
||
``isassignable`` will accept *any* kind of type as an input because its input
|
||
is a ``TypeExpr``. By contrast ``isinstance`` only accepts a simple class object
|
||
(a ``type[]``) as input:
|
||
|
||
- Yes:
|
||
|
||
::
|
||
|
||
from trycast import isassignable
|
||
|
||
if isassignable(some_object, MyTypedDict): # OK: MyTypedDict is a TypeExpr[]
|
||
...
|
||
|
||
- No:
|
||
|
||
::
|
||
|
||
if isinstance(some_object, MyTypedDict): # ERROR: MyTypedDict is not a type[]
|
||
...
|
||
|
||
There are :ref:`many other runtime type checkers <runtime_type_checkers_using_typeexpr>`
|
||
providing useful functions that accept a ``TypeExpr``.
|
||
|
||
|
||
.. _advanced_examples:
|
||
|
||
Advanced Examples
|
||
=================
|
||
|
||
If you want to write your own runtime type checker or some other
|
||
kind of function that manipulates types as values at runtime,
|
||
this section gives examples of how you might implement such a function
|
||
using ``TypeExpr``.
|
||
|
||
|
||
Introspecting TypeExpr Values
|
||
-----------------------------
|
||
|
||
A ``TypeExpr`` is very similar to an ``object`` at runtime, with no additional
|
||
attributes or methods defined.
|
||
|
||
You can use existing introspection functions like ``typing.get_origin`` and
|
||
``typing.get_args`` to extract the components of a type expression that looks
|
||
like ``Origin[Arg1, Arg2, ..., ArgN]``:
|
||
|
||
::
|
||
|
||
import typing
|
||
|
||
def strip_annotated_metadata(typx: TypeExpr[T]) -> TypeExpr[T]:
|
||
if typing.get_origin(typx) is typing.Annotated:
|
||
typx = cast(TypeExpr[T], typing.get_args(typx)[0])
|
||
return typx
|
||
|
||
You can also use ``isinstance`` and ``is`` to distinguish one kind of
|
||
type expression from another:
|
||
|
||
::
|
||
|
||
import types
|
||
import typing
|
||
|
||
def split_union(typx: TypeExpr) -> tuple[TypeExpr, ...]:
|
||
if isinstance(typx, types.UnionType): # X | Y
|
||
return cast(tuple[TypeExpr, ...], typing.get_args(typx))
|
||
if typing.get_origin(typx) is typing.Union: # Union[X, Y]
|
||
return cast(tuple[TypeExpr, ...], typing.get_args(typx))
|
||
if typx in (typing.Never, typing.NoReturn,):
|
||
return ()
|
||
return (typx,)
|
||
|
||
|
||
Combining with a type variable
|
||
------------------------------
|
||
|
||
``TypeExpr[]`` can be parameterized by a type variable that is used elsewhere within
|
||
the same function definition:
|
||
|
||
::
|
||
|
||
def as_instance[T](typx: TypeExpr[T]) -> T | None:
|
||
return typx() if isinstance(typx, type) else None
|
||
|
||
|
||
Combining with type[]
|
||
---------------------
|
||
|
||
Both ``TypeExpr[]`` and ``type[]`` can be parameterized by the same type
|
||
variable within the same function definition:
|
||
|
||
::
|
||
|
||
def as_type[T](typx: TypeExpr[T]) -> type[T] | None:
|
||
return typx if isinstance(typx, type) else None
|
||
|
||
|
||
Combining with TypeIs[] and TypeGuard[]
|
||
---------------------------------------
|
||
|
||
A type variable parameterizing a ``TypeExpr[]`` can also be used by a ``TypeIs[]``
|
||
within the same function definition:
|
||
|
||
::
|
||
|
||
def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...
|
||
|
||
count: int | str = ...
|
||
if isassignable(count, int):
|
||
assert_type(count, int)
|
||
else:
|
||
assert_type(count, str)
|
||
|
||
or by a ``TypeGuard[]`` within the same function definition:
|
||
|
||
::
|
||
|
||
def isdefault[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]:
|
||
return (value == typx()) if isinstance(typx, type) else False
|
||
|
||
value: int | str = ''
|
||
if isdefault(value, int):
|
||
assert_type(value, int)
|
||
assert 0 == value
|
||
elif isdefault(value, str):
|
||
assert_type(value, str)
|
||
assert '' == value
|
||
else:
|
||
assert_type(value, int | str)
|
||
|
||
|
||
Challenges When Accepting All TypeExprs
|
||
---------------------------------------
|
||
|
||
A function that takes an *arbitrary* ``TypeExpr`` as
|
||
input must support a large variety of possible type expressions and is
|
||
not easy to write. Some challenges faced by such a function include:
|
||
|
||
- An ever-increasing number of typing special forms are introduced with
|
||
each new Python version which must be recognized, with special
|
||
handling required for each one.
|
||
- Stringified type annotations [#strann_less_common]_ (like ``'list[str]'``)
|
||
must be *parsed* (to something like ``typing.List[str]``) to be introspected.
|
||
|
||
- In practice it is extremely difficult for stringified type
|
||
annotations to be handled reliably at runtime, so runtime type
|
||
checkers may opt to not support them at all.
|
||
|
||
- Resolving string-based forward references inside type
|
||
expressions to actual values must typically be done using ``eval()``,
|
||
which is difficult/impossible to use in a safe way.
|
||
- Recursive types like ``IntTree = list[typing.Union[int, 'IntTree']]``
|
||
are not possible to fully resolve.
|
||
- Supporting user-defined generic types (like Django’s
|
||
``QuerySet[User]``) requires user-defined functions to
|
||
recognize/parse, which a runtime type checker should provide a
|
||
registration API for.
|
||
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
The following will be true when
|
||
`mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:
|
||
|
||
The mypy type checker supports ``TypeExpr`` types.
|
||
|
||
A reference implementation of the runtime component is provided in the
|
||
``typing_extensions`` module.
|
||
|
||
|
||
Rejected Ideas
|
||
==============
|
||
|
||
.. _widen_type_C_to_support_all_type_expressions:
|
||
|
||
Widen type[C] to support all type expressions
|
||
---------------------------------------------
|
||
|
||
``type`` was `designed`_ to only be used to describe class objects. A
|
||
class object can always be used as the second argument of ``isinstance()``
|
||
and can usually be instantiated by calling it.
|
||
|
||
``TypeExpr`` on the other hand is typically introspected by the user in
|
||
some way, is not necessarily directly instantiable, and is not
|
||
necessarily directly usable in a regular ``isinstance()`` check.
|
||
|
||
It would be possible to widen ``type`` to include the additional values
|
||
allowed by ``TypeExpr`` but it would reduce clarity about the user’s
|
||
intentions when working with a ``type``. Different concepts and usage
|
||
patterns; different spellings.
|
||
|
||
.. _designed: https://mail.python.org/archives/list/typing-sig@python.org/message/D5FHORQVPHX3BHUDGF3A3TBZURBXLPHD/
|
||
|
||
|
||
.. _accept_arbitrary_annotation_expressions:
|
||
|
||
Accept arbitrary annotation expressions
|
||
---------------------------------------
|
||
|
||
Certain typing special forms can be used in *some* but not *all*
|
||
annotation contexts:
|
||
|
||
For example ``Final[]`` can be used as a variable type but not as a
|
||
parameter type or a return type:
|
||
|
||
::
|
||
|
||
some_const: Final[str] = ... # OK
|
||
|
||
def foo(not_reassignable: Final[object]): ... # ERROR: Final[] not allowed here
|
||
|
||
def nonsense() -> Final[object]: ... # ERROR: Final[] not meaningful here
|
||
|
||
``TypeExpr[T]`` does not allow matching such annotation expressions
|
||
because it is not clear what it would mean for such an expression
|
||
to parameterized by a type variable in position ``T``:
|
||
|
||
::
|
||
|
||
def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: ...
|
||
|
||
def foo(some_arg):
|
||
if ismatch(some_arg, Final[int]): # ERROR: Final[int] is not a TypeExpr
|
||
reveal_type(some_arg) # ? NOT Final[int], because invalid for a parameter
|
||
|
||
Functions that wish to operate on *all* kinds of annotation expressions,
|
||
including those that are not ``TypeExpr``\ s, can continue to accept such
|
||
inputs as ``object`` parameters, as they must do so today.
|
||
|
||
|
||
Accept only universal type expressions
|
||
--------------------------------------
|
||
|
||
Earlier drafts of this PEP only allowed ``TypeExpr[]`` to match the subset
|
||
of type expressions which are valid in *all* contexts, excluding
|
||
:ref:`non-universal type expressions <non_universal_typeexpr>`.
|
||
However doing that would effectively
|
||
create a new subset of annotation expressions that Python typing users
|
||
would have to understand, on top of all the existing distinctions between
|
||
“class objects”, “type expressions”, and “annotation expressions”.
|
||
|
||
To avoid introducing yet another concept that everyone has to learn,
|
||
this proposal just rounds ``TypeExpr[]`` to exactly match the existing
|
||
definition of a “type expression”.
|
||
|
||
|
||
Support pattern matching on type expressions
|
||
--------------------------------------------
|
||
|
||
It was asserted that some functions may wish to pattern match on the
|
||
interior of type expressions in their signatures.
|
||
|
||
One use case is to allow a function to explicitly enumerate all the
|
||
*specific* kinds of type expressions it supports as input.
|
||
Consider the following possible pattern matching syntax:
|
||
|
||
::
|
||
|
||
@overload
|
||
def checkcast(typx: TypeExpr[AT=Annotated[T, *Anns]], value: str) -> T: ...
|
||
@overload
|
||
def checkcast(typx: TypeExpr[UT=Union[*Ts]], value: str) -> Union[*Ts]: ...
|
||
@overload
|
||
def checkcast(typx: type[C], value: str) -> C: ...
|
||
# ... (more)
|
||
|
||
All functions observed in the wild that conceptually take a ``TypeExpr[]``
|
||
generally try to support *all* kinds of type expressions, so it doesn’t
|
||
seem valuable to enumerate a particular subset.
|
||
|
||
Additionally the above syntax isn’t precise enough to fully describe the
|
||
actual input constraints for a typical function in the wild. For example
|
||
many functions recognize un-stringified type expressions like
|
||
``list[Movie]`` but may not recognize type expressions with stringified
|
||
subcomponents like ``list['Movie']``.
|
||
|
||
A second use case for pattern matching on the interior of type
|
||
expressions is to explicitly match an ``Annotated[]`` form to pull out the
|
||
interior type argument and strip away the metadata:
|
||
|
||
::
|
||
|
||
def checkcast(
|
||
typx: TypeExpr[T] | TypeExpr[AT=Annotated[T, *Anns]],
|
||
value: object
|
||
) -> T:
|
||
|
||
However ``Annotated[T, metadata]`` is already treated equivalent to ``T`` anyway.
|
||
There’s no additional value in being explicit about this behavior.
|
||
The example above could be more-straightforwardly written as the equivalent:
|
||
|
||
::
|
||
|
||
def checkcast(typx: TypeExpr[T], value: object) -> T:
|
||
|
||
|
||
.. _recognize_uniontype_as_implicit_typeexpr_value:
|
||
|
||
Recognize (T1 | T2) as an implicit TypeExpr value
|
||
-------------------------------------------------
|
||
|
||
It would be nice if a value expression like ``int | str`` could be recognized
|
||
as an implicit ``TypeExpr`` value and be used directly in a context where a
|
||
``TypeExpr`` was expected. However making that possible would require making
|
||
changes to the rules that type checkers use for the ``|`` operator. These rules
|
||
are currently underspecified and would need to be make explicit first,
|
||
before making changes to them. The PEP author is not sufficiently motivated to
|
||
take on that specification work at the time of writing.
|
||
|
||
|
||
Footnotes
|
||
=========
|
||
|
||
.. [#type_c]
|
||
:pep:`Type[C] spells a class object <484#the-type-of-class-objects>`
|
||
|
||
.. [#TypeIsPep]
|
||
:pep:`TypeIs[T] is similar to bool <742>`
|
||
|
||
.. [#DataclassInitVar]
|
||
``dataclass.make_dataclass`` accepts ``InitVar[...]`` as a special case
|
||
in addition to type expressions. Therefore it may unfortunately be necessary
|
||
to continue annotating its ``type`` parameter as ``object`` rather
|
||
than ``TypeExpr``.
|
||
|
||
.. [#forward_ref_normalization]
|
||
Special forms normalize string arguments to ``ForwardRef`` instances
|
||
at runtime using internal helper functions in the ``typing`` module.
|
||
Runtime type checkers may wish to implement similar functions when
|
||
working with string-based forward references.
|
||
|
||
.. [#strann_less_common]
|
||
Stringified type annotations are expected to become less common
|
||
starting in Python 3.14 when :pep:`deferred annotations <649>`
|
||
become available. However there is a large amount of existing code from
|
||
earlier Python versions relying on stringified type annotations that will
|
||
still need to be supported for several years.
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document is placed in the public domain or under the
|
||
CC0-1.0-Universal license, whichever is more permissive.
|