iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 747: TypeExpr: Type Hint for a Type Expression (#3798) 

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This commit is contained in:
David Foster 2024-06-16 15:42:44 -07:00 committed by GitHub
parent c6fc1210bf
commit 3e200c644b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 980 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/CODEOWNERS vendored
View File

@ -627,6 +627,8 @@ peps/pep-0745.rst @hugovk
peps/pep-0746.rst @JelleZijlstra
peps/pep-0749.rst @JelleZijlstra
# ...
peps/pep-0747.rst @JelleZijlstra
# ...
# peps/pep-0754.rst
# ...
peps/pep-0789.rst @njsmith

2
peps/pep-0655.rst
View File

@ -461,7 +461,7 @@ Such operators could be implemented on ``type`` via the ``__pos__``,
grammar.
It was decided that it would be prudent to introduce long-form notation
(i.e. ``Required[]`` and ``NotRequired[]``) before introducing
(i.e. ``Required[]`` and ``NotRequired[]``) before introducing
any short-form notation. Future PEPs may reconsider introducing this
or other short-form notation options.

977
peps/pep-0747.rst Normal file
View File

@ -0,0 +1,977 @@
PEP: 747
Title: TypeExpr: Type Hint for a Type Expression
Author: David Foster <david at dafoster.net>
Sponsor: Jelle Zijlstra <jelle.zijlstra at gmail.com>
Status: Draft
Type: Standards Track
Topic: Typing
Created: 27-May-2024
Python-Version: 3.14
Post-History: `19-Apr-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435>`__, `04-May-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/7/>`__
Abstract
========
:pep:`484` defines the notation ``type[C]`` where ``C`` is a class, to
refer to a class object that is a subtype of ``C``. It explicitly does
not allow ``type[C]`` to refer to arbitrary
:ref:`type expression <typing:type-expression>` objects such
as the runtime object ``str | None``, even if ``C`` is an unbounded
``TypeVar``. [#type_c]_ In cases where that restriction is unwanted, this
PEP proposes a new notation ``TypeExpr[T]`` where ``T`` is a type, to
refer to a either a class object or some other type expression object
that is a subtype of ``T``, allowing any kind of type to be referenced.
This PEP makes no Python grammar changes. Correct usage of
``TypeExpr[]`` is intended to be enforced only by static and runtime
type checkers and need not be enforced by Python itself at runtime.
.. _motivation:
Motivation
==========
The introduction of ``TypeExpr`` allows new kinds of metaprogramming
functions that operate on type expressions to be type-annotated and
understood by type checkers.
For example, here is a function that checks whether a value is
assignable to a variable of a particular type, and if so returns the
original value:
::
def trycast[T](typx: TypeExpr[T], value: object) -> T | None: ...
The use of ``TypeExpr[]`` and the type variable ``T`` enables the return
type of this function to be influenced by a ``typx`` value passed at
runtime, which is quite powerful.
Here is another function that checks whether a value is assignable to a
variable of a particular type, and if so returns ``True`` (as a special
``TypeIs[]`` bool [#TypeIsPep]_):
::
def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...
The use of ``TypeExpr[]`` and ``TypeIs[]`` together enables type
checkers to narrow the return type appropriately depending on what type
expression is passed in:
::
request_json: object = ...
if isassignable(request_json, MyTypedDict):
assert_type(request_json, MyTypedDict) # type is narrowed!
That ``isassignable`` function enables a kind of enhanced ``isinstance``
check which is useful for `checking whether a value decoded from JSON
conforms to a particular structure`_ of nested ``TypedDict``\ s,
lists, unions, ``Literal``\ s, and other types. This kind
of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` but could
not be implemented at the time without a notation similar to
``TypeExpr[]``.
.. _checking whether a value decoded from JSON conforms to a particular structure: https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/
Why can’t ``type[]`` be used?
-----------------------------
One might think you could define the example functions above to take a
``type[C]`` - which is syntax that already exists - rather than a
``TypeExpr[T]``. However if you were to do that then certain type
expressions like ``str | None`` - which are not class objects and
therefore not ``type``\ s at runtime - would be rejected:
::
# NOTE: Uses a type[C] parameter rather than a TypeExpr[T]
def trycast_type[C](typ: type[C], value: object) -> T | None: ...
trycast_type(str, 'hi') # ok; str is a type
trycast_type(Optional[str], 'hi') # ERROR; Optional[str] is not a type
trycast_type(str | int, 'hi') # ERROR; (str | int) is not a type
trycast_type(MyTypedDict, dict(value='hi')) # questionable; accepted by mypy 1.9.0
To solve that problem, ``type[]`` could be widened to include the
additional values allowed by ``TypeExpr``. However doing so would lose
``type[]``\ ’s current ability to spell a class object which always
supports instantiation and ``isinstance`` checks, unlike arbitrary type
expression objects. Therefore ``TypeExpr`` is proposed as new notation
instead.
For a longer explanation of why we don’t just widen ``type[T]`` to
accept all type expressions, see
:ref:`widen_type_C_to_support_all_type_expressions`.
Common kinds of functions that would benefit from TypeExpr
----------------------------------------------------------
`A survey of various Python libraries`_ revealed a few kinds of commonly
defined functions which would benefit from ``TypeExpr[]``:
.. _A survey of various Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886
- Assignability checkers:
- Returns whether a value is assignable to a type expression. If so
then also narrows the type of the value to match the type
expression.
- Pattern 1:
``def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]``
- Pattern 2:
``def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]``
- Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_,
typeguard.\ `check_type`_, xdsl.\ `isa`_
.. _is_bearable: https://github.com/beartype/beartype/issues/255
.. _isassignable: https://github.com/davidfstr/trycast?tab=readme-ov-file#isassignable-api
.. _check_type: https://typeguard.readthedocs.io/en/latest/api.html#typeguard.check_type
.. _isa: https://github.com/xdslproject/xdsl/blob/ac12c9ab0d64618475efb98d1d197bdd79f593c3/xdsl/utils/hints.py#L23
- Converters:
- If a value is assignable to (or coercible to) a type expression,
a *converter* returns the value narrowed to (or coerced to) that type
expression. Otherwise, it raises an exception.
- Pattern 1:
``def convert[T](value: object, typx: TypeExpr[T]) -> T``
- Examples: cattrs.BaseConverter.\ `structure`_, trycast.\ `checkcast`_,
typedload.\ `load`_
- Pattern 2:
::
class Converter[T]:
def __init__(self, typx: TypeExpr[T]) -> None: ...
def convert(self, value: object) -> T: ...
- Examples: pydantic.\ `TypeAdapter(T).validate_python`_,
mashumaro.\ `JSONDecoder(T).decode`_
.. _structure: https://github.com/python-attrs/cattrs/blob/5f5c11627a7f67a23d6212bc7df9f96243c62dc5/src/cattrs/converters.py#L332-L334
.. _checkcast: https://github.com/davidfstr/trycast#checkcast-api
.. _load: https://ltworf.github.io/typedload/
.. _TypeAdapter(T).validate_python: https://stackoverflow.com/a/61021183/604063
.. _JSONDecoder(T).decode: https://github.com/Fatal1ty/mashumaro?tab=readme-ov-file#usage-example
- Typed field definitions:
- Pattern:
::
class Field[T]:
value_type: TypeExpr[T]
- Examples: attrs.\ `make_class`_,
dataclasses.\ `make_dataclass`_ [#DataclassInitVar]_, `openapify`_
.. _make_class: https://www.attrs.org/en/stable/api.html#attrs.make_class
.. _make_dataclass: https://github.com/python/typeshed/issues/11653
.. _openapify: https://github.com/Fatal1ty/openapify/blob/c8d968c7c9c8fd7d4888bd2ddbe18ffd1469f3ca/openapify/core/models.py#L16
The survey also identified some introspection functions that take
annotation expressions as input using plain ``object``\ s which would
*not* gain functionality by marking those inputs as ``TypeExpr[]``:
- General introspection operations:
- Pattern: ``def get_annotation_info(maybe_annx: object) -> object``
- Examples: typing.{`get_origin`_, `get_args`_},
`typing_inspect`_.{is_*_type, get_origin, get_parameters}
.. _get_origin: https://docs.python.org/3/library/typing.html#typing.get_origin
.. _get_args: https://docs.python.org/3/library/typing.html#typing.get_args
.. _typing_inspect: https://github.com/ilevkivskyi/typing_inspect?tab=readme-ov-file#readme
Rationale
=========
Before this PEP existed there were already a few definitions in use to describe
different kinds of type annotations:
::
+----------------------------------+
| +------------------------------+ |
| | +-------------------------+ | |
| | | +---------------------+ | | |
| | | | Class object | | | | = type[C]
| | | +---------------------+ | | |
| | | Type expression object | | | = TypeExpr[T] <-- new!
| | +-------------------------+ | |
| | Annotation expression object | |
| +------------------------------+ |
| Object | = object
+----------------------------------+
- :ref:`Class objects <typing:type-brackets>`,
spelled as ``type[C]``, support ``isinstance`` checks and are callable.
- Examples: ``int``, ``str``, ``MyClass``
- :ref:`Type expressions <typing:type-expression>`
include any type annotation which describes a type.
- Examples: ``list[int]``, ``MyTypedDict``, ``int | str``,
``Literal['square']``, any class object
- :ref:`Annotation expressions <typing:annotation-expression>`
include any type annotation, including those only valid in specific contexts.
- Examples: ``Final[int]``, ``Required[str]``, ``ClassVar[str]``,
any type expression
``TypeExpr`` aligns with an existing definition from the above list -
*type expression* - to avoid introducing yet another subset of type annotations
that users of Python typing need to think about.
``TypeExpr`` aligns with *type expression* specifically
because a type expression is already used to parameterize type variables,
which are used in combination with ``TypeIs`` and ``TypeGuard`` to enable
the compelling examples mentioned in :ref:`Motivation <motivation>`.
Specification
=============
A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>`
such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
A ``TypeExpr`` type is written as
``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
written without brackets as just ``TypeExpr``, in which case a type
checker should apply its usual type inference mechanisms to determine
the type of its argument, possibly ``Any``.
Using TypeExprs
---------------
A ``TypeExpr`` is a new kind of type expression, usable in any context where a
type expression is valid, as a function parameter type, a return type,
or a variable type:
::
def is_union_type(typx: TypeExpr) -> bool: ... # parameter type
::
def union_of[S, T](s: TypeExpr[S], t: TypeExpr[T]) \
-> TypeExpr[S | T]: ... # return type
::
STR_TYPE: TypeExpr = str # variable type
assert_type(STR_TYPE, TypeExpr[str])
Note however that an *unannotated* variable assigned a type expression literal
will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
484 :pep:`reserves that syntax for defining type aliases <484#type-aliases>`:
- No:
::
STR_TYPE = str # OOPS; treated as a type alias!
If you want a type checker to recognize a type expression literal in a bare
assignment you’ll need to explicitly declare the assignment-target as
having ``TypeExpr`` type:
- Yes:
::
STR_TYPE: TypeExpr = str
- Yes:
::
STR_TYPE: TypeExpr
STR_TYPE = str
- Okay, but discouraged:
::
STR_TYPE = str # type: TypeExpr # the type comment is significant
``TypeExpr`` values can be passed around and assigned just like normal
values:
::
def swap1[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
t1_new: TypeExpr[T] = t2 # assigns a TypeExpr value to a new annotated variable
t2_new: TypeExpr[S] = t1
return (t1_new, t2_new)
def swap2[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
t1_new = t2 # assigns a TypeExpr value to a new unannotated variable
t2_new = t1
assert_type(t1_new, TypeExpr[T])
assert_type(t2_new, TypeExpr[S])
return (t1_new, t2_new)
# NOTE: A more straightforward implementation would use isinstance()
def ensure_int(value: object) -> None:
value_type: TypeExpr = type(value) # assigns a type (a subtype of TypeExpr)
assert value_type == int
TypeExpr Values
---------------
A variable of type ``TypeExpr[T]`` where ``T`` is a type, can hold any
**type expression object** - the result of evaluating a
:ref:`type expression <typing:type-expression>`
at runtime - which is a subtype of ``T``.
Incomplete expressions like a bare ``Optional`` or ``Union`` which do
not spell a type are not ``TypeExpr`` values.
``TypeExpr[...]`` is itself a ``TypeExpr`` value:
::
OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK
assert isassignable(Optional[int], OPTIONAL_INT_TYPE)
.. _non_universal_typeexpr:
``TypeExpr[]`` values include *all* type expressions including some
**non-universal type expressions** which are not valid in all annotation contexts.
In particular:
- ``Self`` (valid only in some contexts)
- ``TypeGuard[...]`` (valid only in some contexts)
- ``TypeIs[...]`` (valid only in some contexts)
Explicit TypeExpr Values
''''''''''''''''''''''''
The syntax ``TypeExpr(T)`` (with parentheses) can be used to
spell a ``TypeExpr[T]`` value explicitly:
::
NONE = TypeExpr(None)
INT1 = TypeExpr('int') # stringified type expression
INT2 = TypeExpr(int)
At runtime the ``TypeExpr(...)`` callable returns its single argument unchanged.
.. _implicit_typeexpr_values:
Implicit TypeExpr Values
''''''''''''''''''''''''
Historically static type checkers have only needed to recognize
*type expressions* in contexts where a type expression was expected.
Now *type expression objects* must also be recognized in contexts where a
value expression is expected.
Static type checkers already recognize **class objects** (``type[C]``):
- As a value expression, ``C`` has type ``type[C]``,
for each of the following values of C:
- ``name`` (where ``name`` must refer to a valid in-scope class, type alias, or TypeVar)
- ``name '[' ... ']'``
- ``<type> '[' ... ']'``
The following **unparameterized type expressions** can be recognized unambiguously:
- As a value expression, ``X`` has type ``TypeForm[X]``,
for each of the following values of X:
- ``<Any>``
- ``<Self>``
- ``<LiteralString>``
- ``<NoReturn>``
- ``<Never>``
**None**: The type expression ``None`` (``NoneType``) is ambiguous with the value ``None``,
so must use the explicit ``TypeForm(...)`` syntax:
- As a value expression, ``TypeForm(None)`` has type ``TypeForm[None]``.
- As a value expression, ``None`` continues to have type ``None``.
The following **parameterized type expressions** can be recognized unambiguously:
- As a value expression, ``X`` has type ``TypeForm[X]``,
for each of the following values of X:
- ``<Literal> '[' ... ']'``
- ``<Optional> '[' ... ']'``
- ``<Union> '[' ... ']'``
- ``<Callable> '[' ... ']'``
- ``<tuple> '[' ... ']'``
- ``<TypeGuard> '[' ... ']'``
- ``<TypeIs> '[' ... ']'``
.. _recognizing_annotated:
**Annotated**: The type expression ``Annotated[...]`` is ambiguous with
the annotation expression ``Annotated[...]``,
so must be disambiguated based on its argument type:
- As a value expression, ``Annotated[x, ...]`` has type ``type[C]``
if ``x`` has type ``type[C]``.
- As a value expression, ``Annotated[x, ...]`` has type ``TypeExpr[T]``
if ``x`` has type ``TypeExpr[T]``.
- As a value expression, ``Annotated[x, ...]`` has type ``object``
if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.
**Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``,
so must be disambiguated based on its argument type:
- As a value expression, ``x | y`` has type ``TypeForm[x | y]``
if ``x`` has type ``TypeForm[t1]`` (or ``type[t1]``)
and ``y`` has type ``TypeForm[t2]`` (or ``type[t2]``).
- As a value expression, ``x | y`` has type ``int``
if ``x`` has type ``int`` and ``y`` has type ``int``
The **stringified type expression** ``"T"`` is ambiguous with both
the stringified annotation expression ``"T"``
and the string literal ``"T"``,
so must use the explicit ``TypeForm(...)`` syntax:
- As a value expression, ``TypeForm("T")`` has type ``TypeForm[T]``,
where ``T`` is a valid type expression
- As a value expression, ``"T"`` continues to have type ``Literal["T"]``.
No other kinds of type expressions currently exist.
New kinds of type expressions that are introduced should define how they
will be recognized in a value expression context.
Implicit Annotation Expression Values
'''''''''''''''''''''''''''''''''''''
Extending the rules for :ref:`recognizing type expressions <implicit_typeexpr_values>`
in a value expression context, the following rules for recognizing
annotation expressions are defined:
The following **unparameterized annotation expressions** can be recognized unambiguously:
- As a value expression, ``X`` has type ``object``,
for each of the following values of X:
- ``<TypeAlias>``
The following **parameterized annotation expressions** can be recognized unambiguously:
- As a value expression, ``X`` has type ``object``,
for each of the following values of X:
- ``<Required> '[' ... ']'``
- ``<NotRequired> '[' ... ']'``
- ``<ReadOnly> '[' ... ']'``
- ``<ClassVar> '[' ... ']'``
- ``<Final> '[' ... ']'``
- ``<InitVar> '[' ... ']'``
- ``<Unpack> '[' ... ']'``
**Annotated**: The annotation expression ``Annotated[...]`` is ambiguous with
the type expression ``Annotated[...]``,
so must be :ref:`disambiguated based on its argument type <recognizing_annotated>`.
The following **syntactic annotation expressions**
cannot be recognized in a value expression context at all:
- ``'*' unpackable``
- ``name '.' 'args'`` (where ``name`` must be an in-scope ParamSpec)
- ``name '.' 'kwargs'`` (where ``name`` must be an in-scope ParamSpec)
The **stringified annotation expression** ``"T"`` is ambiguous with both
the stringified type expression ``"T"``
and the string literal ``"T"``, and
cannot be recognized in a value expression context at all:
- As a value expression, ``"T"`` continues to have type ``Literal["T"]``.
No other kinds of annotation expressions currently exist.
New kinds of annotation expressions that are introduced should define how they
will (or will not) be recognized in a value expression context.
Literal[] TypeExprs
'''''''''''''''''''
To simplify static type checking, a ``Literal[...]`` value is *not*
considered assignable to a ``TypeExpr`` variable even if all of its members
spell valid types:
::
STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr
Static vs. Runtime Representations of TypeExprs
'''''''''''''''''''''''''''''''''''''''''''''''
A ``TypeExpr`` value appearing statically in a source file may be normalized
to a different representation at runtime. For example string-based
forward references are normalized at runtime to be ``ForwardRef`` instances
in some contexts: [#forward_ref_normalization]_
::
>>> IntTree = list[typing.Union[int, 'IntTree']]
>>> IntTree
list[typing.Union[int, ForwardRef('IntTree')]]
The runtime representations of ``TypeExpr``\ s are considered implementation
details that may change over time and therefore static type checkers are
not required to recognize them:
::
INT_TREE: TypeExpr = ForwardRef('IntTree') # ERROR: Runtime-only form
Runtime type checkers that wish to assign a runtime-only representation
of a type expression to a ``TypeExpr[]`` variable must use ``cast()`` to
avoid errors from static type checkers:
::
INT_TREE = cast(TypeExpr, ForwardRef('IntTree')) # OK
Subtyping
---------
Whether a ``TypeExpr`` value can be assigned from one variable to another is
determined by the following rules:
``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:
- ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
subtype of ``T2``.
- ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
of ``C2``.
A plain ``type`` can be assigned to a plain ``TypeExpr`` but not the
other way around:
- ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
other way around.)
``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:
- ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.
``TypeExpr[T]``, where ``T`` is a type variable, is assumed to have all
the attributes and methods of ``object`` and is not callable.
Interactions with isinstance() and issubclass()
-----------------------------------------------
The ``TypeExpr`` special form cannot be used as the ``type`` argument to
``isinstance``:
::
>>> isinstance(str, TypeExpr)
TypeError: typing.TypeExpr cannot be used with isinstance()
>>> isinstance(str, TypeExpr[str])
TypeError: isinstance() argument 2 cannot be a parameterized generic
The ``TypeExpr`` special form cannot be used as any argument to
``issubclass``:
::
>>> issubclass(TypeExpr, object)
TypeError: issubclass() arg 1 must be a class
>>> issubclass(object, TypeExpr)
TypeError: typing.TypeExpr cannot be used with issubclass()
Affected signatures in the standard library
-------------------------------------------
Changed signatures
''''''''''''''''''
The following signatures related to type expressions introduce
``TypeExpr`` where previously ``object`` existed:
- ``typing.cast``
- ``typing.assert_type``
Unchanged signatures
''''''''''''''''''''
The following signatures related to annotation expressions continue to
use ``object`` and remain unchanged:
- ``typing.get_origin``
- ``typing.get_args``
The following signatures related to class objects continue to use
``type`` and remain unchanged:
- ``builtins.isinstance``
- ``builtins.issubclass``
- ``builtins.type``
``typing.get_type_hints(..., include_extras=False)`` nearly returns only type
expressions in Python 3.12, stripping out most type qualifiers
(``Required, NotRequired, ReadOnly, Annotated``) but currently preserves a
few type qualifiers which are only allowed in annotation expressions
(``ClassVar, Final, InitVar, Unpack``). It may be desirable to alter the
behavior of this function in the future to also strip out those
qualifiers and actually return type expressions, although this PEP does
not propose those changes now:
- ``typing.get_type_hints(..., include_extras=False)``
- Almost returns only type expressions, but not quite
- ``typing.get_type_hints(..., include_extras=True)``
- Returns annotation expressions
Backwards Compatibility
=======================
Previously the rules for recognizing type expression objects
in a value expression context were not defined, so static type checkers
`varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
to such objects. Existing programs manipulating type expression objects
were already limited in manipulating them as plain ``object`` values,
and such programs should not break with
:ref:`the newly-defined rules <implicit_typeexpr_values>`.
How to Teach This
=================
Most users interacting with ``TypeExpr`` will do so only
in a limited way, by passing a literal type expression to a function
accepting a ``TypeExpr`` input, imported from a runtime type checker
library.
For those implementing runtime type checkers,
``TypeExpr``\ s can be used in combination with other typing features to
write useful functions that accept type expression objects:
Combining with a type variable
------------------------------
``TypeExpr[]`` can be parameterized by a type variable that is used elsewhere within
the same function definition:
::
def as_instance[T](typx: TypeExpr[T]) -> T | None:
return typx() if isinstance(typx, type) else None
Combining with type[]
---------------------
Both ``TypeExpr[]`` and ``type[]`` can be parameterized by the same type
variable within the same function definition:
::
def as_type[T](typx: TypeExpr[T]) -> type[T] | None:
return typx if isinstance(typx, type) else None
Combining with TypeIs[] and TypeGuard[]
---------------------------------------
A type variable parameterizing a ``TypeExpr[]`` can also be used by a ``TypeIs[]``
within the same function definition:
::
def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...
count: int | str = ...
if isassignable(count, int):
assert_type(count, int)
else:
assert_type(count, str)
or by a ``TypeGuard[]`` within the same function definition:
::
def isdefault[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]:
return (value == typx()) if isinstance(typx, type) else False
value: int | str = ''
if isdefault(value, int):
assert_type(value, int)
assert 0 == value
elif isdefault(value, str):
assert_type(value, str)
assert '' == value
else:
assert_type(value, int | str)
Introspecting TypeExpr Values
-----------------------------
A ``TypeExpr`` is very similar to an ``object`` at runtime, with no additional
attributes or methods defined.
You can use existing introspection functions like ``typing.get_origin`` and
``typing.get_args`` to extract the components of a type expression that looks
like ``Origin[Arg1, Arg2, ..., ArgN]``:
::
import typing
def strip_annotated_metadata(typx: TypeExpr[T]) -> TypeExpr[T]:
if typing.get_origin(typx) == typing.Annotated:
typx = cast(TypeExpr[T], typing.get_args(typx)[0])
return typx
You can also use ``isinstance`` and ``==`` to distinguish one kind of
type expression from another:
::
import types
import typing
def split_union(typx: TypeExpr) -> tuple[TypeExpr, ...]:
if isinstance(typx, types.UnionType): # X | Y
return cast(tuple[TypeExpr, ...], typing.get_args(typx))
if typing.get_origin(typx) == typing.Union: # Union[X, Y]
return cast(tuple[TypeExpr, ...], typing.get_args(typx))
if typx in (typing.Never, typing.NoReturn,):
return ()
return (typx,)
Challenges When Accepting All TypeExprs
---------------------------------------
A function that takes an *arbitrary* ``TypeExpr`` as
input must support a large variety of possible type expressions and is
not easy to write. Some challenges faced by such a function include:
- An ever-increasing number of typing special forms are introduced with
each new Python version which must be recognized, with special
handling required for each one.
- Stringified type annotations [#strann_less_common]_ (like ``'list[str]'``)
must be *parsed* (to something like ``typing.List[str]``) to be introspected.
- In practice it is extremely difficult for stringified type
annotations to be handled reliably at runtime, so runtime type
checkers may opt to not support them at all.
- Resolving string-based forward references inside type
expressions to actual values must typically be done using ``eval()``,
which is difficult/impossible to use in a safe way.
- Recursive types like ``IntTree = list[typing.Union[int, 'IntTree']]``
are not possible to fully resolve.
- Supporting user-defined generic types (like Django’s
``QuerySet[User]``) requires user-defined functions to
recognize/parse, which a runtime type checker must provide a
registration API for.
Consider importing a function from an existing runtime type checker
library rather than writing your own.
Reference Implementation
========================
The following will be true when
`mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:
The mypy type checker supports ``TypeExpr`` types.
A reference implementation of the runtime component is provided in the
``typing_extensions`` module.
Rejected Ideas
==============
.. _widen_type_C_to_support_all_type_expressions:
Widen type[C] to support all type expressions
---------------------------------------------
``type`` was `designed`_ to only be used to describe class objects. A
class object can always be used as the second argument of ``isinstance()``
and can usually be instantiated by calling it.
``TypeExpr`` on the other hand is typically introspected by the user in
some way, is not necessarily directly instantiable, and is not
necessarily directly usable in a regular ``isinstance()`` check.
It would be possible to widen ``type`` to include the additional values
allowed by ``TypeExpr`` but it would reduce clarity about the user’s
intentions when working with a ``type``. Different concepts and usage
patterns; different spellings.
.. _designed: https://mail.python.org/archives/list/typing-sig@python.org/message/D5FHORQVPHX3BHUDGF3A3TBZURBXLPHD/
Accept arbitrary annotation expressions
---------------------------------------
Certain typing special forms can be used in *some* but not *all*
annotation contexts:
For example ``Final[]`` can be used as a variable type but not as a
parameter type or a return type:
::
some_const: Final[str] = ... # OK
def foo(not_reassignable: Final[object]): ... # ERROR: Final[] not allowed here
def nonsense() -> Final[object]: ... # ERROR: Final[] not meaningful here
``TypeExpr[T]`` does not allow matching such annotation expressions
because it is not clear what it would mean for such an expression
to parameterized by a type variable in position ``T``:
::
def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: ...
def foo(some_arg):
if ismatch(some_arg, Final[int]): # ERROR: Final[int] is not a TypeExpr
reveal_type(some_arg) # ? NOT Final[int], because invalid for a parameter
Functions that wish to operate on *all* kinds of annotation expressions,
including those that are not ``TypeExpr``\ s, can continue to accept such
inputs as ``object`` parameters, as they must do so today.
Accept only universal type expressions
--------------------------------------
Earlier drafts of this PEP only allowed ``TypeExpr[]`` to match the subset
of type expressions which are valid in *all* contexts, excluding
:ref:`non-universal type expressions <non_universal_typeexpr>`.
However doing that would effectively
create a new subset of annotation expressions that Python typing users
would have to understand, on top of all the existing distinctions between
“class objects”, “type expressions”, and “annotation expressions”.
To avoid introducing yet another concept that everyone has to learn,
this proposal just rounds ``TypeExpr[]`` to exactly match the existing
definition of a “type expression”.
Support pattern matching on type expressions
--------------------------------------------
It was asserted that some functions may wish to pattern match on the
interior of type expressions in their signatures.
One use case is to allow a function to explicitly enumerate all the
*specific* kinds of type expressions it supports as input.
Consider the following possible pattern matching syntax:
::
@overload
def checkcast(typx: TypeExpr[AT=Annotated[T, *Anns]], value: str) -> T: ...
@overload
def checkcast(typx: TypeExpr[UT=Union[*Ts]], value: str) -> Union[*Ts]: ...
@overload
def checkcast(typx: type[C], value: str) -> C: ...
# ... (more)
All functions observed in the wild that conceptually take a ``TypeExpr[]``
generally try to support *all* kinds of type expressions, so it doesn’t
seem valuable to enumerate a particular subset.
Additionally the above syntax isn’t precise enough to fully describe the
actual input constraints for a typical function in the wild. For example
many functions recognize un-stringified type expressions like
``list[Movie]`` but may not recognize type expressions with stringified
subcomponents like ``list['Movie']``.
A second use case for pattern matching on the interior of type
expressions is to explicitly match an ``Annotated[]`` form to pull out the
interior type argument and strip away the metadata:
::
def checkcast(
typx: TypeExpr[T] | TypeExpr[AT=Annotated[T, *Anns]],
value: object
) -> T:
However ``Annotated[T, metadata]`` is already treated equivalent to ``T`` anyway.
There’s no additional value in being explicit about this behavior.
The example above could be more-straightforwardly written as the equivalent:
::
def checkcast(typx: TypeExpr[T], value: object) -> T:
Footnotes
=========
.. [#type_c]
:pep:`Type[C] spells a class object <484#the-type-of-class-objects>`
.. [#TypeIsPep]
:pep:`TypeIs[T] is similar to bool <742>`
.. [#DataclassInitVar]
``dataclass.make_dataclass`` accepts ``InitVar[...]`` as a special case
in addition to type expressions. Therefore it may unfortunately be necessary
to continue annotating its ``type`` parameter as ``object`` rather
than ``TypeExpr``.
.. [#forward_ref_normalization]
Special forms normalize string arguments to ``ForwardRef`` instances
at runtime using internal helper functions in the ``typing`` module.
Runtime type checkers may wish to implement similar functions when
working with string-based forward references.
.. [#strann_less_common]
Stringified type annotations are expected to become less common
starting in Python 3.14 when :pep:`deferred annotations <649>`
become available. However there is a large amount of existing code from
earlier Python versions relying on stringified type annotations that will
still need to be supported for several years.
Copyright
=========
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.