python-peps/peps/pep-0747.rst

PEP: 747
Title: TypeExpr: Type Hint for a Type Expression
Author: David Foster <david at dafoster.net>
Sponsor: Jelle Zijlstra <jelle.zijlstra at gmail.com>
Discussions-To: https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984
Status: Draft
Type: Standards Track
Topic: Typing
Created: 27-May-2024
Python-Version: 3.14
Post-History: `19-Apr-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435>`__, `04-May-2024 <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/7/>`__, `17-Jun-2024 <https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984>`__


Abstract
========

:pep:`484` defines the notation ``type[C]`` where ``C`` is a class, to
refer to a class object that is a subtype of ``C``. It explicitly does
not allow ``type[C]`` to refer to arbitrary
:ref:`type expression <typing:type-expression>` objects such
as the runtime object ``str | None``, even if ``C`` is an unbounded
``TypeVar``. [#type_c]_ In cases where that restriction is unwanted, this
PEP proposes a new notation ``TypeExpr[T]`` where ``T`` is a type, to
refer to a either a class object or some other type expression object
that is a subtype of ``T``, allowing any kind of type to be referenced.

This PEP makes no Python grammar changes. Correct usage of
``TypeExpr[]`` is intended to be enforced only by static and runtime
type checkers and need not be enforced by Python itself at runtime.


.. _motivation:

Motivation
==========

The introduction of ``TypeExpr`` allows new kinds of metaprogramming
functions that operate on type expressions to be type-annotated and
understood by type checkers.

For example, here is a function that checks whether a value is
assignable to a variable of a particular type, and if so returns the
original value:

::

   def trycast[T](typx: TypeExpr[T], value: object) -> T | None: ...

The use of ``TypeExpr[]`` and the type variable ``T`` enables the return
type of this function to be influenced by a ``typx`` value passed at
runtime, which is quite powerful.

Here is another function that checks whether a value is assignable to a
variable of a particular type, and if so returns ``True`` (as a special
``TypeIs[]`` bool [#TypeIsPep]_):

::

   def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...

The use of ``TypeExpr[]`` and ``TypeIs[]`` together enables type
checkers to narrow the return type appropriately depending on what type
expression is passed in:

::

   request_json: object = ...
   if isassignable(request_json, MyTypedDict):
       assert_type(request_json, MyTypedDict)  # type is narrowed!

That ``isassignable`` function enables a kind of enhanced ``isinstance``
check which is useful for `checking whether a value decoded from JSON
conforms to a particular structure`_ of nested ``TypedDict``\ s,
lists, unions, ``Literal``\ s, and other types. This kind
of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` but could
not be implemented at the time without a notation similar to
``TypeExpr[]``.

.. _checking whether a value decoded from JSON conforms to a particular structure: https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/


Why can’t ``type[]`` be used?
-----------------------------

One might think you could define the example functions above to take a
``type[C]`` - which is syntax that already exists - rather than a
``TypeExpr[T]``. However if you were to do that then certain type
expressions like ``str | None`` - which are not class objects and
therefore not ``type``\ s at runtime - would be rejected:

::

   # NOTE: Uses a type[C] parameter rather than a TypeExpr[T]
   def trycast_type[C](typ: type[C], value: object) -> T | None: ...

   trycast_type(str, 'hi')  # ok; str is a type
   trycast_type(Optional[str], 'hi')  # ERROR; Optional[str] is not a type
   trycast_type(str | int, 'hi')  # ERROR; (str | int) is not a type
   trycast_type(MyTypedDict, dict(value='hi'))  # questionable; accepted by mypy 1.9.0

To solve that problem, ``type[]`` could be widened to include the
additional values allowed by ``TypeExpr``. However doing so would lose
``type[]``\ ’s current ability to spell a class object which always
supports instantiation and ``isinstance`` checks, unlike arbitrary type
expression objects. Therefore ``TypeExpr`` is proposed as new notation
instead.

For a longer explanation of why we don’t just widen ``type[T]`` to
accept all type expressions, see
:ref:`widen_type_C_to_support_all_type_expressions`.


.. _runtime_type_checkers_using_typeexpr:

Common kinds of functions that would benefit from TypeExpr
----------------------------------------------------------

`A survey of various Python libraries`_ revealed a few kinds of commonly
defined functions which would benefit from ``TypeExpr[]``:

.. _A survey of various Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886

-  Assignability checkers:

   -  Returns whether a value is assignable to a type expression. If so
      then also narrows the type of the value to match the type
      expression.
   -  Pattern 1:
      ``def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]``
   -  Pattern 2:
      ``def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]``
   -  Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_,
      typeguard.\ `check_type`_, xdsl.\ `isa`_

.. _is_bearable: https://github.com/beartype/beartype/issues/255
.. _isassignable: https://github.com/davidfstr/trycast?tab=readme-ov-file#isassignable-api
.. _check_type: https://typeguard.readthedocs.io/en/latest/api.html#typeguard.check_type
.. _isa: https://github.com/xdslproject/xdsl/blob/ac12c9ab0d64618475efb98d1d197bdd79f593c3/xdsl/utils/hints.py#L23

-  Converters:

   -  If a value is assignable to (or coercible to) a type expression,
      a *converter* returns the value narrowed to (or coerced to) that type
      expression. Otherwise, it raises an exception.
   -  Pattern 1:
      ``def convert[T](value: object, typx: TypeExpr[T]) -> T``

      -  Examples: cattrs.BaseConverter.\ `structure`_, trycast.\ `checkcast`_,
         typedload.\ `load`_

   -  Pattern 2:

      ::

        class Converter[T]:
            def __init__(self, typx: TypeExpr[T]) -> None: ...
            def convert(self, value: object) -> T: ...

      -  Examples: pydantic.\ `TypeAdapter(T).validate_python`_,
         mashumaro.\ `JSONDecoder(T).decode`_

.. _structure: https://github.com/python-attrs/cattrs/blob/5f5c11627a7f67a23d6212bc7df9f96243c62dc5/src/cattrs/converters.py#L332-L334
.. _checkcast: https://github.com/davidfstr/trycast#checkcast-api
.. _load: https://ltworf.github.io/typedload/
.. _TypeAdapter(T).validate_python: https://stackoverflow.com/a/61021183/604063
.. _JSONDecoder(T).decode: https://github.com/Fatal1ty/mashumaro?tab=readme-ov-file#usage-example

-  Typed field definitions:

   -  Pattern:

      ::

        class Field[T]:
            value_type: TypeExpr[T]

   -  Examples: attrs.\ `make_class`_,
      dataclasses.\ `make_dataclass`_ [#DataclassInitVar]_, `openapify`_

.. _make_class: https://www.attrs.org/en/stable/api.html#attrs.make_class
.. _make_dataclass: https://github.com/python/typeshed/issues/11653
.. _openapify: https://github.com/Fatal1ty/openapify/blob/c8d968c7c9c8fd7d4888bd2ddbe18ffd1469f3ca/openapify/core/models.py#L16

The survey also identified some introspection functions that take
annotation expressions as input using plain ``object``\ s which would
*not* gain functionality by marking those inputs as ``TypeExpr[]``:

-  General introspection operations:

   -  Pattern: ``def get_annotation_info(maybe_annx: object) -> object``
   -  Examples: typing.{`get_origin`_, `get_args`_},
      `typing_inspect`_.{is_*_type, get_origin, get_parameters}

.. _get_origin: https://docs.python.org/3/library/typing.html#typing.get_origin
.. _get_args: https://docs.python.org/3/library/typing.html#typing.get_args
.. _typing_inspect: https://github.com/ilevkivskyi/typing_inspect?tab=readme-ov-file#readme


Rationale
=========

Before this PEP existed there were already a few definitions in use to describe
different kinds of type annotations:

.. code-block:: text

   +----------------------------------+ 
   | +------------------------------+ | 
   | | +-------------------------+  | | 
   | | | +---------------------+ |  | | 
   | | | | Class object        | |  | | = type[C]        
   | | | +---------------------+ |  | | 
   | | | Type expression object  |  | | = TypeExpr[T]  <-- new!
   | | +-------------------------+  | | 
   | | Annotation expression object | | 
   | +------------------------------+ | 
   | Object                           | = object         
   +----------------------------------+ 

-  :ref:`Class objects <typing:type-brackets>`,
   spelled as ``type[C]``, support ``isinstance`` checks and are callable.

   -  Examples: ``int``, ``str``, ``MyClass``

-  :ref:`Type expressions <typing:type-expression>`
   include any type annotation which describes a type.

   -  Examples: ``list[int]``, ``MyTypedDict``, ``int | str``,
      ``Literal['square']``, any class object

-  :ref:`Annotation expressions <typing:annotation-expression>`
   include any type annotation, including those only valid in specific contexts.

   -  Examples: ``Final[int]``, ``Required[str]``, ``ClassVar[str]``,
      any type expression

``TypeExpr`` aligns with an existing definition from the above list -
*type expression* - to avoid introducing yet another subset of type annotations
that users of Python typing need to think about.

``TypeExpr`` aligns with *type expression* specifically
because a type expression is already used to parameterize type variables,
which are used in combination with ``TypeIs`` and ``TypeGuard`` to enable
the compelling examples mentioned in :ref:`Motivation <motivation>`.

``TypeExpr`` does not align with *annotation expression* for reasons given in
:ref:`Rejected Ideas » Accept arbitrary annotation expressions <accept_arbitrary_annotation_expressions>`.


Specification
=============

A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>`
such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
A ``TypeExpr`` type is written as
``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
written without brackets as just ``TypeExpr``, which is treated the same as
to ``TypeExpr[Any]``.


Using TypeExprs
---------------

A ``TypeExpr`` is a new kind of type expression, usable in any context where a
type expression is valid, as a function parameter type, a return type, 
or a variable type:

::

   def is_union_type(typx: TypeExpr) -> bool: ...  # parameter type

::

   def union_of[S, T](s: TypeExpr[S], t: TypeExpr[T]) \
       -> TypeExpr[S | T]: ...  # return type

::

   STR_TYPE: TypeExpr = str  # variable type

Note however that an *unannotated* variable assigned a type expression literal
will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
484 :pep:`reserves that syntax for defining type aliases <484#type-aliases>`:

-  No:

   ::

      STR_TYPE = str  # OOPS; treated as a type alias!

If you want a type checker to recognize a type expression literal in a bare
assignment you’ll need to explicitly declare the assignment-target as
having ``TypeExpr`` type:

-  Yes:

   ::

      STR_TYPE: TypeExpr = str

-  Yes:

   ::

      STR_TYPE: TypeExpr
      STR_TYPE = str

-  Okay, but discouraged:

   ::

      STR_TYPE = str  # type: TypeExpr  # the type comment is significant

``TypeExpr`` values can be passed around and assigned just like normal
values:

::

   def swap1[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
       t1_new: TypeExpr[T] = t2  # assigns a TypeExpr value to a new annotated variable
       t2_new: TypeExpr[S] = t1
       return (t1_new, t2_new)

   def swap2[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
       t1_new = t2  # assigns a TypeExpr value to a new unannotated variable
       t2_new = t1
       assert_type(t1_new, TypeExpr[T])
       assert_type(t2_new, TypeExpr[S])
       return (t1_new, t2_new)

   # NOTE: A more straightforward implementation would use isinstance()
   def ensure_int(value: object) -> None:
       value_type: TypeExpr = type(value)  # assigns a type (a subtype of TypeExpr)
       assert value_type == int


TypeExpr Values
---------------

A variable of type ``TypeExpr[T]`` where ``T`` is a type, can hold any
**type expression object** - the result of evaluating a 
:ref:`type expression <typing:type-expression>`
at runtime - which is a subtype of ``T``.

Incomplete expressions like a bare ``Optional`` or ``Union`` which do
not spell a type are not ``TypeExpr`` values.

``TypeExpr[...]`` is itself a ``TypeExpr`` value:

::

   OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None]  # OK
   assert isassignable(int | None, OPTIONAL_INT_TYPE)

.. _non_universal_typeexpr:

``TypeExpr[]`` values include *all* type expressions including some
**non-universal type expressions** which are not valid in all annotation contexts.
In particular:

-  ``Self`` (valid only in some contexts)
-  ``TypeGuard[...]`` (valid only in some contexts)
-  ``TypeIs[...]`` (valid only in some contexts)


Explicit TypeExpr Values
''''''''''''''''''''''''

The syntax ``TypeExpr(T)`` (with parentheses) can be used to
spell a ``TypeExpr[T]`` value explicitly:

::

   NONE = TypeExpr(None)
   INT1 = TypeExpr('int')  # stringified type expression
   INT2 = TypeExpr(int)

At runtime the ``TypeExpr(...)`` callable returns its single argument unchanged.


.. _implicit_typeexpr_values:

Implicit TypeExpr Values
''''''''''''''''''''''''

Historically static type checkers have only needed to recognize
*type expressions* in contexts where a type expression was expected.
Now *type expression objects* must also be recognized in contexts where a
value expression is expected.

Static type checkers already recognize **class objects** (``type[C]``):

-  As a value expression, ``C`` has type ``type[C]``,
   for each of the following values of C:

   -  ``name`` (where ``name`` must refer to a valid in-scope class, type alias, or TypeVar)
   -  ``name '[' ... ']'``
   -  ``<type> '[' ... ']'``

The following **unparameterized type expressions** can be recognized unambiguously:

-  As a value expression, ``X`` has type ``TypeExpr[X]``,
   for each of the following values of X:

   -  ``<Any>``
   -  ``<Self>``
   -  ``<LiteralString>``
   -  ``<NoReturn>``
   -  ``<Never>``

**None**: The type expression ``None`` (``NoneType``) is ambiguous with the value ``None``,
so must use the explicit ``TypeExpr(...)`` syntax:

-  As a value expression, ``TypeExpr(None)`` has type ``TypeExpr[None]``.
-  As a value expression, ``None`` continues to have type ``None``.

The following **parameterized type expressions** can be recognized unambiguously:

-  As a value expression, ``X`` has type ``TypeExpr[X]``,
   for each of the following values of X:

   -  ``<Literal> '[' ... ']'``
   -  ``<Optional> '[' ... ']'``
   -  ``<Union> '[' ... ']'``
   -  ``<Callable> '[' ... ']'``
   -  ``<tuple> '[' ... ']'``
   -  ``<TypeGuard> '[' ... ']'``
   -  ``<TypeIs> '[' ... ']'``

.. _recognizing_annotated:

**Annotated**: The type expression ``Annotated[...]`` is ambiguous with 
the annotation expression ``Annotated[...]``,
so must be disambiguated based on its argument type:

-  As a value expression, ``Annotated[x, ...]`` has type ``type[C]``
   if ``x`` has type ``type[C]``.
-  As a value expression, ``Annotated[x, ...]`` has type ``TypeExpr[T]``
   if ``x`` has type ``TypeExpr[T]``.
-  As a value expression, ``Annotated[x, ...]`` has type ``object``
   if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.

**Union**: The type expression ``T1 | T2`` is ambiguous with 
the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more,
so must be disambiguated based on its argument types:

-  As a value expression, ``x | y`` has type equal to the return type of ``type(x).__or__``
   if ``type(x)`` overrides the ``__or__`` method.

   -  When ``x`` has type ``builtins.type``, ``types.GenericAlias``, or the
      internal type of a typing special form, ``type(x).__or__`` has a return type
      in the format ``TypeExpr[T1 | T2]``.

-  As a value expression, ``x | y`` has type equal to the return type of ``type(y).__ror__``
   if ``type(y)`` overrides the ``__ror__`` method.

   -  When ``y`` has type ``builtins.type``, ``types.GenericAlias``, or the
      internal type of a typing special form, ``type(y).__ror__`` has a return type
      in the format ``TypeExpr[T1 | T2]``.

-  As a value expression, ``x | y`` has type ``UnionType``
   in all other situations.

   -  This rule is intended to be consistent with the preexisting fallback rule
      used by static type checkers.

The **stringified type expression** ``"T"`` is ambiguous with both
the stringified annotation expression ``"T"``
and the string literal ``"T"``,
so must use the explicit ``TypeExpr(...)`` syntax:

-  As a value expression, ``TypeExpr("T")`` has type ``TypeExpr[T]``,
   where ``T`` is a valid type expression
-  As a value expression, ``"T"`` continues to have type ``Literal["T"]``.

No other kinds of type expressions currently exist.

New kinds of type expressions that are introduced should define how they
will be recognized in a value expression context.


Literal[] TypeExprs
'''''''''''''''''''

A value of ``Literal[...]`` type is *not* considered assignable to
a ``TypeExpr`` variable even if all of its members spell valid types because
dynamic values are not allowed in type expressions:

::

   STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
   STRS_TYPE: TypeExpr = STRS_TYPE_NAME  # ERROR: Literal[] value is not a TypeExpr

However ``Literal[...]`` itself is still a ``TypeExpr``:

::

   DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right']  # OK


Static vs. Runtime Representations of TypeExprs
'''''''''''''''''''''''''''''''''''''''''''''''

A ``TypeExpr`` value appearing statically in a source file may be normalized
to a different representation at runtime. For example string-based 
forward references are normalized at runtime to be ``ForwardRef`` instances
in some contexts: [#forward_ref_normalization]_

::

   >>> IntTree = list[typing.Union[int, 'IntTree']]
   >>> IntTree
   list[typing.Union[int, ForwardRef('IntTree')]]

The runtime representations of ``TypeExpr``\ s are considered implementation
details that may change over time and therefore static type checkers are
not required to recognize them:

::

   INT_TREE: TypeExpr = ForwardRef('IntTree')  # ERROR: Runtime-only form

Runtime type checkers that wish to assign a runtime-only representation
of a type expression to a ``TypeExpr[]`` variable must use ``cast()`` to
avoid errors from static type checkers:

::

   INT_TREE = cast(TypeExpr, ForwardRef('IntTree'))  # OK


Subtyping
---------

Whether a ``TypeExpr`` value can be assigned from one variable to another is
determined by the following rules:

Relationship with type
''''''''''''''''''''''

``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:

-  ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
   subtype of ``T2``.
-  ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
   of ``C2``.

An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr``
but not the other way around:

-  ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
   other way around.)

Relationship with UnionType
'''''''''''''''''''''''''''

``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is
the type expression ``X | Y | ...``:

-  ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``.

``UnionType`` is assignable to ``TypeExpr[Any]``.

Relationship with object
''''''''''''''''''''''''

``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:

-  ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.

``TypeExpr[T]``, where ``T`` is a type variable, is assumed to have all
the attributes and methods of ``object`` and is not callable.


Interactions with isinstance() and issubclass()
-----------------------------------------------

The ``TypeExpr`` special form cannot be used as the ``type`` argument to
``isinstance``:

::

   >>> isinstance(str, TypeExpr)
   TypeError: typing.TypeExpr cannot be used with isinstance()

   >>> isinstance(str, TypeExpr[str])
   TypeError: isinstance() argument 2 cannot be a parameterized generic

The ``TypeExpr`` special form cannot be used as any argument to
``issubclass``:

::

   >>> issubclass(TypeExpr, object)
   TypeError: issubclass() arg 1 must be a class

   >>> issubclass(object, TypeExpr)
   TypeError: typing.TypeExpr cannot be used with issubclass()


Affected signatures in the standard library
-------------------------------------------

Changed signatures
''''''''''''''''''

The following signatures related to type expressions introduce
``TypeExpr`` where previously ``object`` or ``Any`` existed:

-  ``typing.cast``
-  ``typing.assert_type``

The following signatures transforming union type expressions introduce
``TypeExpr`` where previously ``UnionType`` existed so that a more-precise
``TypeExpr`` type can be inferred:

-  ``builtins.type[T].__or__``

   -  Old: ``def __or__(self, value: Any, /) -> types.UnionType: ...``
   -  New: ``def __or__[T2](self, value: TypeExpr[T2], /) -> TypeExpr[T | T2]: ...``

-  ``builtins.type[T].__ror__``

   -  Old: ``def __ror__(self, value: Any, /) -> types.UnionType: ...``
   -  New: ``def __ror__[T1](self, value: TypeExpr[T1], /) -> TypeExpr[T1 | T]: ...``

-  ``types.GenericAlias.{__or__,__ror__}``
-  «the internal type of a typing special form»``.{__or__,__ror__}``

However the implementations of those methods continue to return ``UnionType``
instances at runtime so that runtime ``isinstance`` checks like
``isinstance('42', int | str)`` and ``isinstance(int | str, UnionType)``
continue to work.


Unchanged signatures
''''''''''''''''''''

The following signatures related to annotation expressions continue to
use ``object`` and remain unchanged:

-  ``typing.get_origin``
-  ``typing.get_args``

The following signatures related to class objects continue to use
``type`` and remain unchanged:

-  ``builtins.isinstance``
-  ``builtins.issubclass``
-  ``builtins.type``

``typing.get_type_hints(..., include_extras=False)`` nearly returns only type
expressions in Python 3.12, stripping out most type qualifiers
(``Required, NotRequired, ReadOnly, Annotated``) but currently preserves a
few type qualifiers which are only allowed in annotation expressions
(``ClassVar, Final, InitVar, Unpack``). It may be desirable to alter the
behavior of this function in the future to also strip out those
qualifiers and actually return type expressions, although this PEP does
not propose those changes now:

-  ``typing.get_type_hints(..., include_extras=False)``

   -  Almost returns only type expressions, but not quite

-  ``typing.get_type_hints(..., include_extras=True)``

   -  Returns annotation expressions

The following signatures accepting union type expressions continue
to use ``UnionType``:

-  ``builtins.isinstance``
-  ``builtins.issubclass``
-  ``typing.get_origin`` (used in an ``@overload``)

The following signatures transforming union type expressions continue
to use ``UnionType`` because it is not possible to infer a more-precise
``TypeExpr`` type:

-  ``types.UnionType.{__or__,__ror__}``


Backwards Compatibility
=======================

As a value expression, ``X | Y`` previously had type ``UnionType`` (via :pep:`604`)
but this PEP gives it the more-precise static type ``TypeExpr[X | Y]`` 
(a subtype of ``UnionType``) while continuing to return a ``UnionType`` instance at runtime.
Preserving compability with ``UnionType`` is important because ``UnionType``
supports ``isinstance`` checks, unlike ``TypeExpr``, and existing code relies
on being able to perform those checks.

The rules for recognizing other kinds of type expression objects
in a value expression context were not previously defined, so static type checkers
`varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
to such objects. Existing programs manipulating type expression objects
were already limited in manipulating them as plain ``object`` values,
and such programs should not break with
:ref:`the newly-defined rules <implicit_typeexpr_values>`.


How to Teach This
=================

Normally when using type annotations in Python you're concerned with defining
the shape of values allowed to be passed to a function parameter, returned
by a function, or stored in a variable:

.. code-block:: text

                 parameter type   return type
                 |                |
                 v                v
   def plus(n1: int, n2: int) -> int:
       sum: int = n1 + n2
             ^
             |
             variable type
       
       return sum

However type annotations themselves are valid values in Python and can be
assigned to variables and manipulated like any other data in a program:

.. code-block:: text

    a variable                    a type
    |                             |
    v                             v
   MAYBE_INT_TYPE: TypeExpr = int | None
                    ^ 
                    | 
                    the type of a type

``TypeExpr[]`` is how you spell the type of a variable containing a
type annotation object describing a type.

``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only
spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``.
``TypeExpr[]`` by contrast can additionally spell more complex types, 
including those with brackets (like ``list[int]``) or pipes (like ``int | None``),
and including special types like ``Any``, ``LiteralString``, or ``Never``.

A ``TypeExpr`` variable looks similar to a ``TypeAlias`` definition, but
can only be used where a dynamic value is expected.
``TypeAlias`` (and the ``type`` statement) by contrast define a name that can
be used where a fixed type is expected:

-  Okay, but discouraged in Python 3.12+:

   ::

      MaybeFloat: TypeAlias = float | None
      def sqrt(n: float) -> MaybeFloat: ...

-  Yes:

   ::

      type MaybeFloat = float | None
      def sqrt(n: float) -> MaybeFloat: ...

-  No:

   ::

      maybe_float: TypeExpr = float | None
      def sqrt(n: float) -> maybe_float: ...  # ERROR: Can't use TypeExpr value in a type annotation

It is uncommon for a programmer to define their *own* function which accepts
a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common
for a programmer to pass a literal type expression to an *existing* function
accepting a ``TypeExpr`` input which was imported from a runtime type checker
library.

For example the ``isassignable`` function from the ``trycast`` library
can be used like Python's built-in ``isinstance`` function to check whether
a value matches the shape of a particular type.
``isassignable`` will accept *any* kind of type as an input because its input
is a ``TypeExpr``. By contrast ``isinstance`` only accepts a simple class object
(a ``type[]``) as input:

-  Yes:

   ::

      from trycast import isassignable
      
      if isassignable(some_object, MyTypedDict):  # OK: MyTypedDict is a TypeExpr[]
          ...

-  No:

   ::

      if isinstance(some_object, MyTypedDict):  # ERROR: MyTypedDict is not a type[]
          ...

There are :ref:`many other runtime type checkers <runtime_type_checkers_using_typeexpr>`
providing useful functions that accept a ``TypeExpr``.


.. _advanced_examples:

Advanced Examples
=================

If you want to write your own runtime type checker or some other
kind of function that manipulates types as values at runtime,
this section gives examples of how you might implement such a function
using ``TypeExpr``.


Introspecting TypeExpr Values
-----------------------------

A ``TypeExpr`` is very similar to an ``object`` at runtime, with no additional
attributes or methods defined.

You can use existing introspection functions like ``typing.get_origin`` and
``typing.get_args`` to extract the components of a type expression that looks
like ``Origin[Arg1, Arg2, ..., ArgN]``:

::

   import typing

   def strip_annotated_metadata(typx: TypeExpr[T]) -> TypeExpr[T]:
       if typing.get_origin(typx) is typing.Annotated:
           typx = cast(TypeExpr[T], typing.get_args(typx)[0])
       return typx

You can also use ``isinstance`` and ``is`` to distinguish one kind of
type expression from another:

::

   import types
   import typing

   def split_union(typx: TypeExpr) -> tuple[TypeExpr, ...]:
       if isinstance(typx, types.UnionType):  # X | Y
           return cast(tuple[TypeExpr, ...], typing.get_args(typx))
       if typing.get_origin(typx) is typing.Union:  # Union[X, Y]
           return cast(tuple[TypeExpr, ...], typing.get_args(typx))
       if typx in (typing.Never, typing.NoReturn,):
           return ()
       return (typx,)


Combining with a type variable
------------------------------

``TypeExpr[]`` can be parameterized by a type variable that is used elsewhere within
the same function definition:

::

   def as_instance[T](typx: TypeExpr[T]) -> T | None:
       return typx() if isinstance(typx, type) else None


Combining with type[]
---------------------

Both ``TypeExpr[]`` and ``type[]`` can be parameterized by the same type
variable within the same function definition:

::

   def as_type[T](typx: TypeExpr[T]) -> type[T] | None:
       return typx if isinstance(typx, type) else None


Combining with TypeIs[] and TypeGuard[]
---------------------------------------

A type variable parameterizing a ``TypeExpr[]`` can also be used by a ``TypeIs[]``
within the same function definition:

::

   def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...

   count: int | str = ...
   if isassignable(count, int):
       assert_type(count, int)
   else:
       assert_type(count, str)

or by a ``TypeGuard[]`` within the same function definition:

::

   def isdefault[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]:
       return (value == typx()) if isinstance(typx, type) else False

   value: int | str = ''
   if isdefault(value, int):
       assert_type(value, int)
       assert 0 == value
   elif isdefault(value, str):
       assert_type(value, str)
       assert '' == value
   else:
       assert_type(value, int | str)


Challenges When Accepting All TypeExprs
---------------------------------------

A function that takes an *arbitrary* ``TypeExpr`` as
input must support a large variety of possible type expressions and is
not easy to write. Some challenges faced by such a function include:

-  An ever-increasing number of typing special forms are introduced with
   each new Python version which must be recognized, with special
   handling required for each one.
-  Stringified type annotations [#strann_less_common]_ (like ``'list[str]'``)
   must be *parsed* (to something like ``typing.List[str]``) to be introspected.

   -  In practice it is extremely difficult for stringified type
      annotations to be handled reliably at runtime, so runtime type
      checkers may opt to not support them at all.

-  Resolving string-based forward references inside type
   expressions to actual values must typically be done using ``eval()``,
   which is difficult/impossible to use in a safe way.
-  Recursive types like ``IntTree = list[typing.Union[int, 'IntTree']]``
   are not possible to fully resolve.
-  Supporting user-defined generic types (like Django’s
   ``QuerySet[User]``) requires user-defined functions to
   recognize/parse, which a runtime type checker should provide a
   registration API for.


Reference Implementation
========================

The following will be true when
`mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:

    The mypy type checker supports ``TypeExpr`` types.

A reference implementation of the runtime component is provided in the
``typing_extensions`` module.


Rejected Ideas
==============

.. _widen_type_C_to_support_all_type_expressions:

Widen type[C] to support all type expressions
---------------------------------------------

``type`` was `designed`_ to only be used to describe class objects. A
class object can always be used as the second argument of ``isinstance()``
and can usually be instantiated by calling it.

``TypeExpr`` on the other hand is typically introspected by the user in
some way, is not necessarily directly instantiable, and is not
necessarily directly usable in a regular ``isinstance()`` check.

It would be possible to widen ``type`` to include the additional values
allowed by ``TypeExpr`` but it would reduce clarity about the user’s
intentions when working with a ``type``. Different concepts and usage
patterns; different spellings.

.. _designed: https://mail.python.org/archives/list/typing-sig@python.org/message/D5FHORQVPHX3BHUDGF3A3TBZURBXLPHD/


.. _accept_arbitrary_annotation_expressions:

Accept arbitrary annotation expressions
---------------------------------------

Certain typing special forms can be used in *some* but not *all*
annotation contexts:

For example ``Final[]`` can be used as a variable type but not as a
parameter type or a return type:

::

   some_const: Final[str] = ...  # OK

   def foo(not_reassignable: Final[object]): ...  # ERROR: Final[] not allowed here

   def nonsense() -> Final[object]: ...  # ERROR: Final[] not meaningful here

``TypeExpr[T]`` does not allow matching such annotation expressions
because it is not clear what it would mean for such an expression
to parameterized by a type variable in position ``T``:

::

   def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: ...

   def foo(some_arg):
       if ismatch(some_arg, Final[int]):  # ERROR: Final[int] is not a TypeExpr
           reveal_type(some_arg)  # ? NOT Final[int], because invalid for a parameter

Functions that wish to operate on *all* kinds of annotation expressions,
including those that are not ``TypeExpr``\ s, can continue to accept such
inputs as ``object`` parameters, as they must do so today.


Accept only universal type expressions
--------------------------------------

Earlier drafts of this PEP only allowed ``TypeExpr[]`` to match the subset
of type expressions which are valid in *all* contexts, excluding
:ref:`non-universal type expressions <non_universal_typeexpr>`.
However doing that would effectively
create a new subset of annotation expressions that Python typing users
would have to understand, on top of all the existing distinctions between
“class objects”, “type expressions”, and “annotation expressions”.

To avoid introducing yet another concept that everyone has to learn,
this proposal just rounds ``TypeExpr[]`` to exactly match the existing
definition of a “type expression”.


Support pattern matching on type expressions
--------------------------------------------

It was asserted that some functions may wish to pattern match on the
interior of type expressions in their signatures.

One use case is to allow a function to explicitly enumerate all the
*specific* kinds of type expressions it supports as input.
Consider the following possible pattern matching syntax:

::

   @overload
   def checkcast(typx: TypeExpr[AT=Annotated[T, *Anns]], value: str) -> T: ...
   @overload
   def checkcast(typx: TypeExpr[UT=Union[*Ts]], value: str) -> Union[*Ts]: ...
   @overload
   def checkcast(typx: type[C], value: str) -> C: ...
   # ... (more)

All functions observed in the wild that conceptually take a ``TypeExpr[]``
generally try to support *all* kinds of type expressions, so it doesn’t
seem valuable to enumerate a particular subset.

Additionally the above syntax isn’t precise enough to fully describe the
actual input constraints for a typical function in the wild. For example
many functions recognize un-stringified type expressions like
``list[Movie]`` but may not recognize type expressions with stringified
subcomponents like ``list['Movie']``.

A second use case for pattern matching on the interior of type
expressions is to explicitly match an ``Annotated[]`` form to pull out the
interior type argument and strip away the metadata:

::

   def checkcast(
       typx: TypeExpr[T] | TypeExpr[AT=Annotated[T, *Anns]],
       value: object
   ) -> T:

However ``Annotated[T, metadata]`` is already treated equivalent to ``T`` anyway.
There’s no additional value in being explicit about this behavior.
The example above could be more-straightforwardly written as the equivalent:

::

   def checkcast(typx: TypeExpr[T], value: object) -> T:


Footnotes
=========

.. [#type_c]
   :pep:`Type[C] spells a class object <484#the-type-of-class-objects>`

.. [#TypeIsPep]
   :pep:`TypeIs[T] is similar to bool <742>`

.. [#DataclassInitVar]
   ``dataclass.make_dataclass`` accepts ``InitVar[...]`` as a special case
   in addition to type expressions. Therefore it may unfortunately be necessary
   to continue annotating its ``type`` parameter as ``object`` rather
   than ``TypeExpr``.

.. [#forward_ref_normalization]
   Special forms normalize string arguments to ``ForwardRef`` instances
   at runtime using internal helper functions in the ``typing`` module.
   Runtime type checkers may wish to implement similar functions when
   working with string-based forward references.

.. [#strann_less_common]
   Stringified type annotations are expected to become less common
   starting in Python 3.14 when :pep:`deferred annotations <649>`
   become available. However there is a large amount of existing code from
   earlier Python versions relying on stringified type annotations that will
   still need to be supported for several years.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.