python-peps/pep-0695.rst

PEP: 695
Title: Type Parameter Syntax
Author: Eric Traut <erictr at microsoft.com>
Sponsor: Guido van Rossum <guido@python.org>
Discussions-To: typing-sig@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 15-Jun-2022
Python-Version: 3.12


Abstract
========

This PEP specifies an improved syntax for specifying type parameters within
a generic class, function, or type alias. It also introduces a new statement
for declaring type aliases.


Motivation
==========

:pep:`484` introduced type variables into the language. :pep:`612` built
upon this concept by introducing parameter specifications, and
:pep:`646` added variadic type variables.

While generic types and type parameters have grown in popularity, the
syntax for specifying type parameters still feels "bolted on" to Python.
This is a source of confusion among Python developers.

There is consensus within the Python static typing community that it is time
to provide a formal syntax and bring Python in alignment with other modern
programming languages that support generic types.

An analysis of 25 popular typed Python libraries revealed that type
variables (in particular, the ``typing.TypeVar`` symbol) were used in
14% of modules. This percentage is likely to increase if type parameters
become less cumbersome to use.


Points of Confusion
-------------------

While the use of type variables has become widespread, the manner in which
they are specified within code is the source of confusion among many
Python developers. There are a couple of factors that contribute to this
confusion.

The scoping rules for type variables are difficult to understand. Type
variables are typically allocated within the global scope, but their semantic
meaning is valid only when used within the context of a generic class,
function or or type alias. A single runtime instance of a type variable may be
reused in multiple generic contexts, and it has a different semantic meaning
in each of these contexts. This PEP proposes to eliminate this source of
confusion by declaring type parameters at a natural place within a class,
function or type alias declaration statement.

Generic type aliases are often misused because it is not clear to developers
that a type argument must be supplied when the type alias is used. This leads
to an implied type argument of ``Any``, which is rarely the intent. This PEP
proposes to add new syntax that makes generic type alias declarations
clear.

:pep:`483` and :pep:`484` introduced the concept of "variance" for a type
variable used within a generic class. Type variables can be invariant,
covariant, or contravariant. The concept of variance is an advanced detail
of type theory that is not well understood by most Python developers, yet
they are immediately confronted with the concept today when defining their
first generic class. This PEP largely eliminates the need for most developers
to understand the concept of variance when defining generic classes.

When more than one type parameter is used with a generic class or type alias,
the rules for type parameter ordering can be confusing. It is normally based on
the order in which they first appear within a class or type alias declaration
statement. However, this can be overridden in a class definition by
including a "Generic" or "Protocol" base class. For example, in the class
declaration ``class ClassA(Mapping[K, V])``, the type parameters are
ordered as ``K`` and then ``V``. However, in the class declaration
``class ClassB(Mapping[K, V], Generic[V, K])``, the type parameters are
ordered as ``V`` and then ``K``. This PEP proposes to make type parameter
ordering explicit in all cases.

The practice of sharing a type variable across multiple generic contexts
creates other problems today. Modern editors provide features like "find
all references" and "rename all references" that operate on symbols at the
semantic level. When a type parameter is shared among multiple generic
classes, functions, and type aliases, all references are semantically
equivalent.

Type variables defined within the global scope also need to be given a name
that starts with an underscore to indicate that the variable is private to
the module. Globally-defined type variables are also often given names to
indicate their variance, leading to cumbersome names like "_T_contra" and
"_KT_co". The current mechanisms for allocating type variables also requires
the developer to supply a redundant name in quotes (e.g. ``T = TypeVar("T")``).
This PEP eliminates the need for the redundant name and cumbersome
variable names.

Defining type parameters today requires importing the ``TypeVar`` and 
``Generic`` symbols from the ``typing`` module. Over the past several releases
of Python, efforts have been made to eliminate the need to import ``typing``
symbols for common use cases, and the PEP furthers this goal.


Summary Examples
================

Defining a generic class prior to this PEP looks something like this.

::

    from typing import Generic, TypeVar

    _T_co = TypeVar("_T_co", covariant=True, bound=str)

    class ClassA(Generic[_T_co]):
        ...


With the new syntax, it looks like this.

::

    class ClassA[T: str]:
        ...


Here is an example of a generic function today.

::

    from typing import ParamSpec, TypeVar, Callable

    _P = ParamSpec("_P")
    _R = TypeVar("_R")

    def func(cb: Callable[_P, _R], *args: _P.args, **kwargs: _P.kwargs) -> _R:
        ...

And the new syntax.

::

    from typing import Callable

    def func[**P, R](cb: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> R:
        ...


Here is an example of a generic type alias today.

::

    from typing import TypeAlias

    _T = TypeVar("_T")

    ListOrSet: TypeAlias = list[_T] | set[_T]


And with the new syntax.

::

    type ListOrSet[T] = list[T] | set[T]


Specification
=============

Type Parameter Declarations
---------------------------

We propose to add new syntax for declaring type parameters for generic
classes, functions, and type aliases. The syntax adds support for
a comma-delimited list of type parameters in square brackets after
the name of the class, function, or type alias.

Simple (non-variadic) type variables are declared with an unadorned name.
Variadic type variables are preceded by ``*``. Parameter specifications are
preceded by ``**``.

::

    # This generic class is parameterized by a TypeVar T, a
    # TypeVarTuple Ts, and a ParamSpec P.
    class ChildClass[T, *Ts, **P]: ...

There is no need to include ``Generic`` as a base class. Its inclusion as
a base class is implied by the presence of type parameters, and it will
automatically be included in the ``__mro__`` and ``__orig_bases`` attributes
for the class. The explicit use of a ``Generic`` base class will result in a
runtime error.

::

    class ClassA[T](Generic[T]): ...  # Runtime error


A ``Protocol`` base class with type arguments will not generate a runtime
error, but type checkers should generate an error in this case because
the use of type arguments is not needed, and the order of type parameters
for the class are no longer dictated by their order in the ``Protocol``
base class.

::

    class ClassA[S, T](Protocol): ... # OK

    class ClassB[S, T](Protocol[S, T]): ... # Recommended type checker error


Type parameter names within a generic class, function, or type alias must
be unique. Type parameters for a generic function cannot overlap the name
of a function parameter. A duplicate name generates a syntax error at compile
time.

::

    class ClassA[T, *T]: ... # Syntax Error

    def func1[T, **T](): ... # Syntax Error

    def func2[T](T): ... # Syntax Error


Class type parameter names are not mangled if they begin with a double
underscore. Mangling would not make sense because type parameters, unlike other
class-scoped variables, cannot be accessed through the class dictionary, and
the notion of a "private" type parameter doesn't make sense. Other class-scoped
variables are mangled if they begin with a double underscore, so the mangled
name is used to determine whether there is a name collision with type parameters.

::

    class ClassA[__T, _ClassA__S]:
        __T = 0  # OK
        __S = 0  # Syntax Error (because mangled name is _ClassA__S)


Type Parameter Scopes
---------------------

A type parameter declared as part of a generic class is valid within the
class body and inner scopes contained therein. Type parameters are also
accessible when evaluating the argument list (base classes and any keyword
arguments) that comprise the class definition. This allows base classes
to be parameterized by these type parameters. Type parameters are not
accessible outside of the class body, including in any class decorators.

::

    class ClassA[T](BaseClass[T], param = Foo[T]): ...  # OK

    print(T)  # Runtime error: 'T' is not defined

    @dec(Foo[T])  # Runtime error: 'T' is not defined
    class ClassA[T]: ...

A type parameter declared as part of a generic function is valid within
the function body and any scopes contained therein. It is also valid within
parameter and return type annotations. Default argument values for function
parameters are evaluated outside of this scope, so type parameters are
not accessible in default value expressions. Likewise, type parameters are not
in scope for function decorators.

::

    def func1[T](a: T) -> T: ...  # OK

    print(T)  # Runtime error: 'T' is not defined

    def func2[T](a = list[T]): ...  # Runtime error: 'T' is not defined

    @dec(list[T])  # Runtime error: 'T' is not defined
    def func3[T](): ...


Upper Bound Specification
-------------------------

For a non-variadic type parameter, an "upper bound" type can be specified
through the use of a type annotation expression. If an upper bound is
not specified, the upper bound is assumed to be ``object``.

::

    class ClassA[T: str]: ...

The specified upper bound type must use an expression form that is allowed in
type annotations. More complex expression forms should be flagged
as an error by a type checker. Quoted forward declarations are allowed.

The specified upper bound type must be concrete. An attempt to use a generic
type should be flagged as an error by a type checker.

::

    class ClassA[T: dict[str, int]]: ...  # OK

    class ClassB[T: "ForwardDeclaration"]: ...  # OK

    class ClassC[T: dict[str, V]]: ...  # Type checker error: generic type

    class ClassD[T: [str, int]]: ...  # Type checker error: illegal expression form


Constrained Type Specification
------------------------------

For a non-variadic type parameter, a set of two or more "constrained types"
can be specified through the use of a literal tuple expression that contains
two or more types.

::

    class ClassA[AnyStr: (str, bytes)]: ...  # OK

    class ClassB[T: ("ForwardDeclaration", bytes)]: ...  # OK

    class ClassC[T: ()]: ...  # Type checker error: two or more types required

    class ClassD[T: (str, )]: ...  # Type checker error: two or more types required

    t1 = (bytes, str)
    class ClassE[T: t1]: ...  # Type checker error: literal tuple expression required


If the specified type is not a tuple expression or the tuple expression includes
complex expression forms that are not allowed in a type annotation, a type
checker should generate an error. Quoted forward declarations are allowed.

::

    class ClassF[T: (3, bytes)]: ...  # Type checker error: invalid expression form


The specified constrained types must be concrete. An attempt to use a generic
type should be flagged as an error by a type checker. This is consistent with
the existing rules enforced by type checkers for a ``TypeVar`` constructor call.

::

    class ClassG[T: (list[S], str)]: ...  # Type checker error: generic type



Generic Type Alias
------------------

We propose to introduce a new statement for declaring type aliases. Similar
to ``class`` and ``def`` statements, a ``type`` statement defines a scope
for type parameters. 

::

    # A non-generic type alias
    type IntOrStr = int | str

    # A generic type alias
    type ListOrSet[T] = list[T] | set[T]


Type aliases can refer to themselves without the use of quotes.

::

    # A type alias that refers to a forward-declared type
    type AnimalOrVegetable = Animal | "Vegetable"

    # A generic self-referential type alias
    type RecursiveList[T] = T | list[RecursiveList[T]]


The ``type`` keyword is a new soft keyword. It is interpreted as a keyword
only in this part of the grammar. In all other locations, it is assumed to
be an identifier name.

Type parameters declared as part of a generic type alias are valid only
when evaluating the right-hand side of the type alias.

As with ``typing.TypeAlias``, type checkers should restrict the right-hand
expression to expression forms that are allowed within type annotations.
The use of more complex expression forms (call expressions, ternary operators,
arithmetic operators, comparison operators, etc.) should be flagged as an
error.

Type alias expressions are not allowed to use traditional type variables.
Type checkers should generate an error in this case.

::

    T = TypeVar("T")
    type MyList = list[T]  # Type checker error: traditional type variable usage


We propose to deprecate the existing ``typing.TypeAlias`` introduced in
:pep:`613`. The new syntax eliminates its need entirely.


Runtime Type Alias Class
------------------------

At runtime, a ``type`` statement will generate an instance of
``typing.TypeAliasType``. This class represents the type. Its attributes
include:

* ``__name__`` is a str representing the name of the type alias
* ``__parameters__`` is a tuple of ``TypeVar``, ``TypeVarTuple``, or
    ``ParamSpec`` objects that parameterize the type alias if it is generic
* ``__value__`` is the evaluated value of the type alias

The ``__value__`` attribute initially has a value of ``None`` while the type
alias expression is evaluated. It is then updated after a successful evaluation.
This allows for self-referential type aliases.


Variance Inference
------------------

We propose to eliminate the need for variance to be specified for type
parameters. Instead, type checkers will infer the variance of type parameters
based on their usage within a class. Type parameters can be invariant,
covariant, or contravariant depending on how they are used.

Python type checkers already include the ability to determine the variance of
type parameters for the purpose of validating variance within a generic
protocol class. This capability can be used for all classes (whether or not
they are protocols) to calculate the variance of each type parameter. This
eliminates the need for most developers to understand the concept of variance.
It also eliminates the need to introduce a dedicated syntax for specifying
variance.

The algorithm for computing the variance of a type parameter is as follows.

For each type parameter in a generic class:

1. If the type parameter is variadic (``TypeVarTuple``) or a parameter
specification (``ParamSpec``), it is always considered invariant. No further
inference is needed.

2. If the type parameter comes from a traditional ``TypeVar`` declaration and
is not specified as ``autovariance`` (see below), its variance is specified
by the ``TypeVar`` constructor call. No further inference is needed.

3. Create two specialized versions of the class. We'll refer to these as
``upper`` and ``lower`` specializations. In both of these specializations,
replace all type parameters other than the one being inferred by a dummy type
instance. In the ``upper`` specialized class, specialize the target type
parameter with an ``object`` instance. In the ``lower`` specialized class,
specialize the target type parameter with itself. This specialization
ignores the type parameter's upper bound or constraints.

4. Determine whether ``lower`` can be assigned to ``upper`` using normal type
compatibility rules. If so, the target type parameter is covariant. If not,
determine whether ``upper`` can be assigned to ``lower``. If so, the target
type parameter is contravariant. If neither of these combinations are
assignable, the target type parameter is invariant.

Here is an example.

::

    class ClassA[T1, T2, T3](list[T1]):
        def method1(self, a: T2) -> None:
            ...
        
        def method2(self) -> T3:
            ...

To determine the variance of ``T1``, we specialize ``ClassA`` as follows:

::

    upper = ClassA[object, Dummy, Dummy]
    lower = ClassA[T1, Dummy, Dummy]

We find that ``upper`` is not assignable to ``lower`` nor is ``lower``
assignable to ``upper`` using standard type compatibility checks, so we
can conclude that ``T1`` is invariant.

To determine the variance of ``T2``, we specialize ``ClassA`` as follows:

::

    upper = ClassA[Dummy, object, Dummy]
    lower = ClassA[Dummy, T2, Dummy]

Since ``upper`` is assignable to ``lower``, ``T2`` is contravariant.

To determine the variance of ``T3``, we specialize ``ClassA`` as follows:

::

    upper = ClassA[Dummy, Dummy, object]
    lower = ClassA[Dummy, Dummy, T3]

Since ``lower`` is assignable to ``upper``, ``T3`` is covariant.


Auto Variance For TypeVar
-------------------------

The existing ``TypeVar`` class constructor accepts keyword parameters named
``covariant`` and ``contravariant``. If both of these are ``False``, the
type variable is assumed to be invariant. We propose to add another keyword
parameter named ``autovariance``. A corresponding instance variable
``__autovariance__`` can be accessed at runtime to determine whether the
variance is inferred. Type variables that are implicitly allocated using the
new syntax will always have ``__autovariance__`` set to ``True``.

A generic class that uses the traditional syntax may include combinations of
type variables with explicit and inferred variance.

::

    T1 = TypeVar("T1", autovariance=True)  # Inferred variance
    T2 = TypeVar("T2")  # Invariant
    T3 = TypeVar("T3", covariant=True)  # Covariant

    # A type checker should infer the variance for T1 but use the
    # specified variance for T2 and T3.
    class ClassA(Generic[T1, T2, T3]): ...


Compatibility with Traditional TypeVars
---------------------------------------

The existing mechanism for allocating ``TypeVar``, ``TypeVarTuple``, and
``ParamSpec`` is retained for backward compatibility. However, these
"traditional" type variables should not be combined with type parameters
allocated using the new syntax. Such a combination should be flagged as
an error by type checkers. This is necessary because the type parameter
order is ambiguous.

It is OK to combine traditional type variables with new-style type parameters
if the class, function, or type alias does not use the new syntax. The
new-style type parameters must come from an outer scope in this case.

::

    K = TypeVar("K")

    class ClassA[V](dict[K, V]): ...  # Type checker error

    class ClassB[K, V](dict[K, V]): ...  # OK

    class ClassC[V]:
        # The use of K and V for "method1" is OK because it uses the
        # "traditional" generic function mechanism where type parameters
        # are implicit. In this case V comes from an outer scope (ClassC)
        # and K is introduced implicitly as a type parameter for "method1".
        def method1(self, a: V, b: K) -> V | K: ...

        # The use of M and K are not allowed for "method2". A type checker
        # should generate an error in this case because this method uses the
        # new syntax for type parameters, and all type parameters associated
        # with the method must be explicitly declared. In this case, ``K``
        # is not declared by "method2", nor is it supplied defined an outer
        # scope.
        def method2[M](self, a: M, b: K) -> M | K: ...


Runtime Implementation
======================

Grammar Changes
---------------

This PEP introduces a new soft keyword ``type``. It modifies the grammar
in the following ways:

1. Addition of optional type parameter clause in ``class`` and ``def`` statements.

::
    
    type_params: '[' t=type_param_seq  ']'

    type_param_seq: a[asdl_typeparam_seq*]=','.type_param+ [',']

    type_param:
        | a=NAME b=[type_param_bound]
        | '*' a=NAME
        | '**' a=NAME

    type_param_bound: ":" e=expression


2. Addition of new ``type`` statement for defining type aliases.

::

    type_alias[stmt_ty]:
        | "type" n=NAME t=[type_params] '=' b=expression {
            CHECK_VERSION(stmt_ty, 12, "Type statement is", _PyAST_TypeAlias(n->v.Name.id, t, b, EXTRA)) }


AST Changes
-----------

This PEP introduces a new AST node type called ``TypeAlias``.

::

    TypeAlias(identifier name, typeparam* typeparams, expr value)

It also adds an AST node that represents a type parameter.

::

    typeparam = TypeVar(identifier name, expr? bound)
        | ParamSpec(identifier name)
        | TypeVarTuple(identifier name)

It also modifies existing AST nodes ``FunctionDef``, ``AsyncFunctionDef`` and
``ClassDef`` to include an additional optional attribute called ``typeparam*``
that includes a list of type parameters associated with the function or class.


Compiler Changes
----------------

The compiler maintains a list of "active type variables" as it recursively
generates byte codes for the program. Consider the following example.

::

    class Outer[K, V]:
        # Active type variables are K and V

        class Inner[T]:
            # Active type variables are K, V, and T

            def method[M](self, a: M) -> M:
                # Active type variables are K, V, T, and M
                ...

An active type variable symbol cannot be used for other purposes within
these scopes. This includes local parameters, local variables, variables
bound from other scopes (nonlocal or global), or other type parameters. An
attempt to reuse a type variable name in one of these manners results in
a syntax error.

::

    class ClassA[K, V]:
        class Inner[K]: # Syntax error: K already in use as type variable
            ...

    class ClassB[K, V]:
        def method(self, K): # Syntax error: K already in use as type variable
            ...

    class ClassC[T, T]: # Syntax error: T already in use as type variable
        ...

    def func1[T]():
        ...


A type variable is considered "active" when compiling the arguments for
a class declaration, the type annotations for a function declaration, and
the right-hand expression in a type alias declaration. Type variable are
not considered "active" when compiling the default argument expressions for
a function declaration or decorator expressions for classes or functions.

::

    T = list

    @decorator(T) # T in decorator refers to outer variable
    class ClassA[T](Base[T], metaclass=Meta[T]) # T refers to type variable
        ...

    @decorator(T) # T in decorator refers to outer variable
    def func1[T](a: list[T]) -> T: # T refers to type variable
        ...

    def func2[T](a = T): # T in default refers to outer variable
        ...


When a ``class``, ``def``, or ``type`` statement includes one or more type
parameters, the compiler emits byte codes to construct the corresponding
``typing.TypeVar``, ``typing.TypeVarTuple``, or ``typing.ParamSpec`` instances.
It then builds a new tuple that includes all active type variables and stores
this new tuple in a local variable. Active type variables include all type
parameters declared by outer ``class`` and ``def`` scopes plus those declared
by the ``class``, ``def``, or ``type`` statement itself. (In the reference
implementation, the local variable happens to have the name
``__type_variables__``, but this is an implementation detail. Other Python
compilers or future versions of the CPython compiler may choose a different
name or an entirely anonymous local variable slot for this purpose.)

When a type variable is referenced, the compiler generates opcodes that
load the active type variable tuple from either the local variable or (if
there are no local type variables) through the use of a new opcode called
``LOAD_TYPEVARS`` that loads the tuple of active type variables from the
current function object. It then emits opcodes to index into this tuple to
fetch the desired type variable.

When a new function is created, the "active type variables" tuple is copied
to the C struct field ``func_typevars`` of the function object, making the type
variables from outer scopes available to inner scopes of the function or class.

A new read-only attribute called ``__type_variables__`` is available on class,
function, and type alias objects. This attribute is a tuple of the active
type variables that are visible within the scope of that class, function,
or type alias. This attribute is used for runtime evaluation of stringified
(forward referenced) type annotations that include references to type
parameters. Functions like ``typing.get_type_hints`` can use this attribute
to populate the ``locals`` dictionary with values for type parameters that
are in scope when calling ``eval`` to evaluate the stringified expression.


Library Changes
---------------

Several classes in the ``typing`` module that are currently implemented in
Python must be reimplemented in C. This includes: ``TypeVar``,
``TypeVarTuple``, ``ParamSpec``, and ``Generic``. The new class
``TypeAliasType`` (described above) also must be implemented in C.

The ``typing.get_type_hints`` must be updated to use the new
``__type_variables__`` attribute.


Reference Implementation
========================

This proposal is prototyped in the CPython code base in
`this fork <https://github.com/erictraut/cpython/tree/type_param_syntax2>`_.

The Pyright type checker supports the behavior described in this PEP.


Rejected Ideas
==============

Prefix Clause
-------------
We explored various syntactic options for specifying type parameters that
preceded ``def`` and ``class`` statements. One such variant we considered
used a ``using`` clause as follows:

::

    using S, T
    class ClassA: ...

This option was rejected because the scoping rules for the type parameters
were less clear. Also, this syntax did not interact well with class and
function decorators, which are common in Python. Only one other popular
programming language, C++, uses this approach.

We likewise considered prefix forms that looked like decorators (e.g.,
``@using(S, T)``). This idea was rejected because such forms would be confused
with regular decorators, and they would not compose well with existing
decorators. Furthermore, decorators are logically executed after the statement
they are decorating, so it would be confusing for them to introduce symbols
(type parameters) that are visible within the "decorated" statement, which is
logically executed before the decorator itself.


Angle Brackets
--------------
Many languages that support generics make use of angle brackets. (Refer to
the table at the end of Appendix A for a summary.) We explored the use of
angle brackets for type parameter declarations in Python, but we ultimately
rejected it for two reasons. First, angle brackets are not considered
"paired" by the Python scanner, so end-of-line characters between a ``<``
and ``>`` token are retained. That means any line breaks within a list of
type parameters would require the use of unsightly and cumbersome ``\`` escape
sequences. Second, Python has already established the use of square brackets
for explicit specialization of a generic type (e.g., ``list[int]``). We
concluded that it would be inconsistent and confusing to use angle brackets
for generic declarations but square brackets for explicit specialization. All
other languages that we surveyed were consistent in this regard.


Bounds Syntax
-------------
We explored various syntactic options for specifying the bounds and constraints
for a type variable. We considered, but ultimately rejected, the use
of a ``<:`` token like in Scala, the use of an ``extends`` or ``with``
keyword like in various other languages, and the use of a function call
syntax similar to today's ``typing.TypeVar`` constructor. The simple colon
syntax is consistent with many other programming languages (see Appendix A),
and it was heavily preferred by a cross section of Python developers who
were surveyed.


Explicit Variance
-----------------
We considered adding syntax for specifying whether a type parameter is intended
to be invariant, covariant, or contravariant. The ``typing.TypeVar`` mechanism
in Python requires this. A few other languages including Scala and C#
also require developers to specify the variance. We rejected this idea because
variance can generally be inferred, and most modern programming languages
do infer variance based on usage. Variance is an advanced topic that
many developers find confusing, so we want to eliminate the need to 
understand this concept for most Python developers.


Name Mangling
-------------
When considering implementation options, we considered a "name mangling"
approach where each type parameter was given a unique "mangled" name by the
compiler. This mangled name would be based on the qualified name of the
generic class, function or type alias it was associated with. This approach
was rejected because qualified names are not necessarily unique, which means
the mangled name would need to be based on some other randomized value.
Furthermore, this approach is not compatible with techniques used for
evaluating quoted (forward referenced) type annotations.


Lambda Lifting
--------------
When considering implementation options, we considered introducing a new
scope and executing the ``class``, ``def``, or ``type`` statement within
a lambda -- a technique that is sometimes referred to as "lambda lifting".
We ultimately rejected this idea because it did not work well for statements
within a class body (because class-scoped symbols cannot be accessed by
inner scopes). It also introduced many odd behaviors for scopes that were
further nested within the lambda.


Appendix A: Survey of Type Parameter Syntax
===========================================

Support for generic types is found in many programming languages. In this
section, we provide a survey of the options used by other popular programming
languages. This is relevant because familiarity with other languages will
make it easier for Python developers to understand this concept. We provide
additional details here (for example, default type argument support) that
may be useful when considering future extensions to the Python type system.


C++
---

C++ uses angle brackets in combination with keywords ``template`` and
``typename`` to declare type parameters. It uses angle brackets for
specialization.

C++20 introduced the notion of generalized constraints, which can act
like protocols in Python. A collection of constraints can be defined in
a named entity called a ``concept``.

Variance is not explicitly specified, but constraints can enforce variance.

A default type argument can be specified using the ``=`` operator.

.. code-block:: c++

    // Generic class
    template <typename>
    class ClassA
    {
        // Constraints are supported through compile-time assertions.
        static_assert(std::is_base_of<BaseClass, T>::value);

    public:
        Container<T> t;
    };

    // Generic function with default type argument
    template <typename S = int>
    S func1(ClassA<S> a, S b) {};

    // C++20 introduced a more generalized notion of "constraints"
    // and "concepts", which are named constraints.

    // A sample concept
    template<typename T>
    concept Hashable = requires(T a)
    {
        { std::hash<T>{}(a) } -> std::convertible_to<std::size_t>;
    };

    // Use of a concept in a template
    template<Hashable T>
    void func2(T value) {}

    // Alternative use of concept
    template<typename T> requires Hashable<T>
    void func3(T value) {}

    // Alternative use of concept
    template<typename T>
    void func3(T value) requires Hashable<T> {}


Java
----

Java uses angle brackets to declare type parameters and for specialization.
By default, type parameters are invariant.
The ``extends`` keyword is used to specify an upper bound. The ``super`` keyword
is used to specify a contravariant bound.

Java uses use-site variance. The compiler places limits on which methods and
members can be accessed based on the use of a generic type. Variance is
not specified explicitly.

Java provides no way to specify a default type argument.

.. code-block:: java

    // Generic class
    public class ClassA<T> {
        public Container<T> t;

        // Generic method
        public <S extends Number> void method1(S value) { }

        // Use site variance
        public void method1(ClassA<? super Integer> value) { }
    }
    

C#
--

C# uses angle brackets to declare type parameters and for specialization.
The ``where`` keyword and a colon is used to specify the bound for a type
parameter.

C# uses declaration-site variance using the keywords ``in`` and ``out`` for
contravariance and covariance, respectively. By default, type parameters are
invariant.

C# provides no way to specify a default type argument.

.. code-block:: c#

    // Generic class with bounds on type parameters
    public class ClassA<S, T>
        where T : SomeClass1
        where S : SomeClass2
    {
        // Generic method
        public void MyMethod<U>(U value) where U : SomeClass3 { }
    }

    // Contravariant and covariant type parameters
    public class ClassB<in S, out T>
    {
        public T MyMethod(S value) { }
    }



TypeScript
----------

TypeScript uses angle brackets to declare type parameters and for 
specialization. The ``extends`` keyword is used to specify a bound. It can be
combined with other type operators such as ``keyof``.

TypeScript uses declaration-site variance. Variance is inferred from
usage, not specified explicitly. TypeScript 4.7 introduced the ability
to specify variance using ``in`` and ``out`` keywords. This was added to handle
extremely complex types where inference of variance was expensive,
yet the maintainers state that is useful for increasing readability.

A default type argument can be specified using the ``=`` operator.

TypeScript supports the ``type`` keyword to declare a type alias, and this
syntax supports generics.

.. code-block:: typescript
    
    // Generic interface
    interface InterfaceA<S, T extends SomeInterface1> {
        val1: S;
        val2: T;

        method1<U extends SomeInterface2>(val: U): S
    }

    // Generic function
    function func1<T, K extends keyof T>(ojb: T, key: K) { }

    // Contravariant and covariant type parameters (TypeScript 4.7)
    interface InterfaceB<in S, out T> { }

    // Type parameter with default
    interface InterfaceC<T = SomeInterface3> { }

    // Generic type alias
    type MyType<T extends SomeInterface4> = Array<T>


Scala
-----

In Scala, square brackets are used to declare type parameters. Square
brackets are also used for specialization. The ``<:`` and ``>:`` operators
are used to specify upper and lower bounds, respectively.

Scala uses use-site variance but also allows declaration-site variance
specification. It uses a ``+`` or ``-`` prefix operator for covariance and
contravariance, respectively.

Scala provides no way to specify a default type argument.

It does support higher-kinded types (type parameters that accept type
type parameters).

.. code-block:: scala

    
    // Generic class; type parameter has upper bound
    class ClassA[A <: SomeClass1]
    {
        // Generic method; type parameter has lower bound
        def method1[B >: A](val: B) ...
    }

    // Use of an upper and lower bound with the same type parameter
    class ClassB[A >: SomeClass1 <: SomeClass2] { }

    // Contravariant and covariant type parameters
    class ClassC[+A, -B] { }

    // Higher-kinded type
    trait Collection[T[_]]
    {
        def method1[A](a: A): T[A]
        def method2[B](b: T[B]): B
    }

    // Generic type alias
    type MyType[T <: Int] = Container[T]


Swift
-----

Swift uses angle brackets to declare type parameters and for specialization.
The upper bound of a type parameter is specified using a colon.

Swift doesn't support generic variance; all type parameters are invariant.

Swift provides no way to specify a default type argument.

.. code-block:: swift

    // Generic class
    class ClassA<T> {
        // Generic method
        func method1<X>(val: T) -> X { }
    }

    // Type parameter with upper bound constraint
    class ClassB<T: SomeClass1> {}

    // Generic type alias
    typealias MyType<A> = Container<A>


Rust
----

Rust uses angle brackets to declare type parameters and for specialization.
The upper bound of a type parameter is specified using a colon. Alternatively
a ``where`` clause can specify various constraints.

Rust does not have traditional object oriented inheritance or variance.
Subtyping in Rust is very restricted and occurs only due to variance with
respect to lifetimes.

A default type argument can be specified using the ``=`` operator.

.. code-block:: rust

    // Generic class
    struct StructA<T> { // T's lifetime is inferred as covariant
        x: T
    }

    fn f<'a>(
        mut short_lifetime: StructA<&'a i32>,
        mut long_lifetime: StructA<&'static i32>,
    ) {
        long_lifetime = short_lifetime;
        // error: StructA<&'a i32> is not a subtype of StructA<&'static i32>
        short_lifetime = long_lifetime;
        // valid: StructA<&'static i32> is a subtype of StructA<&'a i32>
    }

    // Type parameter with bound
    struct StructB<T: SomeTrait> {}

    // Type parameter with additional constraints
    struct StructC<T>
    where
        T: Iterator,
        T::Item: Copy
    {}

    // Generic function
    fn func1<T>(val: &[T]) -> T { }

    // Generic type alias
    type MyType<T> = StructC<T>;


Kotlin
------

Kotlin uses angle brackets to declare type parameters and for specialization.
By default, type parameters are invariant. The upper bound of a type is
specified using a colon.
Alternatively, a ``where`` clause can specify various constraints.

Kotlin supports declaration-site variance where variance of type parameters is
explicitly declared using ``in`` and ``out`` keywords. It also supports use-site
variance which limits which methods and members can be used.

Kotlin provides no way to specify a default type argument.

.. code-block:: kotlin

    // Generic class
    class ClassA<T>

    // Type parameter with upper bound
    class ClassB<T : SomeClass1>

    // Contravariant and covariant type parameters
    class ClassC<in S, out T>

    // Generic function
    fun <T> func1(): T {

        // Use site variance
        val covariantA: ClassA<out Number>
        val contravariantA: ClassA<in Number>
    }

    // Generic type alias
    typealias TypeAliasFoo<T> = ClassA<T>


Julia
-----

Julia uses curly braces to declare type parameters and for specialization.
The ``<:`` operator can be used within a ``where`` clause to declare
upper and lower bounds on a type.

.. code-block:: julia

    // Generic struct; type parameter with upper and lower bounds
    struct StructA{T} where Int <: T <: Number
        x::T
    end

    // Generic function
    function func1{T <: Real}(v::Container{T})

    // Alternate form of generic function
    function func2(v::Container{T} where T <: Real)

Dart
----

Dart uses angle brackets to declare type parameters and for specialization.
The upper bound of a type is specified using the ``extends`` keyword.
By default, type parameters are covariant.

Dart supports declaration-site variance, where variance of type parameters is
explicitly declared using ``in``, ``out`` and ``inout`` keywords.
It does not support use-site variance.

Dart provides no way to specify a default type argument.

.. code-block:: dart

    // Generic class
    class ClassA<T> { }

    // Type parameter with upper bound
    class ClassB<T extends SomeClass1> { }

    // Contravariant and covariant type parameters
    class ClassC<in S, out T> { }

    // Generic function
    T func1<T>() { }

    // Generic type alias
    typedef TypeDefFoo<T> = ClassA<T>;

Go
--

Go uses square brackets to declare type parameters and for specialization.
The upper bound of a type is specified after the name of the parameter, and
must always be specified. The keyword ``any`` is used for an unbound type parameter.

Go doesn't support variance; all type parameters are invariant.

Go provides no way to specify a default type argument.

Go does not support generic type aliases.

.. code-block:: go

    // Generic type without a bound
    type TypeA[T any] struct {
        t T
    }

    // Type parameter with upper bound
    type TypeB[T SomeType1] struct { }

    // Generic function
    func func1[T any]() { }


Summary
-------

+------------+----------+---------+--------+----------+-----------+-----------+
|            | Decl     | Upper   | Lower  | Default  | Variance  | Variance  |
|            | Syntax   | Bound   | Bound  | Value    | Site      |           |
+============+==========+=========+========+==========+===========+===========+
| C++        | template | n/a     | n/a    | =        | n/a       | n/a       |
|            | <>       |         |        |          |           |           |
+------------+----------+---------+--------+----------+-----------+-----------+
| Java       | <>       | extends |        |          | use       | super,    |
|            |          |         |        |          |           | extends   |
+------------+----------+---------+--------+----------+-----------+-----------+
| C#         | <>       | where   |        |          | decl      | in, out   |
+------------+----------+---------+--------+----------+-----------+-----------+
| TypeScript | <>       | extends |        | =        | decl      | inferred, |
|            |          |         |        |          |           | in, out   |
+------------+----------+---------+--------+----------+-----------+-----------+
| Scala      | []       | T <: X  | T >: X |          | use, decl | +, -      |
+------------+----------+---------+--------+----------+-----------+-----------+
| Swift      | <>       | T: X    |        |          | n/a       | n/a       |
+------------+----------+---------+--------+----------+-----------+-----------+
| Rust       | <>       | T: X,   |        | =        | n/a       | n/a       |
|            |          | where   |        |          |           |           |
+------------+----------+---------+--------+----------+-----------+-----------+
| Kotlin     | <>       | T: X,   |        |          | use, decl | in, out   |
|            |          | where   |        |          |           |           |
+------------+----------+---------+--------+----------+-----------+-----------+
| Julia      | {}       | T <: X  | X <: T |          | n/a       | n/a       |
+------------+----------+---------+--------+----------+-----------+-----------+
| Dart       | <>       | extends |        |          | decl      | in, out,  |
|            |          |         |        |          |           | inout     |
+------------+----------+---------+--------+----------+-----------+-----------+
| Go         | []       | T X     |        |          | n/a       | n/a       |
+------------+----------+---------+--------+----------+-----------+-----------+
| Python     | []       | T: X    |        |          | decl      | inferred  |
| (proposed) |          |         |        |          |           |           |
+------------+----------+---------+--------+----------+-----------+-----------+


Acknowledgements
================

Thanks to Sebastian Rittau for kick-starting the discussions that led to this
proposal, to Jukka Lehtosalo for proposing the syntax for type alias
statements and to Jelle Zijlstra, Daniel Moisset, and Guido van Rossum
for their valuable feedback and suggested improvements to the specification
and implementation.


Copyright
=========

This document is placed in the public domain or under the CC0-1.0-Universal
license, whichever is more permissive.