PEP 696: Improve section "Using another TypeVarLike as the default" (#2777)

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

pep-0696.rst

@@ -15,7 +15,8 @@ Abstract
 
 This PEP introduces the concept of type defaults for
 ``TypeVarLike``\ s (``TypeVar``, ``ParamSpec`` and ``TypeVarTuple``),
-which act as defaults for a type parameter when none is specified.
+which act as defaults for a type parameter when one is not specified or
+the constraint solver isn't able to solve a type parameter to anything.
 
 Default type argument support is available in some popular languages
 such as C++, TypeScript, and Rust. A survey of type parameter syntax in
@@ -27,7 +28,7 @@ and can be found in its
 Motivation
 ----------
 
-.. code:: py
+.. code-block:: py
 
    T = TypeVar("T", default=int)  # This means that if no type is specified T = int
 
@@ -42,7 +43,7 @@ One place this `regularly comes
 up <https://github.com/python/typing/issues/975>`__ is ``Generator``. I
 propose changing the *stub definition* to something like:
 
-.. code:: py
+.. code-block:: py
 
    YieldT = TypeVar("YieldT")
    SendT = TypeVar("SendT", default=None)
@@ -54,7 +55,7 @@ propose changing the *stub definition* to something like:
 
 This is also useful for a ``Generic`` that is commonly over one type.
 
-.. code:: py
+.. code-block:: py
 
    class Bot: ...
 
@@ -73,20 +74,20 @@ also helps non-typing users who rely on auto-complete to speed up their
 development.
 
 This design pattern is common in projects like:
- - `discord.py <https://github.com/Rapptz/discord.py>`__ - where the
+ - `discord.py <https://github.com/Rapptz/discord.py>`__ — where the
    example above was taken from.
- - `NumPy <https://github.com/numpy/numpy>`__ - the default for types
+ - `NumPy <https://github.com/numpy/numpy>`__ — the default for types
    like ``ndarray``'s ``dtype`` would be ``float64``. Currently it's
    ``Unknown`` or ``Any``.
- - `TensorFlow <https://github.com/tensorflow/tensorflow>`__ (this
+ - `TensorFlow <https://github.com/tensorflow/tensorflow>`__ — this
    could be used for Tensor similarly to ``numpy.ndarray`` and would be
-   useful to simplify the definition of ``Layer``).
+   useful to simplify the definition of ``Layer``.
 
 
 Specification
 -------------
 
-Default ordering and subscription rules
+Default Ordering and Subscription Rules
 '''''''''''''''''''''''''''''''''''''''
 
 The order for defaults should follow the standard function parameter
@@ -95,7 +96,7 @@ a ``default`` value. Doing so should ideally raise a ``TypeError`` in
 ``typing._GenericAlias``/``types.GenericAlias``, and a type checker
 should flag this an error.
 
-.. code:: py
+.. code-block:: py
 
    DefaultStrT = TypeVar("DefaultStrT", default=str)
    DefaultIntT = TypeVar("DefaultIntT", default=int)
@@ -141,7 +142,7 @@ future, this might be possible (see `Interaction with PEP
 ``TypeVar`` \ s but use a ``list`` or ``tuple`` of types or an ellipsis
 literal "``...``".
 
-.. code:: py
+.. code-block:: py
 
    DefaultP = ParamSpec("DefaultP", default=(str, int))
 
@@ -156,44 +157,124 @@ literal "``...``".
 ``TypeVarTuple`` defaults are defined using the same syntax as
 ``TypeVar`` \ s but use an unpacked tuple of types instead of a single type.
 
-.. code:: py
+.. code-block:: py
 
    DefaultTs = TypeVarTuple("DefaultTs", default=Unpack[tuple[str, int]])
 
-   class Foo(Generic[DefaultTs]): ...
+   class Foo(Generic[*DefaultTs]): ...
 
    reveal_type(Foo())             # type is Foo[str, int]
    reveal_type(Foo[int, bool]())  # type is Foo[int, bool]
 
-Using another ``TypeVarLike`` as the default
+Using Another ``TypeVarLike`` as ``default``
 ''''''''''''''''''''''''''''''''''''''''''''
 
-To use another ``TypeVarLike``\ s as the default they have to be of the
+This allows for a value to be used again when the constraints solver
-same type. When using another ``TypeVarLike`` (T1) as the default, the default
+fails to solve a constraint for a type, or the type parameter to a
-for the ``TypeVarLike`` (T2), T1 must be used before in the signature
+generic is missing but another type parameter is specified.
-of the class it appears in before T2. T2's bound must be a subtype of
+
-T1's bound.
+To use another ``TypeVarLike`` as a default the ``default`` and the
+``TypeVarLike`` must be the same type (a ``TypeVar``'s default must be
+a ``TypeVar``, etc.).
 
 `This could be used on builtins.slice <https://github.com/python/typing/issues/159>`__
 where the ``start`` parameter should default to ``int``, ``stop``
 default to the type of ``start`` and step default to ``int | None``.
 
-.. code:: py
+.. code-block:: py
 
    StartT = TypeVar("StartT", default=int)
    StopT = TypeVar("StopT", default=StartT)
    StepT = TypeVar("StepT", default=int | None)
 
-   class slice(Generic[StartT, StopT, StepT]): ...  # Valid
+   class slice(Generic[StartT, StopT, StepT]): ...
 
    reveal_type(slice())                        # type is slice[int, int, int | None]
    reveal_type(slice[str]())                   # type is slice[str, str, int | None]
-   reveal_type(slice[str, str, timedelta]())  # type is slice[str, str, timedelta]
+   reveal_type(slice[str, bool, timedelta]())  # type is slice[str, bool, timedelta]
 
-   StartT = TypeVar("StartT", default="StopT")
+When using a ``TypeVarLike`` as the default to another ``TypeVarLike``.
+Where ``T1`` is the default for ``T2`` the following rules apply.
+
+.. _scoping-rules:
+
+Scoping Rules
+~~~~~~~~~~~~~
+
+``T1`` must be used before ``T2`` in the parameter list of the generic,
+or be bound in an outer class or function scope.
+
+.. code-block:: py
+
+   DefaultT = TypeVar("DefaultT", default=T)
+
+   class Foo(Generic[T, DefaultT]): ...   # Valid
+   def bar(x: T, y: DefaultT): ...        # Valid
+   class Foo(Generic[T]):
+       class Bar(Generic[DefaultT]): ...  # Valid
+   def outer(x: T):
+       def inner(y: DefaultT): ...        # Valid
+
+   StartT = TypeVar("StartT", default="StopT")  # Swapped defaults around from previous example
    StopT = TypeVar("StopT", default=int)
    class slice(Generic[StartT, StopT, StepT]): ...
-                       ^^^^^^  # Invalid: ordering does not allow StopT to bound yet
+                     # ^^^^^^ Invalid: ordering does not allow StopT to be bound
+   def baz(x: DefaultT, y: T): ...
+            # ^^^^^^^^ Invalid: ordering does not allow DefaultT to be bound
+
+Bound Rules
+~~~~~~~~~~~
+
+``T2``'s bound must be a subtype of ``T1``'s bound.
+
+.. code-block:: py
+
+   T = TypeVar("T", bound=float)
+   TypeVar("Ok", default=T, bound=int)        # Valid
+   TypeVar("AlsoOk", default=T, bound=float)  # Valid
+   TypeVar("Invalid", default=T, bound=str)   # Invalid: str is not a subtype of float
+
+Constraint Rules
+~~~~~~~~~~~~~~~~
+
+The constraints of ``T2`` must be a superset of the constraints of ``T1``.
+
+.. code-block:: py
+
+   T1 = TypeVar("T1", bound=int)
+   TypeVar("Invalid", float, str, default=T1)         # Invalid: upper bound int is incompatible with constraints float or str
+
+   T1 = TypeVar("T1", int, str)
+   TypeVar("AlsoOk", int, str, bool, default=T1)      # Valid
+   TypeVar("AlsoInvalid", bool, complex, default=T1)  # Invalid: {bool, complex} is not a superset of {int, str}
+
+
+``TypeVarLike``\s as Parameters to Generics
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``TypeVarLike``\ s are valid as parameters to generics inside of a
+``default`` when the first parameter is in scope as determined by the
+:ref:`previous section <scoping-rules>`.
+
+.. code-block:: py
+
+   T = TypeVar("T")
+   ListDefaultT = TypeVar("ListDefaultT", default=list[T])
+
+   class Bar(Generic[T, ListDefaultT]):
+       def __init__(self, x: T, y: ListDefaultT): ...
+
+   reveal_type(Bar[int])             # type is Bar[int, list[int]]
+   reveal_type(Bar[int, list[str]])  # type is Bar[int, list[str]]
+   reveal_type(Bar[int, str])        # type is Bar[int, str]
+
+Specialisation Rules
+~~~~~~~~~~~~~~~~~~~~
+
+``TypeVarLike``\ s currently cannot be further subscripted. This might
+change if `Higher Kinded TypeVars <https://github.com/python/typing/issues/548>`__
+are implemented.
+
 
 ``Generic`` ``TypeAlias``\ es
 '''''''''''''''''''''''''''''
@@ -204,11 +285,11 @@ that hasn't been overridden it should be treated like it was
 substituted into the ``TypeAlias``. However, it can be specialised
 further down the line.
 
-.. code:: py
+.. code-block:: py
 
    class SomethingWithNoDefaults(Generic[T, T2]): ...
 
-   MyAlias: TypeAlias = SomethingWithNoDefaults[int, DefaultStrT]  # valid
+   MyAlias: TypeAlias = SomethingWithNoDefaults[int, DefaultStrT]  # Valid
    reveal_type(MyAlias())        # type is SomethingWithNoDefaults[int, str]
    reveal_type(MyAlias[bool]())  # type is SomethingWithNoDefaults[int, bool]
 
@@ -220,17 +301,18 @@ Subclassing
 Subclasses of ``Generic``\ s with ``TypeVarLike``\ s that have defaults
 behave similarly to ``Generic`` ``TypeAlias``\ es.
 
-.. code:: py
+.. code-block:: py
 
-   class SubclassMe(Generic[T, DefaultStrT]): ...
+   class SubclassMe(Generic[T, DefaultStrT]):
+       x: DefaultStrT
 
    class Bar(SubclassMe[int, DefaultStrT]): ...
    reveal_type(Bar())        # type is Bar[str]
    reveal_type(Bar[bool]())  # type is Bar[bool]
 
-   class Foo(SubclassMe[int]): ...
+   class Foo(SubclassMe[float]): ...
 
-   reveal_type(Foo())  # type is <subclass of SubclassMe[int, int]>
+   reveal_type(Foo().x)  # type is str
 
    Foo[str]  # Invalid: Foo cannot be further subscripted
 
@@ -239,14 +321,14 @@ behave similarly to ``Generic`` ``TypeAlias``\ es.
    class Spam(Baz): ...
    reveal_type(Spam())  # type is <subclass of Baz[int, str]>
 
-Using bound and default
+Using ``bound`` and ``default``
-'''''''''''''''''''''''
+'''''''''''''''''''''''''''''''
 
 If both ``bound`` and ``default`` are passed ``default`` must be a
 subtype of ``bound``. Otherwise the type checker should generate an
 error.
 
-.. code:: py
+.. code-block:: py
 
    TypeVar("Ok", bound=float, default=int)     # Valid
    TypeVar("Invalid", bound=str, default=int)  # Invalid: the bound and default are incompatible
@@ -258,7 +340,7 @@ For constrained ``TypeVar``\ s, the default needs to be one of the
 constraints. A type checker should generate an error even if it is a
 subtype of one of the constraints.
 
-.. code:: py
+.. code-block:: py
 
    TypeVar("Ok", float, str, default=float)     # Valid
    TypeVar("Invalid", float, str, default=int)  # Invalid: expected one of float or str got int
@@ -268,7 +350,7 @@ Function Defaults
 
 ``TypeVarLike``\ s currently can only be used where a parameter can go unsolved.
 
-.. code:: py
+.. code-block:: py
 
    def foo(a: DefaultStrT | None = None) -> DefaultStrT: ...
 
@@ -289,7 +371,8 @@ module.
   of ``TypeVar``, ``ParamSpec``, and ``TypeVarTuple``.
 
    -  the type passed to default would be available as a ``__default__``
-      attribute.
+      attribute and a sentinel would need to be added that can be
+      checked against to see if it was supplied.
 
 The following changes would be required to both ``GenericAlias``\ es:
 
@@ -308,7 +391,7 @@ If this PEP is accepted, the syntax proposed in :pep:`695` will be
 extended to introduce a way to specify defaults for type parameters
 using the "=" operator inside of the square brackets like so:
 
-.. code:: py
+.. code-block:: py
 
    # TypeVars
    class Foo[T = str]: ...
@@ -322,6 +405,12 @@ using the "=" operator inside of the square brackets like so:
    class Qux[*Ts = *tuple[int, bool]]: ...
    def ham[*Us = *tuple[str]](): ...
 
+   # TypeAliases
+   type Foo[T, U = str] = Bar[T, U]
+   type Baz[**P = (int, str)] = Spam[**P]
+   type Qux[*Ts = *tuple[str]] = Ham[*Ts]
+   type Rab[U, T = str] = Bar[T, U]
+
 This functionality was included in the initial draft of :pep:`695` but
 was removed due to scope creep.
 
@@ -336,7 +425,9 @@ Grammar Changes
         | '*' a=NAME d=[type_param_default]
         | '**' a=NAME d=[type_param_default]
 
-    type_param_default: '=' e=expression
+    type_param_default:
+        | '=' e=expression
+        | '=' e=starred_expression
 
 This would mean that ``TypeVarLike``\ s with defaults proceeding those
 with non-defaults can be checked at compile time.
@@ -345,10 +436,10 @@ with non-defaults can be checked at compile time.
 Rejected Alternatives
 ---------------------
 
-Allowing the ``TypeVarLike``\ s defaults to be passed to ``type.__new__``'s ``**kwargs``
+Allowing the ``TypeVarLike``\s Defaults to Be Passed to ``type.__new__``'s ``**kwargs``
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
-.. code:: py
+.. code-block:: py
 
    T = TypeVar("T")
 
@@ -365,7 +456,7 @@ at runtime.
 
 Ideally, if :pep:`637` wasn't rejected, the following would be acceptable:
 
-.. code:: py
+.. code-block:: py
 
    T = TypeVar("T")
 
@@ -373,10 +464,10 @@ Ideally, if :pep:`637` wasn't rejected, the following would be acceptable:
    class Box(Generic[T = int]):
        value: T | None = None
 
-Allowing non-defaults to follow defaults
+Allowing Non-defaults to Follow Defaults
 ''''''''''''''''''''''''''''''''''''''''
 
-.. code:: py
+.. code-block:: py
 
    YieldT = TypeVar("YieldT", default=Any)
    SendT = TypeVar("SendT", default=Any)
@@ -394,12 +485,12 @@ above two forms were valid. Changing the argument order now would also
 break a lot of codebases. This is also solvable in most cases using a
 ``TypeAlias``.
 
-.. code:: py
+.. code-block:: py
 
    Coro: TypeAlias = Coroutine[Any, Any, T]
    Coro[int] == Coroutine[Any, Any, int]
 
-Having ``default`` implicitly be ``bound``
+Having ``default`` Implicitly Be ``bound``
 ''''''''''''''''''''''''''''''''''''''''''
 
 In an earlier version of this PEP, the ``default`` was implicitly set
@@ -407,7 +498,7 @@ to ``bound`` if no value was passed for ``default``. This while
 convenient, could have a ``TypeVarLike`` with no default follow a
 ``TypeVarLike`` with a default. Consider:
 
-.. code:: py
+.. code-block:: py
 
    T = TypeVar("T", bound=int)  # default is implicitly int
    U = TypeVar("U")