From 71daa072442babda4e3298b878406525a855117f Mon Sep 17 00:00:00 2001 From: James Hilton-Balfe Date: Thu, 20 Oct 2022 01:52:52 +0100 Subject: [PATCH] PEP 696: Improve section "Using another TypeVarLike as the default" (#2777) Co-authored-by: C.A.M. Gerlach Co-authored-by: Eric Traut Co-authored-by: Jelle Zijlstra --- pep-0696.rst | 189 ++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 140 insertions(+), 49 deletions(-) diff --git a/pep-0696.rst b/pep-0696.rst index 06bb93172..757e2a9e4 100644 --- a/pep-0696.rst +++ b/pep-0696.rst @@ -15,7 +15,8 @@ Abstract This PEP introduces the concept of type defaults for ``TypeVarLike``\ s (``TypeVar``, ``ParamSpec`` and ``TypeVarTuple``), -which act as defaults for a type parameter when none is specified. +which act as defaults for a type parameter when one is not specified or +the constraint solver isn't able to solve a type parameter to anything. Default type argument support is available in some popular languages such as C++, TypeScript, and Rust. A survey of type parameter syntax in @@ -27,7 +28,7 @@ and can be found in its Motivation ---------- -.. code:: py +.. code-block:: py T = TypeVar("T", default=int) # This means that if no type is specified T = int @@ -42,7 +43,7 @@ One place this `regularly comes up `__ is ``Generator``. I propose changing the *stub definition* to something like: -.. code:: py +.. code-block:: py YieldT = TypeVar("YieldT") SendT = TypeVar("SendT", default=None) @@ -54,7 +55,7 @@ propose changing the *stub definition* to something like: This is also useful for a ``Generic`` that is commonly over one type. -.. code:: py +.. code-block:: py class Bot: ... @@ -73,20 +74,20 @@ also helps non-typing users who rely on auto-complete to speed up their development. This design pattern is common in projects like: - - `discord.py `__ - where the + - `discord.py `__ — where the example above was taken from. - - `NumPy `__ - the default for types + - `NumPy `__ — the default for types like ``ndarray``'s ``dtype`` would be ``float64``. Currently it's ``Unknown`` or ``Any``. - - `TensorFlow `__ (this + - `TensorFlow `__ — this could be used for Tensor similarly to ``numpy.ndarray`` and would be - useful to simplify the definition of ``Layer``). + useful to simplify the definition of ``Layer``. Specification ------------- -Default ordering and subscription rules +Default Ordering and Subscription Rules ''''''''''''''''''''''''''''''''''''''' The order for defaults should follow the standard function parameter @@ -95,7 +96,7 @@ a ``default`` value. Doing so should ideally raise a ``TypeError`` in ``typing._GenericAlias``/``types.GenericAlias``, and a type checker should flag this an error. -.. code:: py +.. code-block:: py DefaultStrT = TypeVar("DefaultStrT", default=str) DefaultIntT = TypeVar("DefaultIntT", default=int) @@ -141,7 +142,7 @@ future, this might be possible (see `Interaction with PEP ``TypeVar`` \ s but use a ``list`` or ``tuple`` of types or an ellipsis literal "``...``". -.. code:: py +.. code-block:: py DefaultP = ParamSpec("DefaultP", default=(str, int)) @@ -156,44 +157,124 @@ literal "``...``". ``TypeVarTuple`` defaults are defined using the same syntax as ``TypeVar`` \ s but use an unpacked tuple of types instead of a single type. -.. code:: py +.. code-block:: py DefaultTs = TypeVarTuple("DefaultTs", default=Unpack[tuple[str, int]]) - class Foo(Generic[DefaultTs]): ... + class Foo(Generic[*DefaultTs]): ... reveal_type(Foo()) # type is Foo[str, int] reveal_type(Foo[int, bool]()) # type is Foo[int, bool] -Using another ``TypeVarLike`` as the default +Using Another ``TypeVarLike`` as ``default`` '''''''''''''''''''''''''''''''''''''''''''' -To use another ``TypeVarLike``\ s as the default they have to be of the -same type. When using another ``TypeVarLike`` (T1) as the default, the default -for the ``TypeVarLike`` (T2), T1 must be used before in the signature -of the class it appears in before T2. T2's bound must be a subtype of -T1's bound. +This allows for a value to be used again when the constraints solver +fails to solve a constraint for a type, or the type parameter to a +generic is missing but another type parameter is specified. + +To use another ``TypeVarLike`` as a default the ``default`` and the +``TypeVarLike`` must be the same type (a ``TypeVar``'s default must be +a ``TypeVar``, etc.). `This could be used on builtins.slice `__ where the ``start`` parameter should default to ``int``, ``stop`` default to the type of ``start`` and step default to ``int | None``. -.. code:: py +.. code-block:: py StartT = TypeVar("StartT", default=int) StopT = TypeVar("StopT", default=StartT) StepT = TypeVar("StepT", default=int | None) - class slice(Generic[StartT, StopT, StepT]): ... # Valid + class slice(Generic[StartT, StopT, StepT]): ... - reveal_type(slice()) # type is slice[int, int, int | None] - reveal_type(slice[str]()) # type is slice[str, str, int | None] - reveal_type(slice[str, str, timedelta]()) # type is slice[str, str, timedelta] + reveal_type(slice()) # type is slice[int, int, int | None] + reveal_type(slice[str]()) # type is slice[str, str, int | None] + reveal_type(slice[str, bool, timedelta]()) # type is slice[str, bool, timedelta] - StartT = TypeVar("StartT", default="StopT") +When using a ``TypeVarLike`` as the default to another ``TypeVarLike``. +Where ``T1`` is the default for ``T2`` the following rules apply. + +.. _scoping-rules: + +Scoping Rules +~~~~~~~~~~~~~ + +``T1`` must be used before ``T2`` in the parameter list of the generic, +or be bound in an outer class or function scope. + +.. code-block:: py + + DefaultT = TypeVar("DefaultT", default=T) + + class Foo(Generic[T, DefaultT]): ... # Valid + def bar(x: T, y: DefaultT): ... # Valid + class Foo(Generic[T]): + class Bar(Generic[DefaultT]): ... # Valid + def outer(x: T): + def inner(y: DefaultT): ... # Valid + + StartT = TypeVar("StartT", default="StopT") # Swapped defaults around from previous example StopT = TypeVar("StopT", default=int) class slice(Generic[StartT, StopT, StepT]): ... - ^^^^^^ # Invalid: ordering does not allow StopT to bound yet + # ^^^^^^ Invalid: ordering does not allow StopT to be bound + def baz(x: DefaultT, y: T): ... + # ^^^^^^^^ Invalid: ordering does not allow DefaultT to be bound + +Bound Rules +~~~~~~~~~~~ + +``T2``'s bound must be a subtype of ``T1``'s bound. + +.. code-block:: py + + T = TypeVar("T", bound=float) + TypeVar("Ok", default=T, bound=int) # Valid + TypeVar("AlsoOk", default=T, bound=float) # Valid + TypeVar("Invalid", default=T, bound=str) # Invalid: str is not a subtype of float + +Constraint Rules +~~~~~~~~~~~~~~~~ + +The constraints of ``T2`` must be a superset of the constraints of ``T1``. + +.. code-block:: py + + T1 = TypeVar("T1", bound=int) + TypeVar("Invalid", float, str, default=T1) # Invalid: upper bound int is incompatible with constraints float or str + + T1 = TypeVar("T1", int, str) + TypeVar("AlsoOk", int, str, bool, default=T1) # Valid + TypeVar("AlsoInvalid", bool, complex, default=T1) # Invalid: {bool, complex} is not a superset of {int, str} + + +``TypeVarLike``\s as Parameters to Generics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``TypeVarLike``\ s are valid as parameters to generics inside of a +``default`` when the first parameter is in scope as determined by the +:ref:`previous section `. + +.. code-block:: py + + T = TypeVar("T") + ListDefaultT = TypeVar("ListDefaultT", default=list[T]) + + class Bar(Generic[T, ListDefaultT]): + def __init__(self, x: T, y: ListDefaultT): ... + + reveal_type(Bar[int]) # type is Bar[int, list[int]] + reveal_type(Bar[int, list[str]]) # type is Bar[int, list[str]] + reveal_type(Bar[int, str]) # type is Bar[int, str] + +Specialisation Rules +~~~~~~~~~~~~~~~~~~~~ + +``TypeVarLike``\ s currently cannot be further subscripted. This might +change if `Higher Kinded TypeVars `__ +are implemented. + ``Generic`` ``TypeAlias``\ es ''''''''''''''''''''''''''''' @@ -204,11 +285,11 @@ that hasn't been overridden it should be treated like it was substituted into the ``TypeAlias``. However, it can be specialised further down the line. -.. code:: py +.. code-block:: py class SomethingWithNoDefaults(Generic[T, T2]): ... - MyAlias: TypeAlias = SomethingWithNoDefaults[int, DefaultStrT] # valid + MyAlias: TypeAlias = SomethingWithNoDefaults[int, DefaultStrT] # Valid reveal_type(MyAlias()) # type is SomethingWithNoDefaults[int, str] reveal_type(MyAlias[bool]()) # type is SomethingWithNoDefaults[int, bool] @@ -220,17 +301,18 @@ Subclassing Subclasses of ``Generic``\ s with ``TypeVarLike``\ s that have defaults behave similarly to ``Generic`` ``TypeAlias``\ es. -.. code:: py +.. code-block:: py - class SubclassMe(Generic[T, DefaultStrT]): ... + class SubclassMe(Generic[T, DefaultStrT]): + x: DefaultStrT class Bar(SubclassMe[int, DefaultStrT]): ... reveal_type(Bar()) # type is Bar[str] reveal_type(Bar[bool]()) # type is Bar[bool] - class Foo(SubclassMe[int]): ... + class Foo(SubclassMe[float]): ... - reveal_type(Foo()) # type is + reveal_type(Foo().x) # type is str Foo[str] # Invalid: Foo cannot be further subscripted @@ -239,14 +321,14 @@ behave similarly to ``Generic`` ``TypeAlias``\ es. class Spam(Baz): ... reveal_type(Spam()) # type is -Using bound and default -''''''''''''''''''''''' +Using ``bound`` and ``default`` +''''''''''''''''''''''''''''''' If both ``bound`` and ``default`` are passed ``default`` must be a subtype of ``bound``. Otherwise the type checker should generate an error. -.. code:: py +.. code-block:: py TypeVar("Ok", bound=float, default=int) # Valid TypeVar("Invalid", bound=str, default=int) # Invalid: the bound and default are incompatible @@ -258,7 +340,7 @@ For constrained ``TypeVar``\ s, the default needs to be one of the constraints. A type checker should generate an error even if it is a subtype of one of the constraints. -.. code:: py +.. code-block:: py TypeVar("Ok", float, str, default=float) # Valid TypeVar("Invalid", float, str, default=int) # Invalid: expected one of float or str got int @@ -268,7 +350,7 @@ Function Defaults ``TypeVarLike``\ s currently can only be used where a parameter can go unsolved. -.. code:: py +.. code-block:: py def foo(a: DefaultStrT | None = None) -> DefaultStrT: ... @@ -289,7 +371,8 @@ module. of ``TypeVar``, ``ParamSpec``, and ``TypeVarTuple``. - the type passed to default would be available as a ``__default__`` - attribute. + attribute and a sentinel would need to be added that can be + checked against to see if it was supplied. The following changes would be required to both ``GenericAlias``\ es: @@ -308,7 +391,7 @@ If this PEP is accepted, the syntax proposed in :pep:`695` will be extended to introduce a way to specify defaults for type parameters using the "=" operator inside of the square brackets like so: -.. code:: py +.. code-block:: py # TypeVars class Foo[T = str]: ... @@ -322,6 +405,12 @@ using the "=" operator inside of the square brackets like so: class Qux[*Ts = *tuple[int, bool]]: ... def ham[*Us = *tuple[str]](): ... + # TypeAliases + type Foo[T, U = str] = Bar[T, U] + type Baz[**P = (int, str)] = Spam[**P] + type Qux[*Ts = *tuple[str]] = Ham[*Ts] + type Rab[U, T = str] = Bar[T, U] + This functionality was included in the initial draft of :pep:`695` but was removed due to scope creep. @@ -336,7 +425,9 @@ Grammar Changes | '*' a=NAME d=[type_param_default] | '**' a=NAME d=[type_param_default] - type_param_default: '=' e=expression + type_param_default: + | '=' e=expression + | '=' e=starred_expression This would mean that ``TypeVarLike``\ s with defaults proceeding those with non-defaults can be checked at compile time. @@ -345,10 +436,10 @@ with non-defaults can be checked at compile time. Rejected Alternatives --------------------- -Allowing the ``TypeVarLike``\ s defaults to be passed to ``type.__new__``'s ``**kwargs`` -'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' +Allowing the ``TypeVarLike``\s Defaults to Be Passed to ``type.__new__``'s ``**kwargs`` +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -.. code:: py +.. code-block:: py T = TypeVar("T") @@ -365,7 +456,7 @@ at runtime. Ideally, if :pep:`637` wasn't rejected, the following would be acceptable: -.. code:: py +.. code-block:: py T = TypeVar("T") @@ -373,10 +464,10 @@ Ideally, if :pep:`637` wasn't rejected, the following would be acceptable: class Box(Generic[T = int]): value: T | None = None -Allowing non-defaults to follow defaults +Allowing Non-defaults to Follow Defaults '''''''''''''''''''''''''''''''''''''''' -.. code:: py +.. code-block:: py YieldT = TypeVar("YieldT", default=Any) SendT = TypeVar("SendT", default=Any) @@ -394,12 +485,12 @@ above two forms were valid. Changing the argument order now would also break a lot of codebases. This is also solvable in most cases using a ``TypeAlias``. -.. code:: py +.. code-block:: py Coro: TypeAlias = Coroutine[Any, Any, T] Coro[int] == Coroutine[Any, Any, int] -Having ``default`` implicitly be ``bound`` +Having ``default`` Implicitly Be ``bound`` '''''''''''''''''''''''''''''''''''''''''' In an earlier version of this PEP, the ``default`` was implicitly set @@ -407,7 +498,7 @@ to ``bound`` if no value was passed for ``default``. This while convenient, could have a ``TypeVarLike`` with no default follow a ``TypeVarLike`` with a default. Consider: -.. code:: py +.. code-block:: py T = TypeVar("T", bound=int) # default is implicitly int U = TypeVar("U")