PEP 646: tweaks, clarifications, and move summary example up (#1828)

Readability changes:
* Tweak framing to also acknowledge that variadic generics have been wanted for other things too (though as far as I've been able to tell there isn't a consistent enough pattern to the other use cases to justify including them in the PEP)
* Move the Array example much earlier (both because reading a detailed example is going to be a faster way of getting the gist for people in a hurry, and because we want to make it clear to numerical computing folks from the start that we can also specify a datatype here). (We still have a more detailed Array example later; I'm leaving that there for the time being pending some discussion with Pradeep, but eventually I guess it'll be removed and absorbed into this earlier example)
* Make it clear that other types can appear in `Union` alongside `TypeVarTuple` instances.

Semantic changes:
* Disallow a `Union` of more than one type variable tuple (I thought this would be easier than concatenation of more than one type variable tuple, but Pradeep points out it could still get hairy for type checkers, so disallow it for now to be on the safe safe)

Also some *more* type fixes...
This commit is contained in:
Matthew Rahtz 2021-02-22 04:41:48 +00:00 committed by GitHub
parent a23ba419ff
commit a2339f34ea
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 72 additions and 31 deletions

View File

@ -18,7 +18,8 @@ Abstract
PEP 484 introduced ``TypeVar``, enabling creation of generics parameterised PEP 484 introduced ``TypeVar``, enabling creation of generics parameterised
with a single type. In this PEP, we introduce ``TypeVarTuple``, enabling parameterisation with a single type. In this PEP, we introduce ``TypeVarTuple``, enabling parameterisation
with an *arbitrary* number of types - that is, a *variadic* type variable, with an *arbitrary* number of types - that is, a *variadic* type variable,
enabling *variadic* generics. This allows the type of array-like structures enabling *variadic* generics. This enables a wide variety of use cases.
In particular, it allows the type of array-like structures
in numerical computing libraries such as NumPy and TensorFlow to be in numerical computing libraries such as NumPy and TensorFlow to be
parameterised with the array *shape*, enabling static type checkers parameterised with the array *shape*, enabling static type checkers
to catch shape-related bugs in code that uses these libraries. to catch shape-related bugs in code that uses these libraries.
@ -26,10 +27,15 @@ to catch shape-related bugs in code that uses these libraries.
Motivation Motivation
========== ==========
In the context of numerical computation with libraries such as NumPy and Variadic generics have long been a requested feature, for a myriad of
TensorFlow, the *shape* of arguments is often just as important as the use cases [#typing193]_. One particular use case - a use case with potentially
argument *type*. For example, consider the following function which converts a large impact, and the main case this PEP targets - concerns typing in
batch [#batch]_ of videos to grayscale: numerical libraries.
In the context of numerical computation with libraries such as NumPy and TensorFlow,
the *shape* of arguments is often just as important as the argument *type*.
For example, consider the following function which converts a batch [#batch]_
of videos to grayscale:
:: ::
@ -73,6 +79,54 @@ and so on throughout their code) and for the authors of array libraries (who wou
Variadic generics are necessary for an ``Array`` that is generic in an arbitrary Variadic generics are necessary for an ``Array`` that is generic in an arbitrary
number of axes to be cleanly defined as a single class. number of axes to be cleanly defined as a single class.
Summary Examples
================
Cutting right to the chase, this PEP allows an ``Array`` class that is generic
in its shape (and datatype) to be defined using a newly-introduced
arbitrary-length type variable, ``TypeVarTuple``, as follows:
::
from typing import TypeVar, TypeVarTuple
DType = TypeVar('DType')
Shape = TypeVarTuple('Shape')
class Array(Generic[DType, *Shape]):
def __abs__(self) -> Array[DType, *Shape]: ...
def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]: ...
Such an ``Array`` can be used to support a number of different kinds of
shape annotations. For example, we can add labels describing the
semantic meaning of each axis:
::
from typing import NewType
Height = NewType('Height', int)
Width = NewType('Width', int)
x: Array[float, Height, Width] = Array()
We could also add annotations describing the actual size of each axis:
::
from typing import Literal
L640 = Literal[640]
L480 = Literal[480]
x: Array[int, L480, L640] = Array()
For consistency, we use semantic axis annotations as the basis of the examples
in this PEP, but this PEP is agnostic about which of these two (or possibly other)
ways of using ``Array`` is preferable; that decision is left to library authors.
Specification Specification
============= =============
@ -141,10 +195,6 @@ signatures and variable annotations:
def get_shape(self) -> Tuple[*Shape]: def get_shape(self) -> Tuple[*Shape]:
return self._shape return self._shape
def __abs__(self) -> Array[*Shape]: ...
def __add__(self, other: Array[*Shape]) -> Array[*Shape]: ...
shape = (Height(480), Width(640)) shape = (Height(480), Width(640))
x: Array[Height, Width] = Array(shape) x: Array[Height, Width] = Array(shape)
y = abs(x) # Inferred type is Array[Height, Width] y = abs(x) # Inferred type is Array[Height, Width]
@ -306,8 +356,6 @@ As of this PEP, only a single type variable tuple may appear in a type parameter
class Array(Generic[*Ts1, *Ts2]): ... # Error class Array(Generic[*Ts1, *Ts2]): ... # Error
(``Union`` is the one exception to this rule; see `Type Variable Tuples with Union`_.)
Type Concatenation Type Concatenation
------------------ ------------------
@ -326,7 +374,7 @@ prefixed and/or suffixed:
a: Array[Height, Width] a: Array[Height, Width]
b = add_batch_axis(a) # Inferred type is Array[Batch, Height, Width] b = add_batch_axis(a) # Inferred type is Array[Batch, Height, Width]
c = add_batch_axis(b) # Array[Height, Width] c = del_batch_axis(b) # Array[Height, Width]
d = add_batch_channels(a) # Array[Batch, Height, Width, Channels] d = add_batch_channels(a) # Array[Batch, Height, Width, Channels]
@ -447,29 +495,20 @@ Type variable tuples can also be used with ``Union``:
f(1, 'foo') # Inferred type is Union[int, str] f(1, 'foo') # Inferred type is Union[int, str]
More than one type variable tuple may appear in the the parameter list Here, if the type variable tuple is empty (e.g. if we had ``*args: *Ts``
to ``Union``:
::
def cond_random_choice(
cond: bool,
cond_true: Tuple[*Ts1],
cond_false: Tuple[*Ts2]
) -> Union[*Ts1, *Ts2]:
if cond:
return random.choice(cond_true)
else:
return random.choice(cond_false)
# Inferred type is Union[int, str, float]
cond_random_choice(True, (1, 'foo'), (0.0, 'bar'))
If the type variable tuple is empty (e.g. if we had ``*args: Tuple[*Ts]``
and didn't pass any arguments), the type checker should and didn't pass any arguments), the type checker should
raise an error on the ``Union`` (matching the behaviour of ``Union`` raise an error on the ``Union`` (matching the behaviour of ``Union``
at runtime, which requires at least one type argument). at runtime, which requires at least one type argument).
Other types can also be included in the ``Union``:
::
def f(*args :*Ts) -> Union[int, str, *Ts]: ...
However, note that as elsewhere, only a single type variable tuple
may occur within a ``Union``.
Aliases Aliases
------- -------
@ -704,6 +743,8 @@ Footnotes
References References
========== ==========
.. [#typing193] Python typing issue #193: https://github.com/python/typing/issues/193
.. [#pep-612] PEP 612, "Parameter Specification Variables": .. [#pep-612] PEP 612, "Parameter Specification Variables":
https://www.python.org/dev/peps/pep-0612 https://www.python.org/dev/peps/pep-0612