PEP 646: tweaks, clarifications, and move summary example up (#1828)
Readability changes: * Tweak framing to also acknowledge that variadic generics have been wanted for other things too (though as far as I've been able to tell there isn't a consistent enough pattern to the other use cases to justify including them in the PEP) * Move the Array example much earlier (both because reading a detailed example is going to be a faster way of getting the gist for people in a hurry, and because we want to make it clear to numerical computing folks from the start that we can also specify a datatype here). (We still have a more detailed Array example later; I'm leaving that there for the time being pending some discussion with Pradeep, but eventually I guess it'll be removed and absorbed into this earlier example) * Make it clear that other types can appear in `Union` alongside `TypeVarTuple` instances. Semantic changes: * Disallow a `Union` of more than one type variable tuple (I thought this would be easier than concatenation of more than one type variable tuple, but Pradeep points out it could still get hairy for type checkers, so disallow it for now to be on the safe safe) Also some *more* type fixes...
This commit is contained in:
parent
a23ba419ff
commit
a2339f34ea
103
pep-0646.rst
103
pep-0646.rst
|
@ -18,7 +18,8 @@ Abstract
|
||||||
PEP 484 introduced ``TypeVar``, enabling creation of generics parameterised
|
PEP 484 introduced ``TypeVar``, enabling creation of generics parameterised
|
||||||
with a single type. In this PEP, we introduce ``TypeVarTuple``, enabling parameterisation
|
with a single type. In this PEP, we introduce ``TypeVarTuple``, enabling parameterisation
|
||||||
with an *arbitrary* number of types - that is, a *variadic* type variable,
|
with an *arbitrary* number of types - that is, a *variadic* type variable,
|
||||||
enabling *variadic* generics. This allows the type of array-like structures
|
enabling *variadic* generics. This enables a wide variety of use cases.
|
||||||
|
In particular, it allows the type of array-like structures
|
||||||
in numerical computing libraries such as NumPy and TensorFlow to be
|
in numerical computing libraries such as NumPy and TensorFlow to be
|
||||||
parameterised with the array *shape*, enabling static type checkers
|
parameterised with the array *shape*, enabling static type checkers
|
||||||
to catch shape-related bugs in code that uses these libraries.
|
to catch shape-related bugs in code that uses these libraries.
|
||||||
|
@ -26,10 +27,15 @@ to catch shape-related bugs in code that uses these libraries.
|
||||||
Motivation
|
Motivation
|
||||||
==========
|
==========
|
||||||
|
|
||||||
In the context of numerical computation with libraries such as NumPy and
|
Variadic generics have long been a requested feature, for a myriad of
|
||||||
TensorFlow, the *shape* of arguments is often just as important as the
|
use cases [#typing193]_. One particular use case - a use case with potentially
|
||||||
argument *type*. For example, consider the following function which converts a
|
large impact, and the main case this PEP targets - concerns typing in
|
||||||
batch [#batch]_ of videos to grayscale:
|
numerical libraries.
|
||||||
|
|
||||||
|
In the context of numerical computation with libraries such as NumPy and TensorFlow,
|
||||||
|
the *shape* of arguments is often just as important as the argument *type*.
|
||||||
|
For example, consider the following function which converts a batch [#batch]_
|
||||||
|
of videos to grayscale:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
|
@ -73,6 +79,54 @@ and so on throughout their code) and for the authors of array libraries (who wou
|
||||||
Variadic generics are necessary for an ``Array`` that is generic in an arbitrary
|
Variadic generics are necessary for an ``Array`` that is generic in an arbitrary
|
||||||
number of axes to be cleanly defined as a single class.
|
number of axes to be cleanly defined as a single class.
|
||||||
|
|
||||||
|
Summary Examples
|
||||||
|
================
|
||||||
|
|
||||||
|
Cutting right to the chase, this PEP allows an ``Array`` class that is generic
|
||||||
|
in its shape (and datatype) to be defined using a newly-introduced
|
||||||
|
arbitrary-length type variable, ``TypeVarTuple``, as follows:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
from typing import TypeVar, TypeVarTuple
|
||||||
|
|
||||||
|
DType = TypeVar('DType')
|
||||||
|
Shape = TypeVarTuple('Shape')
|
||||||
|
|
||||||
|
class Array(Generic[DType, *Shape]):
|
||||||
|
|
||||||
|
def __abs__(self) -> Array[DType, *Shape]: ...
|
||||||
|
|
||||||
|
def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]: ...
|
||||||
|
|
||||||
|
Such an ``Array`` can be used to support a number of different kinds of
|
||||||
|
shape annotations. For example, we can add labels describing the
|
||||||
|
semantic meaning of each axis:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
from typing import NewType
|
||||||
|
|
||||||
|
Height = NewType('Height', int)
|
||||||
|
Width = NewType('Width', int)
|
||||||
|
|
||||||
|
x: Array[float, Height, Width] = Array()
|
||||||
|
|
||||||
|
We could also add annotations describing the actual size of each axis:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
L640 = Literal[640]
|
||||||
|
L480 = Literal[480]
|
||||||
|
|
||||||
|
x: Array[int, L480, L640] = Array()
|
||||||
|
|
||||||
|
For consistency, we use semantic axis annotations as the basis of the examples
|
||||||
|
in this PEP, but this PEP is agnostic about which of these two (or possibly other)
|
||||||
|
ways of using ``Array`` is preferable; that decision is left to library authors.
|
||||||
|
|
||||||
Specification
|
Specification
|
||||||
=============
|
=============
|
||||||
|
|
||||||
|
@ -141,10 +195,6 @@ signatures and variable annotations:
|
||||||
def get_shape(self) -> Tuple[*Shape]:
|
def get_shape(self) -> Tuple[*Shape]:
|
||||||
return self._shape
|
return self._shape
|
||||||
|
|
||||||
def __abs__(self) -> Array[*Shape]: ...
|
|
||||||
|
|
||||||
def __add__(self, other: Array[*Shape]) -> Array[*Shape]: ...
|
|
||||||
|
|
||||||
shape = (Height(480), Width(640))
|
shape = (Height(480), Width(640))
|
||||||
x: Array[Height, Width] = Array(shape)
|
x: Array[Height, Width] = Array(shape)
|
||||||
y = abs(x) # Inferred type is Array[Height, Width]
|
y = abs(x) # Inferred type is Array[Height, Width]
|
||||||
|
@ -306,8 +356,6 @@ As of this PEP, only a single type variable tuple may appear in a type parameter
|
||||||
|
|
||||||
class Array(Generic[*Ts1, *Ts2]): ... # Error
|
class Array(Generic[*Ts1, *Ts2]): ... # Error
|
||||||
|
|
||||||
(``Union`` is the one exception to this rule; see `Type Variable Tuples with Union`_.)
|
|
||||||
|
|
||||||
Type Concatenation
|
Type Concatenation
|
||||||
------------------
|
------------------
|
||||||
|
|
||||||
|
@ -326,7 +374,7 @@ prefixed and/or suffixed:
|
||||||
|
|
||||||
a: Array[Height, Width]
|
a: Array[Height, Width]
|
||||||
b = add_batch_axis(a) # Inferred type is Array[Batch, Height, Width]
|
b = add_batch_axis(a) # Inferred type is Array[Batch, Height, Width]
|
||||||
c = add_batch_axis(b) # Array[Height, Width]
|
c = del_batch_axis(b) # Array[Height, Width]
|
||||||
d = add_batch_channels(a) # Array[Batch, Height, Width, Channels]
|
d = add_batch_channels(a) # Array[Batch, Height, Width, Channels]
|
||||||
|
|
||||||
|
|
||||||
|
@ -447,29 +495,20 @@ Type variable tuples can also be used with ``Union``:
|
||||||
|
|
||||||
f(1, 'foo') # Inferred type is Union[int, str]
|
f(1, 'foo') # Inferred type is Union[int, str]
|
||||||
|
|
||||||
More than one type variable tuple may appear in the the parameter list
|
Here, if the type variable tuple is empty (e.g. if we had ``*args: *Ts``
|
||||||
to ``Union``:
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
def cond_random_choice(
|
|
||||||
cond: bool,
|
|
||||||
cond_true: Tuple[*Ts1],
|
|
||||||
cond_false: Tuple[*Ts2]
|
|
||||||
) -> Union[*Ts1, *Ts2]:
|
|
||||||
if cond:
|
|
||||||
return random.choice(cond_true)
|
|
||||||
else:
|
|
||||||
return random.choice(cond_false)
|
|
||||||
|
|
||||||
# Inferred type is Union[int, str, float]
|
|
||||||
cond_random_choice(True, (1, 'foo'), (0.0, 'bar'))
|
|
||||||
|
|
||||||
If the type variable tuple is empty (e.g. if we had ``*args: Tuple[*Ts]``
|
|
||||||
and didn't pass any arguments), the type checker should
|
and didn't pass any arguments), the type checker should
|
||||||
raise an error on the ``Union`` (matching the behaviour of ``Union``
|
raise an error on the ``Union`` (matching the behaviour of ``Union``
|
||||||
at runtime, which requires at least one type argument).
|
at runtime, which requires at least one type argument).
|
||||||
|
|
||||||
|
Other types can also be included in the ``Union``:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def f(*args :*Ts) -> Union[int, str, *Ts]: ...
|
||||||
|
|
||||||
|
However, note that as elsewhere, only a single type variable tuple
|
||||||
|
may occur within a ``Union``.
|
||||||
|
|
||||||
Aliases
|
Aliases
|
||||||
-------
|
-------
|
||||||
|
|
||||||
|
@ -704,6 +743,8 @@ Footnotes
|
||||||
References
|
References
|
||||||
==========
|
==========
|
||||||
|
|
||||||
|
.. [#typing193] Python typing issue #193: https://github.com/python/typing/issues/193
|
||||||
|
|
||||||
.. [#pep-612] PEP 612, "Parameter Specification Variables":
|
.. [#pep-612] PEP 612, "Parameter Specification Variables":
|
||||||
https://www.python.org/dev/peps/pep-0612
|
https://www.python.org/dev/peps/pep-0612
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue