PEP 646: Update draft (#1856)
This is the more-or-less final version now. I'll do one last read-through to confirm everything's consistent, then post in typing-sig to confirm it's ready for the Steering Committee. Semantic changes: * Remove support for type variable tuples in `Union`. Apparently the implementation would be tricky in Pyre because of special-casing around `Union`. That could be evidence that it would be tricky in Mypy and pytype. Since we don't have a specific use-case in mind, I think this is fine. * Support concatenation in `Callable`. Pradeep pointed ou that `ParamSpec` doesn't cover all potential use-cases because it doesn't support concatenating suffixes (in turn because `ParamSpec`s can contain keyword arguments, and concatenation is positional). Readability changes: * Remove the section on a full `Array` example. We've moved the important thing here - an explicit confirmation that `Array` can be generic in both datatype and shape - to the beginning of the PEP to make it more obvious. We've removed the `Ndim` thing after someone pointed out that e.g. `Ndim[Literal[2]]` would be incompatible with `Shape[Any, Any]`; I just don't think it's going to work. All that's left is the aliases, which we've moved into the Aliases section itself. Other changes: * Fill out Backwards Compatibility section. * Reference initial CPython implementation.
This commit is contained in:
parent
a3f0183936
commit
b95239fef1
163
pep-0646.rst
163
pep-0646.rst
|
@ -476,38 +476,14 @@ Type variable tuples can also be used in the arguments section of a
|
||||||
Process(target=func, args=(0, 'foo')) # Valid
|
Process(target=func, args=(0, 'foo')) # Valid
|
||||||
Process(target=func, args=('foo', 0)) # Error
|
Process(target=func, args=('foo', 0)) # Error
|
||||||
|
|
||||||
However, note that as of this PEP, if a type variable tuple does appear in
|
Other types and normal type variables can also be prefixed/suffixed
|
||||||
the arguments section of a ``Callable``, it must appear alone.
|
to the type variable tuple:
|
||||||
That is, `Type Concatenation`_ is not supported in the context of ``Callable``.
|
|
||||||
(Use cases where this might otherwise be desirable are likely covered through use
|
|
||||||
of either a ``ParamSpec`` from PEP 612, or a type variable tuple in the ``__call__``
|
|
||||||
signature of a callback protocol from PEP 544.)
|
|
||||||
|
|
||||||
Type Variable Tuples with ``Union``
|
|
||||||
-----------------------------------
|
|
||||||
|
|
||||||
Type variable tuples can also be used with ``Union``:
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
def f(*args: *Ts) -> Union[*Ts]:
|
|
||||||
return random.choice(args)
|
|
||||||
|
|
||||||
f(1, 'foo') # Inferred type is Union[int, str]
|
|
||||||
|
|
||||||
Here, if the type variable tuple is empty (e.g. if we had ``*args: *Ts``
|
|
||||||
and didn't pass any arguments), the type checker should
|
|
||||||
raise an error on the ``Union`` (matching the behaviour of ``Union``
|
|
||||||
at runtime, which requires at least one type argument).
|
|
||||||
|
|
||||||
Other types can also be included in the ``Union``:
|
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
def f(*args :*Ts) -> Union[int, str, *Ts]: ...
|
T = TypeVar('T')
|
||||||
|
|
||||||
However, note that as elsewhere, only a single type variable tuple
|
def foo(f: Callable[[int, *Ts, T], Tuple[T, *Ts]]): ...
|
||||||
may occur within a ``Union``.
|
|
||||||
|
|
||||||
Aliases
|
Aliases
|
||||||
-------
|
-------
|
||||||
|
@ -518,33 +494,65 @@ a similar way to regular type variables:
|
||||||
::
|
::
|
||||||
|
|
||||||
IntTuple = Tuple[int, *Ts]
|
IntTuple = Tuple[int, *Ts]
|
||||||
|
NamedArray = Tuple[str, Array[*Ts]]
|
||||||
|
|
||||||
IntTuple[float, bool] # Equivalent to Tuple[int, float, bool]
|
IntTuple[float, bool] # Equivalent to Tuple[int, float, bool]
|
||||||
|
NamedArray[Height] # Equivalent to Tuple[str, Array[Height]]
|
||||||
|
|
||||||
As this example shows, all type parameters passed to the alias are
|
As this example shows, all type parameters passed to the alias are
|
||||||
bound to the type variable tuple. If no type parameters are given,
|
bound to the type variable tuple.
|
||||||
or if an explicitly empty list of type parameters are given,
|
|
||||||
type variable tuple in the alias is simply ignored:
|
Importantly for our ``Array`` example (see `Summary Examples`_), this
|
||||||
|
allows us to define convenience aliases for arrays of a fixed shape
|
||||||
|
or datatype:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
# Both equivalent to Tuple[int]
|
Shape = TypeVarTuple('Shape')
|
||||||
IntTuple
|
DType = TypeVar('DType')
|
||||||
IntTuple[()]
|
class Array(Generic[DType, *Shape]):
|
||||||
|
|
||||||
|
# E.g. Float32Array[Height, Width, Channels]
|
||||||
|
Float32Array = Array[np.float32, *Shape]
|
||||||
|
|
||||||
|
# E.g. Array1D[np.uint8]
|
||||||
|
Array1D = Array[DType, Any]
|
||||||
|
|
||||||
|
If an explicitly empty type parameter list is given, the type variable
|
||||||
|
tuple in the alias is set empty:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
IntTuple[()] # Equivalent to Tuple[int]
|
||||||
|
NamedArray[()] # Equivalent to Tuple[str, Array[()]]
|
||||||
|
|
||||||
|
If the type parameter list is omitted entirely, the alias is
|
||||||
|
compatible with arbitrary type parameters:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def takes_float_array_of_any_shape(x: Float32Array): ...
|
||||||
|
x: Float32Array[Height, Width] = Array()
|
||||||
|
takes_float_array_of_any_shape(x) # Valid
|
||||||
|
|
||||||
|
def takes_float_array_with_specific_shape(y: Float32Array[Height, Width]): ...
|
||||||
|
y: Float32Array = Array()
|
||||||
|
takes_float_array_with_specific_shape(y) # Valid
|
||||||
|
|
||||||
Normal ``TypeVar`` instances can also be used in such aliases:
|
Normal ``TypeVar`` instances can also be used in such aliases:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
T = TypeVar('T')
|
T = TypeVar('T')
|
||||||
Foo = Tuple[T, *Ts]
|
Foo = Tuple[*Ts, T]
|
||||||
|
|
||||||
# T is bound to `int`; Ts is bound to `bool, str`
|
|
||||||
Foo[int, bool, str]
|
|
||||||
|
|
||||||
Note that the same rules for `Type Concatenation`_ apply for aliases.
|
|
||||||
In particular, only one ``TypeVarTuple`` may occur within an alias,
|
|
||||||
and the ``TypeVarTuple`` must be at the end of the alias.
|
|
||||||
|
|
||||||
|
# Ts bound to Tuple[int], T to int
|
||||||
|
Foo[str, int]
|
||||||
|
# Ts bound to Tuple[()], T to int
|
||||||
|
Foo[int]
|
||||||
|
# T bound to Any, Ts to an arbitrary number of Any
|
||||||
|
Foo
|
||||||
|
|
||||||
Overloads for Accessing Individual Types
|
Overloads for Accessing Individual Types
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
|
@ -574,51 +582,6 @@ overloads for each possible rank is, of course, a rather cumbersome
|
||||||
solution. However, it's the best we can do without additional type
|
solution. However, it's the best we can do without additional type
|
||||||
manipulation mechanisms, which are beyond the scope of this PEP.)
|
manipulation mechanisms, which are beyond the scope of this PEP.)
|
||||||
|
|
||||||
An Ideal Array Type: One Possible Example
|
|
||||||
=========================================
|
|
||||||
|
|
||||||
Type variable tuples allow us to make significant progress on the
|
|
||||||
typing of arrays. However, the array class we have sketched
|
|
||||||
out in this PEP is still missing some desirable features. [#typing-ideas]_
|
|
||||||
|
|
||||||
The most crucial feature missing is the ability to specify
|
|
||||||
the data type (e.g. ``np.float32`` or ``np.uint8``). This is important
|
|
||||||
because some numerical computing libraries will silently cast
|
|
||||||
types, which can easily lead to hard-to-diagnose bugs.
|
|
||||||
|
|
||||||
Additionally, it might be useful to be able to specify the rank
|
|
||||||
instead of the full shape. This could be useful for cases where
|
|
||||||
axes don't have obvious semantic meaning like 'height' or 'width',
|
|
||||||
or where the array is very high-dimensional and writing out all
|
|
||||||
the axes would be too verbose.
|
|
||||||
|
|
||||||
Here is one possible example of how these features might be implemented
|
|
||||||
in a complete array type.
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
# E.g. Ndim[Literal[3]]
|
|
||||||
Integer = TypeVar('Integer')
|
|
||||||
class Ndim(Generic[Integer]): ...
|
|
||||||
|
|
||||||
# E.g. Shape[Height, Width]
|
|
||||||
# (Where Height and Width are custom types)
|
|
||||||
Axes = TypeVarTuple('Axes')
|
|
||||||
class Shape(Generic[*Axes]): ...
|
|
||||||
|
|
||||||
DataType = TypeVar('DataType')
|
|
||||||
ShapeType = TypeVar('ShapeType', Ndim, Shape)
|
|
||||||
|
|
||||||
# The most verbose type
|
|
||||||
# E.g. Array[np.float32, Ndim[Literal[3]]
|
|
||||||
# Array[np.uint8, Shape[Height, Width, Channels]]
|
|
||||||
class Array(Generic[DataType, ShapeType]): ...
|
|
||||||
|
|
||||||
# Type aliases for less verbosity
|
|
||||||
# E.g. Float32Array[Height, Width, Channels]
|
|
||||||
Float32Array = Array[np.float32, Shape[*Axes]]
|
|
||||||
# E.g. Array1D[np.uint8]
|
|
||||||
Array1D = Array[DataType, Ndim[Literal[1]]]
|
|
||||||
|
|
||||||
Rationale and Rejected Ideas
|
Rationale and Rejected Ideas
|
||||||
============================
|
============================
|
||||||
|
@ -710,20 +673,32 @@ in how much of their code they wish to annotate, and to enable compatibility
|
||||||
between old unannotated code and new versions of libraries which do use
|
between old unannotated code and new versions of libraries which do use
|
||||||
these type annotations.
|
these type annotations.
|
||||||
|
|
||||||
|
|
||||||
Backwards Compatibility
|
Backwards Compatibility
|
||||||
=======================
|
=======================
|
||||||
|
|
||||||
TODO
|
The ``Unpack`` version of the PEP should be back-portable to previous
|
||||||
|
versions of Python.
|
||||||
|
|
||||||
* ``Tuple`` needs to be upgraded to support parameterization with a
|
Gradual typing is enabled by the fact that unparameterised variadic classes
|
||||||
type variable tuple.
|
are compatible with an arbitrary number of type parameters. This means
|
||||||
|
that if existing classes are made generic, a) all existing (unparameterised)
|
||||||
|
uses of the class will still work, and b) parameterised and unparameterised
|
||||||
|
versions of the class can be used together (relevant if, for example, library
|
||||||
|
code is updated to use parameters while user code is not, or vice-versa).
|
||||||
|
|
||||||
|
|
||||||
Reference Implementation
|
Reference Implementation
|
||||||
========================
|
========================
|
||||||
|
|
||||||
Two reference implementations exist: one in Pyre, as of TODO, and one in
|
Two reference implementations of type-checking functionality exist:
|
||||||
Pyright, as of v1.1.108.
|
one in Pyre, as of TODO, and one in Pyright, as of v1.1.108.
|
||||||
|
|
||||||
|
A preliminary implementation of the ``Unpack`` version of the PEP in CPython
|
||||||
|
is available in `cpython/23527`_. A preliminary version of the version
|
||||||
|
using the star operator, based on an early implementation of PEP 637,
|
||||||
|
is also available at `mrahtz/cpython/pep637+646`_.
|
||||||
|
|
||||||
|
|
||||||
Footnotes
|
Footnotes
|
||||||
==========
|
==========
|
||||||
|
@ -758,6 +733,10 @@ References
|
||||||
|
|
||||||
.. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/
|
.. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/
|
||||||
|
|
||||||
|
.. _cpython/23527: https://github.com/python/cpython/pull/24527
|
||||||
|
|
||||||
|
.. _mrahtz/cpython/pep637+646: https://github.com/mrahtz/cpython/tree/pep637%2B646
|
||||||
|
|
||||||
|
|
||||||
Acknowledgements
|
Acknowledgements
|
||||||
================
|
================
|
||||||
|
|
Loading…
Reference in New Issue