PEP 646: Update draft (#1856)
This is the more-or-less final version now. I'll do one last read-through to confirm everything's consistent, then post in typing-sig to confirm it's ready for the Steering Committee. Semantic changes: * Remove support for type variable tuples in `Union`. Apparently the implementation would be tricky in Pyre because of special-casing around `Union`. That could be evidence that it would be tricky in Mypy and pytype. Since we don't have a specific use-case in mind, I think this is fine. * Support concatenation in `Callable`. Pradeep pointed ou that `ParamSpec` doesn't cover all potential use-cases because it doesn't support concatenating suffixes (in turn because `ParamSpec`s can contain keyword arguments, and concatenation is positional). Readability changes: * Remove the section on a full `Array` example. We've moved the important thing here - an explicit confirmation that `Array` can be generic in both datatype and shape - to the beginning of the PEP to make it more obvious. We've removed the `Ndim` thing after someone pointed out that e.g. `Ndim[Literal[2]]` would be incompatible with `Shape[Any, Any]`; I just don't think it's going to work. All that's left is the aliases, which we've moved into the Aliases section itself. Other changes: * Fill out Backwards Compatibility section. * Reference initial CPython implementation.
This commit is contained in:
Matthew Rahtz
GitHub
parent
a3f0183936
commit
b95239fef1
163
pep-0646.rst
163
pep-0646.rst
|
|
@ -476,38 +476,14 @@ Type variable tuples can also be used in the arguments section of a
|
|||
Process(target=func, args=(0, 'foo')) # Valid
|
||||
Process(target=func, args=('foo', 0)) # Error
|
||||
|
||||
However, note that as of this PEP, if a type variable tuple does appear in
|
||||
the arguments section of a ``Callable``, it must appear alone.
|
||||
That is, `Type Concatenation`_ is not supported in the context of ``Callable``.
|
||||
(Use cases where this might otherwise be desirable are likely covered through use
|
||||
of either a ``ParamSpec`` from PEP 612, or a type variable tuple in the ``__call__``
|
||||
signature of a callback protocol from PEP 544.)
|
||||
|
||||
Type Variable Tuples with ``Union``
|
||||
-----------------------------------
|
||||
|
||||
Type variable tuples can also be used with ``Union``:
|
||||
|
||||
::
|
||||
|
||||
def f(*args: *Ts) -> Union[*Ts]:
|
||||
return random.choice(args)
|
||||
|
||||
f(1, 'foo') # Inferred type is Union[int, str]
|
||||
|
||||
Here, if the type variable tuple is empty (e.g. if we had ``*args: *Ts``
|
||||
and didn't pass any arguments), the type checker should
|
||||
raise an error on the ``Union`` (matching the behaviour of ``Union``
|
||||
at runtime, which requires at least one type argument).
|
||||
|
||||
Other types can also be included in the ``Union``:
|
||||
Other types and normal type variables can also be prefixed/suffixed
|
||||
to the type variable tuple:
|
||||
|
||||
::
|
||||
|
||||
def f(*args :*Ts) -> Union[int, str, *Ts]: ...
|
||||
T = TypeVar('T')
|
||||
|
||||
However, note that as elsewhere, only a single type variable tuple
|
||||
may occur within a ``Union``.
|
||||
def foo(f: Callable[[int, *Ts, T], Tuple[T, *Ts]]): ...
|
||||
|
||||
Aliases
|
||||
-------
|
||||
|
|
@ -518,33 +494,65 @@ a similar way to regular type variables:
|
|||
::
|
||||
|
||||
IntTuple = Tuple[int, *Ts]
|
||||
NamedArray = Tuple[str, Array[*Ts]]
|
||||
|
||||
IntTuple[float, bool] # Equivalent to Tuple[int, float, bool]
|
||||
NamedArray[Height] # Equivalent to Tuple[str, Array[Height]]
|
||||
|
||||
As this example shows, all type parameters passed to the alias are
|
||||
bound to the type variable tuple. If no type parameters are given,
|
||||
or if an explicitly empty list of type parameters are given,
|
||||
type variable tuple in the alias is simply ignored:
|
||||
bound to the type variable tuple.
|
||||
|
||||
Importantly for our ``Array`` example (see `Summary Examples`_), this
|
||||
allows us to define convenience aliases for arrays of a fixed shape
|
||||
or datatype:
|
||||
|
||||
::
|
||||
|
||||
# Both equivalent to Tuple[int]
|
||||
IntTuple
|
||||
IntTuple[()]
|
||||
Shape = TypeVarTuple('Shape')
|
||||
DType = TypeVar('DType')
|
||||
class Array(Generic[DType, *Shape]):
|
||||
|
||||
# E.g. Float32Array[Height, Width, Channels]
|
||||
Float32Array = Array[np.float32, *Shape]
|
||||
|
||||
# E.g. Array1D[np.uint8]
|
||||
Array1D = Array[DType, Any]
|
||||
|
||||
If an explicitly empty type parameter list is given, the type variable
|
||||
tuple in the alias is set empty:
|
||||
|
||||
::
|
||||
|
||||
IntTuple[()] # Equivalent to Tuple[int]
|
||||
NamedArray[()] # Equivalent to Tuple[str, Array[()]]
|
||||
|
||||
If the type parameter list is omitted entirely, the alias is
|
||||
compatible with arbitrary type parameters:
|
||||
|
||||
::
|
||||
|
||||
def takes_float_array_of_any_shape(x: Float32Array): ...
|
||||
x: Float32Array[Height, Width] = Array()
|
||||
takes_float_array_of_any_shape(x) # Valid
|
||||
|
||||
def takes_float_array_with_specific_shape(y: Float32Array[Height, Width]): ...
|
||||
y: Float32Array = Array()
|
||||
takes_float_array_with_specific_shape(y) # Valid
|
||||
|
||||
Normal ``TypeVar`` instances can also be used in such aliases:
|
||||
|
||||
::
|
||||
|
||||
T = TypeVar('T')
|
||||
Foo = Tuple[T, *Ts]
|
||||
|
||||
# T is bound to `int`; Ts is bound to `bool, str`
|
||||
Foo[int, bool, str]
|
||||
|
||||
Note that the same rules for `Type Concatenation`_ apply for aliases.
|
||||
In particular, only one ``TypeVarTuple`` may occur within an alias,
|
||||
and the ``TypeVarTuple`` must be at the end of the alias.
|
||||
Foo = Tuple[*Ts, T]
|
||||
|
||||
# Ts bound to Tuple[int], T to int
|
||||
Foo[str, int]
|
||||
# Ts bound to Tuple[()], T to int
|
||||
Foo[int]
|
||||
# T bound to Any, Ts to an arbitrary number of Any
|
||||
Foo
|
||||
|
||||
Overloads for Accessing Individual Types
|
||||
----------------------------------------
|
||||
|
||||
|
|
@ -574,51 +582,6 @@ overloads for each possible rank is, of course, a rather cumbersome
|
|||
solution. However, it's the best we can do without additional type
|
||||
manipulation mechanisms, which are beyond the scope of this PEP.)
|
||||
|
||||
An Ideal Array Type: One Possible Example
|
||||
=========================================
|
||||
|
||||
Type variable tuples allow us to make significant progress on the
|
||||
typing of arrays. However, the array class we have sketched
|
||||
out in this PEP is still missing some desirable features. [#typing-ideas]_
|
||||
|
||||
The most crucial feature missing is the ability to specify
|
||||
the data type (e.g. ``np.float32`` or ``np.uint8``). This is important
|
||||
because some numerical computing libraries will silently cast
|
||||
types, which can easily lead to hard-to-diagnose bugs.
|
||||
|
||||
Additionally, it might be useful to be able to specify the rank
|
||||
instead of the full shape. This could be useful for cases where
|
||||
axes don't have obvious semantic meaning like 'height' or 'width',
|
||||
or where the array is very high-dimensional and writing out all
|
||||
the axes would be too verbose.
|
||||
|
||||
Here is one possible example of how these features might be implemented
|
||||
in a complete array type.
|
||||
|
||||
::
|
||||
|
||||
# E.g. Ndim[Literal[3]]
|
||||
Integer = TypeVar('Integer')
|
||||
class Ndim(Generic[Integer]): ...
|
||||
|
||||
# E.g. Shape[Height, Width]
|
||||
# (Where Height and Width are custom types)
|
||||
Axes = TypeVarTuple('Axes')
|
||||
class Shape(Generic[*Axes]): ...
|
||||
|
||||
DataType = TypeVar('DataType')
|
||||
ShapeType = TypeVar('ShapeType', Ndim, Shape)
|
||||
|
||||
# The most verbose type
|
||||
# E.g. Array[np.float32, Ndim[Literal[3]]
|
||||
# Array[np.uint8, Shape[Height, Width, Channels]]
|
||||
class Array(Generic[DataType, ShapeType]): ...
|
||||
|
||||
# Type aliases for less verbosity
|
||||
# E.g. Float32Array[Height, Width, Channels]
|
||||
Float32Array = Array[np.float32, Shape[*Axes]]
|
||||
# E.g. Array1D[np.uint8]
|
||||
Array1D = Array[DataType, Ndim[Literal[1]]]
|
||||
|
||||
Rationale and Rejected Ideas
|
||||
============================
|
||||
|
|
@ -710,20 +673,32 @@ in how much of their code they wish to annotate, and to enable compatibility
|
|||
between old unannotated code and new versions of libraries which do use
|
||||
these type annotations.
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
TODO
|
||||
The ``Unpack`` version of the PEP should be back-portable to previous
|
||||
versions of Python.
|
||||
|
||||
* ``Tuple`` needs to be upgraded to support parameterization with a
|
||||
type variable tuple.
|
||||
Gradual typing is enabled by the fact that unparameterised variadic classes
|
||||
are compatible with an arbitrary number of type parameters. This means
|
||||
that if existing classes are made generic, a) all existing (unparameterised)
|
||||
uses of the class will still work, and b) parameterised and unparameterised
|
||||
versions of the class can be used together (relevant if, for example, library
|
||||
code is updated to use parameters while user code is not, or vice-versa).
|
||||
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
Two reference implementations exist: one in Pyre, as of TODO, and one in
|
||||
Pyright, as of v1.1.108.
|
||||
Two reference implementations of type-checking functionality exist:
|
||||
one in Pyre, as of TODO, and one in Pyright, as of v1.1.108.
|
||||
|
||||
A preliminary implementation of the ``Unpack`` version of the PEP in CPython
|
||||
is available in `cpython/23527`_. A preliminary version of the version
|
||||
using the star operator, based on an early implementation of PEP 637,
|
||||
is also available at `mrahtz/cpython/pep637+646`_.
|
||||
|
||||
|
||||
Footnotes
|
||||
==========
|
||||
|
|
@ -758,6 +733,10 @@ References
|
|||
|
||||
.. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/
|
||||
|
||||
.. _cpython/23527: https://github.com/python/cpython/pull/24527
|
||||
|
||||
.. _mrahtz/cpython/pep637+646: https://github.com/mrahtz/cpython/tree/pep637%2B646
|
||||
|
||||
|
||||
Acknowledgements
|
||||
================
|
||||
|
|
|
|||
Loading…
Reference in New Issue