diff --git a/pep-0646.rst b/pep-0646.rst index 289b89c43..c4f4bdcc3 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -132,8 +132,6 @@ Specification In order to support the above use cases, we introduce ``TypeVarTuple``. This serves as a placeholder not for a single type but for an *arbitrary* number of types, and behaving like a number of ``TypeVar`` instances packed in a ``Tuple``. -These are described in detail below. - Type Variable Tuples -------------------- @@ -164,7 +162,7 @@ Type variable tuples behave like a number of individual type variables packed in The ``Shape`` type variable tuple here behaves like ``Tuple[T1, T2]``, where ``T1`` and ``T2`` are type variables. To use these type variables -as type parameters of ``Array``, we must **unpack** the type variable tuple using +as type parameters of ``Array``, we must *unpack* the type variable tuple using the star operator: ``*Shape``. The signature of ``Array`` then behaves as if we had simply written ``class Array(Generic[T1, T2]): ...``. @@ -213,8 +211,8 @@ This is, in fact, deliberately not possible: type variable tuples must for two reasons: * To avoid potential confusion about whether to use a type variable tuple - in a packed or unpacked form ("Hmm, should I do ``-> Shape``, - or ``-> Tuple[Shape]``, or ``-> Tuple[*Shape]``...?") + in a packed or unpacked form ("Hmm, should I write '``-> Shape``', + or '``-> Tuple[Shape]``', or '``-> Tuple[*Shape]``'...?") * To improve readability: the star also functions as an explicit visual indicator that the type variable tuple is not a normal type variable. @@ -224,7 +222,7 @@ for two reasons: Note that the use of the star operator in this context requires a grammar change, and is therefore available only in new versions of Python. To enable use of type variable tuples in older versions of Python, we introduce the ``Unpack`` type -operator that can be used in place of the star operator to unpack type variable tuples: +operator that can be used in place of the star operator: :: @@ -244,13 +242,13 @@ To keep this PEP minimal, ``TypeVarTuple`` does not yet support specification of * Type bounds (``TypeVar('T', bound=ParentClass)``) We leave the decision of how these arguments should behave to a future PEP, when variadic generics have been tested in the field. As of this PEP, type variable tuples are -**invariant**. +invariant. Behaviour when Type Parameters are not Specified '''''''''''''''''''''''''''''''''''''''''''''''' When a generic class parameterised by a type variable tuple is used without -any type parameters, it behaves as if its type parameters are ``Any, ...`` +any type parameters, it behaves as if its type parameters are '``Any, ...``' (an arbitrary number of ``Any``): :: @@ -263,8 +261,8 @@ any type parameters, it behaves as if its type parameters are ``Any, ...`` takes_any_array(y) # Also valid This enables gradual typing: existing functions accepting, for example, -a plain ``tf.Tensor`` will still be valid even if called with -a parameterised ``Tensor[Height, Width]``. +a plain TensorFlow ``Tensor`` will still be valid even if ``Tensor`` is made +generic and calling code passes a ``Tensor[Height, Width]``. This also works in the opposite direction: @@ -275,7 +273,7 @@ This also works in the opposite direction: z: Array takes_specific_array(z) -Thereby even if libraries are updated to use types like ``Array[Height, Width]``, +This way, even if libraries are updated to use types like ``Array[Height, Width]``, users of those libraries won't be forced to also apply type annotations to all of their code; users still have a choice about what parts of their code to type and which parts to not. @@ -303,9 +301,10 @@ that is, ``Tuple[int, int]``, but not ``Tuple[int, ...]``. Note that, as a result of this rule, omitting the type parameter list is the *only* way of instantiating a generic type with an arbitrary number of -type parameters. For example, ``Array`` may *behave* like ``Array[Any, ...]``, -but it cannot be instantiated using ``Array[Any, ...]``, because this would -bind its type variable tuple to ``Tuple[Any, ...]``: +type parameters. (We plan to introduce a more deliberate syntax for this +case in a future PEP.) For example, an unparameterised ``Array`` may +*behave* like ``Array[Any, ...]``, but it cannot be instantiated using +``Array[Any, ...]``, because this would bind its type variable tuple to ``Tuple[Any, ...]``: :: @@ -370,7 +369,9 @@ prefixed and/or suffixed: def add_batch_axis(x: Array[*Shape]) -> Array[Batch, *Shape]: ... def del_batch_axis(x: Array[Batch, *Shape]) -> Array[*Shape]: ... - def add_batch_channels(x: Array[*Shape]) -> Array[Batch, *Shape, Channels]: ... + def add_batch_channels( + x: Array[*Shape] + ) -> Array[Batch, *Shape, Channels]: ... a: Array[Height, Width] b = add_batch_axis(a) # Inferred type is Array[Batch, Height, Width] @@ -396,7 +397,7 @@ Normal ``TypeVar`` instances can also be prefixed and/or suffixed: ``*args`` as a Type Variable Tuple ---------------------------------- -PEP 484 states that when a type annotation is provided for ``*args``, each argument +PEP 484 states that when a type annotation is provided for ``*args``, every argument must be of the type annotated. That is, if we specify ``*args`` to be type ``int``, then *all* arguments must be of type ``int``. This limits our ability to specify the type signatures of functions that take heterogeneous argument types. @@ -494,15 +495,15 @@ a similar way to regular type variables: :: IntTuple = Tuple[int, *Ts] - NamedArray = Tuple[str, Array[*Ts]] + NamedTuple = Tuple[str, Tuple[*Ts]] IntTuple[float, bool] # Equivalent to Tuple[int, float, bool] - NamedArray[Height] # Equivalent to Tuple[str, Array[Height]] + NamedTuple[int, int] # Equivalent to Tuple[str, Tuple[int, int]] As this example shows, all type parameters passed to the alias are bound to the type variable tuple. -Importantly for our ``Array`` example (see `Summary Examples`_), this +Importantly for our original ``Array`` example (see `Summary Examples`_), this allows us to define convenience aliases for arrays of a fixed shape or datatype: @@ -524,7 +525,7 @@ tuple in the alias is set empty: :: IntTuple[()] # Equivalent to Tuple[int] - NamedArray[()] # Equivalent to Tuple[str, Array[()]] + NamedTuple[()] # Equivalent to Tuple[str, Tuple[()]] If the type parameter list is omitted entirely, the alias is compatible with arbitrary type parameters: @@ -535,7 +536,9 @@ compatible with arbitrary type parameters: x: Float32Array[Height, Width] = Array() takes_float_array_of_any_shape(x) # Valid - def takes_float_array_with_specific_shape(y: Float32Array[Height, Width]): ... + def takes_float_array_with_specific_shape( + y: Float32Array[Height, Width] + ): ... y: Float32Array = Array() takes_float_array_with_specific_shape(y) # Valid @@ -556,7 +559,8 @@ Normal ``TypeVar`` instances can also be used in such aliases: Overloads for Accessing Individual Types ---------------------------------------- -For situations where we require access to each individual type, overloads can be used with individual ``TypeVar`` instances in place of the type variable tuple: +For situations where we require access to each individual type in the type variable tuple, +overloads can be used with individual ``TypeVar`` instances in place of the type variable tuple: :: @@ -580,16 +584,16 @@ For situations where we require access to each individual type, overloads can be (For array shape operations in particular, having to specify overloads for each possible rank is, of course, a rather cumbersome solution. However, it's the best we can do without additional type -manipulation mechanisms, which are beyond the scope of this PEP.) +manipulation mechanisms. We plan to introduce these in a future PEP.) Rationale and Rejected Ideas ============================ -Supporting Variadicity Through aliases +Supporting Variadicity Through Aliases -------------------------------------- -As noted in the introduction, it **is** possible to avoid variadic generics +As noted in the introduction, it *is* possible to avoid variadic generics by simply defining aliases for each possible number of type parameters: :: @@ -636,7 +640,7 @@ of the following examples should type-check correctly: y: Array takes_specific_array(y) -Note that this is in contrast to the behaviour of the only existing +Note that this is in contrast to the behaviour of the only currently-existing variadic type in Python, ``Tuple``: :: @@ -715,28 +719,6 @@ Footnotes shape begins with 'time × batch', then ``videos_batch[1][0]`` would select the same frame. -References -========== - -.. [#typing193] Python typing issue #193: https://github.com/python/typing/issues/193 - -.. [#pep-612] PEP 612, "Parameter Specification Variables": - https://www.python.org/dev/peps/pep-0612 - -.. [#numeric-stack] Static typing of Python numeric stack: - https://paper.dropbox.com/doc/Static-typing-of-Python-numeric-stack-summary-6ZQzTkgN6e0oXko8fEWwN - -.. [#typing-ideas] Ideas for array shape typing in Python: https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/edit - -.. [#syntax-proposal] Shape annotation syntax proposal: - https://docs.google.com/document/d/1But-hjet8-djv519HEKvBN6Ik2lW3yu0ojZo6pG9osY/edit - -.. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/ - -.. _cpython/23527: https://github.com/python/cpython/pull/24527 - -.. _mrahtz/cpython/pep637+646: https://github.com/mrahtz/cpython/tree/pep637%2B646 - Acknowledgements ================ @@ -751,24 +733,45 @@ Thank you especially to **Lucio**, for suggesting the star syntax, which has mad Resources ========= -Discussions on variadic generics in Python started in 2016 with `Issue 193`__ -on the python/typing GitHub repository. - -__ https://github.com/python/typing/issues/193 +Discussions on variadic generics in Python started in 2016 with Issue 193 +on the python/typing GitHub repository [#typing193]_. Inspired by this discussion, **Ivan Levkivskyi** made a concrete proposal -at PyCon 2019, summarised in `Type system improvements`__ -and `Static typing of Python numeric stack`__. - -__ https://paper.dropbox.com/doc/Type-system-improvements-HHOkniMG9WcCgS0LzXZAe - -__ https://paper.dropbox.com/doc/Static-typing-of-Python-numeric-stack-summary-6ZQzTkgN6e0oXko8fEWwN +at PyCon 2019, summarised in notes on 'Type system improvements' [#type-improvements]_ +and 'Static typing of Python numeric stack' [#numeric-stack]_. Expanding on these ideas, **Mark Mendoza** and **Vincent Siles** gave a presentation on -`Variadic Type Variables for Decorators and Tensors`__ at the 2019 Python +'Variadic Type Variables for Decorators and Tensors' [#variadic-type-variables]_ at the 2019 Python Typing Summit. -__ https://github.com/facebook/pyre-check/blob/ae85c0c6e99e3bbfc92ec55104bfdc5b9b3097b2/docs/Variadic_Type_Variables_for_Decorators_and_Tensors.pdf + +References +========== + +.. [#typing193] Python typing issue #193: + https://github.com/python/typing/issues/193 + +.. [#type-improvements] Ivan Levkivskyi, 'Type system improvements', PyCon 2019: + https://paper.dropbox.com/doc/Type-system-improvements-HHOkniMG9WcCgS0LzXZAe + +.. [#numeric-stack] Ivan Levkivskyi, 'Static typing of Python numeric stack', PyCon 2019: + https://paper.dropbox.com/doc/Static-typing-of-Python-numeric-stack-summary-6ZQzTkgN6e0oXko8fEWwN + +.. [#typing-ideas] Stephan Hoyer, 'Ideas for array shape typing in Python': + https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/edit + +.. [#variadic-type-variables] Mark Mendoza, 'Variadic Type Variables for Decorators and Tensors', Python Typing Summit 2019: + https://github.com/facebook/pyre-check/blob/ae85c0c6e99e3bbfc92ec55104bfdc5b9b3097b2/docs/Variadic_Type_Variables_for_Decorators_and_Tensors.pdf + +.. [#syntax-proposal] Matthew Rahtz et al., 'Shape annotation syntax proposal': + https://docs.google.com/document/d/1But-hjet8-djv519HEKvBN6Ik2lW3yu0ojZo6pG9osY/edit + +.. [#arbitrary_len] Discussion on Python typing-sig mailing list: + https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/ + +.. _cpython/23527: https://github.com/python/cpython/pull/24527 + +.. _mrahtz/cpython/pep637+646: https://github.com/mrahtz/cpython/tree/pep637%2B646 Copyright =========