PEP 695: Incorporate feedback (#2706)
Incorporated changes from latest round of feedback, and describe new implementation strategy. Co-authored-by: Eric Traut <erictr@microsoft.com>
This commit is contained in:
Eric Traut
GitHub
parent
ce65237e30
commit
eba9adb2d1
149
pep-0695.rst
149
pep-0695.rst
|
|
@ -91,10 +91,10 @@ equivalent.
|
|||
Type variables defined within the global scope also need to be given a name
|
||||
that starts with an underscore to indicate that the variable is private to
|
||||
the module. Globally-defined type variables are also often given names to
|
||||
indicate their variance leading to cumbersome names like "_T_contra" and
|
||||
indicate their variance, leading to cumbersome names like "_T_contra" and
|
||||
"_KT_co". The current mechanisms for allocating type variables also requires
|
||||
the developer to supply a redundant name in quotes (e.g. ``T = TypeVar("T")``).
|
||||
This PEP eliminates the need for the redundant name and the cumbersome
|
||||
This PEP eliminates the need for the redundant name and cumbersome
|
||||
variable names.
|
||||
|
||||
Defining type parameters today requires importing the ``TypeVar`` and
|
||||
|
|
@ -225,6 +225,20 @@ time.
|
|||
def func2[T](T): ... # Syntax Error
|
||||
|
||||
|
||||
Class type parameter names are not mangled if they begin with a double
|
||||
underscore. Mangling would not make sense because type parameters, unlike other
|
||||
class-scoped variables, cannot be accessed through the class dictionary, and
|
||||
the notion of a "private" type parameter doesn't make sense. Other class-scoped
|
||||
variables are mangled if they begin with a double underscore, so the mangled
|
||||
name is used to determine whether there is a name collision with type parameters.
|
||||
|
||||
::
|
||||
|
||||
class ClassA[__T, _ClassA__S]:
|
||||
__T = 0 # OK
|
||||
__S = 0 # Syntax Error (because mangled name is _ClassA__S)
|
||||
|
||||
|
||||
Type Parameter Scopes
|
||||
---------------------
|
||||
|
||||
|
|
@ -393,7 +407,8 @@ At runtime, a ``type`` statement will generate an instance of
|
|||
include:
|
||||
|
||||
* ``__name__`` is a str representing the name of the type alias
|
||||
* ``__parameters__`` is a tuple of ``TypeVar``, ``TypeVarTuple``, or ``ParamSpec`` objects
|
||||
* ``__parameters__`` is a tuple of ``TypeVar``, ``TypeVarTuple``, or
|
||||
``ParamSpec`` objects that parameterize the type alias if it is generic
|
||||
* ``__value__`` is the evaluated value of the type alias
|
||||
|
||||
The ``__value__`` attribute initially has a value of ``None`` while the type
|
||||
|
|
@ -606,47 +621,47 @@ that includes a list of type parameters associated with the function or class.
|
|||
Compiler Changes
|
||||
----------------
|
||||
|
||||
The compiler maintains a list of "active type parameters" as it recursively
|
||||
The compiler maintains a list of "active type variables" as it recursively
|
||||
generates byte codes for the program. Consider the following example.
|
||||
|
||||
::
|
||||
|
||||
class Outer[K, V]:
|
||||
# Active type parameters are K and V
|
||||
# Active type variables are K and V
|
||||
|
||||
class Inner[T]:
|
||||
# Active type parameters are K, V, and T
|
||||
# Active type variables are K, V, and T
|
||||
|
||||
def method[M](self, a: M) -> M:
|
||||
# Active type parameters are K, V, T, and M
|
||||
# Active type variables are K, V, T, and M
|
||||
...
|
||||
|
||||
An active type parameter symbol cannot be used for other purposes within
|
||||
An active type variable symbol cannot be used for other purposes within
|
||||
these scopes. This includes local parameters, local variables, variables
|
||||
bound from other scopes (nonlocal or global), or other type parameters. An
|
||||
attempt to reuse a type parameter name in one of these manners results in
|
||||
attempt to reuse a type variable name in one of these manners results in
|
||||
a syntax error.
|
||||
|
||||
::
|
||||
|
||||
class ClassA[K, V]:
|
||||
class Inner[K]: # Syntax error: K already in use as type parameter
|
||||
class Inner[K]: # Syntax error: K already in use as type variable
|
||||
...
|
||||
|
||||
class ClassB[K, V]:
|
||||
def method(self, K): # Syntax error: K already in use as type parameter
|
||||
def method(self, K): # Syntax error: K already in use as type variable
|
||||
...
|
||||
|
||||
class ClassC[T, T]: # Syntax error: T already in use as type parameter
|
||||
class ClassC[T, T]: # Syntax error: T already in use as type variable
|
||||
...
|
||||
|
||||
def func1[T]():
|
||||
...
|
||||
|
||||
|
||||
A type parameter is considered "active" when compiling the arguments for
|
||||
A type variable is considered "active" when compiling the arguments for
|
||||
a class declaration, the type annotations for a function declaration, and
|
||||
the right-hand expression in a type alias declaration. Type parameters are
|
||||
the right-hand expression in a type alias declaration. Type variable are
|
||||
not considered "active" when compiling the default argument expressions for
|
||||
a function declaration or decorator expressions for classes or functions.
|
||||
|
||||
|
|
@ -655,39 +670,67 @@ a function declaration or decorator expressions for classes or functions.
|
|||
T = list
|
||||
|
||||
@decorator(T) # T in decorator refers to outer variable
|
||||
class ClassA[T](Base[T], metaclass=Meta[T]) # T refers to type parameter
|
||||
class ClassA[T](Base[T], metaclass=Meta[T]) # T refers to type variable
|
||||
...
|
||||
|
||||
@decorator(T) # T in decorator refers to outer variable
|
||||
def func1[T](a: list[T]) -> T: # T refers to type parameter
|
||||
def func1[T](a: list[T]) -> T: # T refers to type variable
|
||||
...
|
||||
|
||||
def func2[T](a = T): # T in default refers to outer variable
|
||||
...
|
||||
|
||||
|
||||
When a type parameter is referenced, the compiler generates the byte codes
|
||||
to construct a ``typing.TypeParameter`` object. This new class can be thought
|
||||
of as a proxy for ``TypeVar``, ``TypeVarTuple`` or ``ParamSpec``. It has a
|
||||
name and ``args`` and ``kwargs`` properties (required for ``ParamSpec``) so
|
||||
it can be used within a type expression at runtime. It also implements a
|
||||
``__or__`` and ``__ror__`` method so it can be used in unions, an ``__iter__``
|
||||
method so it can be unpacked (needed for ``TypeVarTuple``) and a ``__repr__``
|
||||
method so its name can be printed. It does not allow for runtime differentiation
|
||||
between a ``TypeVar``, ``TypeVarTuple`` or ``ParamSpec``, nor does it
|
||||
allow for runtime introspection of the upper bound, constraints, or
|
||||
variance.
|
||||
When a ``class``, ``def``, or ``type`` statement includes one or more type
|
||||
parameters, the compiler emits byte codes to construct the corresponding
|
||||
``typing.TypeVar``, ``typing.TypeVarTuple``, or ``typing.ParamSpec`` instances.
|
||||
It then builds a new tuple that includes all active type variables and stores
|
||||
this new tuple in a local variable. Active type variables include all type
|
||||
parameters declared by outer ``class`` and ``def`` scopes plus those declared
|
||||
by the ``class``, ``def``, or ``type`` statement itself. (In the reference
|
||||
implementation, the local variable happens to have the name
|
||||
``__type_variables__``, but this is an implementation detail. Other Python
|
||||
compilers or future versions of the CPython compiler may choose a different
|
||||
name or an entirely anonymous local variable slot for this purpose.)
|
||||
|
||||
If a bound expression is provided for a type parameter, it is validated for
|
||||
syntax but is not evaluated at runtime. That is, no byte codes are generated
|
||||
for the bound expression. Type checkers should evaluate its type statically.
|
||||
When a type variable is referenced, the compiler generates opcodes that
|
||||
load the active type variable tuple from either the local variable or (if
|
||||
there are no local type variables) through the use of a new opcode called
|
||||
``LOAD_TYPEVARS`` that loads the tuple of active type variables from the
|
||||
current function object. It then emits opcodes to index into this tuple to
|
||||
fetch the desired type variable.
|
||||
|
||||
When a new function is created, the "active type variables" tuple is copied
|
||||
to the C struct field ``func_typevars`` of the function object, making the type
|
||||
variables from outer scopes available to inner scopes of the function or class.
|
||||
|
||||
A new read-only attribute called ``__type_variables__`` is available on class,
|
||||
function, and type alias objects. This attribute is a tuple of the active
|
||||
type variables that are visible within the scope of that class, function,
|
||||
or type alias. This attribute is used for runtime evaluation of stringified
|
||||
(forward referenced) type annotations that include references to type
|
||||
parameters. Functions like ``typing.get_type_hints`` can use this attribute
|
||||
to populate the ``locals`` dictionary with values for type parameters that
|
||||
are in scope when calling ``eval`` to evaluate the stringified expression.
|
||||
|
||||
|
||||
Library Changes
|
||||
---------------
|
||||
|
||||
Several classes in the ``typing`` module that are currently implemented in
|
||||
Python must be reimplemented in C. This includes: ``TypeVar``,
|
||||
``TypeVarTuple``, ``ParamSpec``, and ``Generic``. The new class
|
||||
``TypeAliasType`` (described above) also must be implemented in C.
|
||||
|
||||
The ``typing.get_type_hints`` must be updated to use the new
|
||||
``__type_variables__`` attribute.
|
||||
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
This proposal is prototyped in the CPython code base in
|
||||
`this fork <https://github.com/erictraut/cpython/tree/type_param_syntax>`_.
|
||||
`this fork <https://github.com/erictraut/cpython/tree/type_param_syntax2>`_.
|
||||
|
||||
The Pyright type checker supports the behavior described in this PEP.
|
||||
|
||||
|
|
@ -712,12 +755,16 @@ function decorators, which are common in Python. Only one other popular
|
|||
programming language, C++, uses this approach.
|
||||
|
||||
|
||||
Extends Keyword
|
||||
---------------
|
||||
We considered introducing an "extends" keyword for specifying the upper bound
|
||||
of a type parameter. This is consistent with Java and TypeScript, two other
|
||||
popular languages. However, "extends" didn't feel very Pythonic. It also didn't
|
||||
lend itself to supporting constrained type parameters.
|
||||
Bounds Syntax
|
||||
-------------
|
||||
We explored various syntactic options for specifying the bounds and constraints
|
||||
for a type variable. We considered, but ultimately rejected, the use
|
||||
of a ``<:`` token like in Scala, the use of an ``extends`` or ``with``
|
||||
keyword like in various other languages, and the use of a function call
|
||||
syntax similar to today's ``typing.TypeVar`` constructor. The simple colon
|
||||
syntax is consistent with many other programming languages (see Appendix A),
|
||||
and it was heavily preferred by a cross section of Python developers who
|
||||
were surveyed.
|
||||
|
||||
|
||||
Explicit Variance
|
||||
|
|
@ -740,6 +787,19 @@ compiler. This mangled name would be based on the qualified name of the
|
|||
generic class, function or type alias it was associated with. This approach
|
||||
was rejected because qualified names are not necessarily unique, which means
|
||||
the mangled name would need to be based on some other randomized value.
|
||||
Furthermore, this approach is not compatible with techniques used for
|
||||
evaluating quoted (forward referenced) type annotations.
|
||||
|
||||
|
||||
Lambda Lifting
|
||||
--------------
|
||||
When considering implementation options, we considered introducing a new
|
||||
scope and executing the ``class``, ``def``, or ``type`` statement within
|
||||
a lambda -- a technique that is sometimes referred to as "lambda lifting".
|
||||
We ultimately rejected this idea because it did not work well for statements
|
||||
within a class body (because class-scoped symbols cannot be accessed by
|
||||
inner scopes). It also introduced many odd behaviors for scopes that were
|
||||
further nested within the lambda.
|
||||
|
||||
|
||||
Appendix A: Survey of Type Parameter Syntax
|
||||
|
|
@ -748,7 +808,10 @@ Appendix A: Survey of Type Parameter Syntax
|
|||
Support for generic types is found in many programming languages. In this
|
||||
section, we provide a survey of the options used by other popular programming
|
||||
languages. This is relevant because familiarity with other languages will
|
||||
make it easier for Python developers to understand this concept.
|
||||
make it easier for Python developers to understand this concept. We provide
|
||||
additional details here (for example, default type argument support) that
|
||||
may be useful when considering future extensions to the Python type system.
|
||||
|
||||
|
||||
C++
|
||||
---
|
||||
|
|
@ -1098,6 +1161,16 @@ Summary
|
|||
+------------+----------+---------+--------+----------+-----------+-----------+
|
||||
|
||||
|
||||
Acknowledgements
|
||||
================
|
||||
|
||||
Thanks to Sebastian Rittau for kick-starting the discussions that led to this
|
||||
proposal, to Jukka Lehtosalo for proposing the syntax for type alias
|
||||
statements and to Jelle Zijlstra, Daniel Moisset, and Guido van Rossum
|
||||
for their valuable feedback and suggested improvements to the specification
|
||||
and implementation.
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue