PEP 646: Explain the results of the new grammar/compiler change (#2189)

This commit is contained in:
Matthew Rahtz 2021-12-13 21:22:29 +00:00 committed by GitHub
parent 641e39fa70
commit 27b476d1c6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 54 additions and 15 deletions

View File

@ -871,9 +871,7 @@ shape properties of numerical computing programs.
Grammar Changes
===============
This PEP requires two grammar changes. Full diffs of ``python.gram``
and simple tests to confirm correct behaviour are available at
https://github.com/mrahtz/cpython/commits/pep646-grammar.
This PEP requires two grammar changes.
Change 1: Star Expressions in Indexes
-------------------------------------
@ -1015,21 +1013,58 @@ Where:
star_annotation: ':' star_expression
This accomplishes the desired outcome (making ``*args: *Ts`` not be a syntax
error) while matching the behaviour of star-unpacking in other contexts:
at runtime, ``__iter__`` is called on the starred object, and a tuple
containing the items of the resulting iterator is set as the type annotion
for ``args``. In other words, at runtime ``*args: *foo`` is equivalent to
``*args: tuple(foo)``.
We also need to deal with the ``star_expression`` that results from this
construction. Normally, a ``star_expression`` occurs within the context
of e.g. a list, so a ``star_expression`` is handled by essentially
calling ``iter()`` on the starred object, and inserting the results
of the resulting iterator into the list at the appropriate place. For
``*args: *Ts``, however, we must process the ``star_expression`` in a
different way.
We do this by instead making a special case for the ``star_expression``
resulting from ``*args: *Ts``, emitting code equivalent to
``[annotation_value] = [*Ts]``. That is, we create an iterator from
``Ts`` by calling ``Ts.__iter__``, fetch a single value from the iterator,
verify that the iterator is exhausted, and set that value as the annotation
value. This results in the unpacked ``TypeVarTuple`` being set directly
as the runtime annotation for ``*args``:
::
>>> Ts = TypeVarTuple('Ts')
>>> def foo(*args: *Ts): pass # Equivalent to `*args: tuple(Ts)`
>>> def foo(*args: *Ts): pass
>>> foo.__annotations__
{'args': (*Ts,)}
{'args': *Ts}
# *Ts is the repr() of Ts._unpacked, an instance of UnpackedTypeVarTuple
This allows the runtime annotation to be consistent with an AST representation
that uses a ``Starred`` node for the annotations of ``args`` - in turn important
for tools that rely on the AST such as mypy to correctly recognise the construction:
::
>>> print(ast.dump(ast.parse('def foo(*args: *Ts): pass'), indent=2))
Module(
body=[
FunctionDef(
name='foo',
args=arguments(
posonlyargs=[],
args=[],
vararg=arg(
arg='args',
annotation=Starred(
value=Name(id='Ts', ctx=Load()),
ctx=Load())),
kwonlyargs=[],
kw_defaults=[],
defaults=[]),
body=[
Pass()],
decorator_list=[])],
type_ignores=[])
Note that the only scenario in which this grammar change allows ``*Ts`` to be
used as a direct annotation (rather than being wrapped in e.g. ``Tuple[*Ts]``)
is ``*args``. Other uses are still invalid:
@ -1045,14 +1080,18 @@ Implications
As with the first grammar change, this change also has a number of side effects.
In particular, the annotation of ``*args`` could be set to a starred object
other than a ``TypeVarTuple`` - for example, the following nonsensical
annotation is possible:
annotations are possible:
::
>>> foo = [1, 2, 3]
>>> def bar(*args: *foo): pass # Equivalent to `*args: tuple(foo)`
>>> foo = [1]
>>> def bar(*args: *foo): pass
>>> bar.__annotations__
{'args': (1, 2, 3)}
{'args': 1}
>>> foo = [1, 2]
>>> def bar(*args: *foo): pass
ValueError: too many values to unpack (expected 1)
Again, prevention of such annotations will need to be done by, say, static
checkers, rather than at the level of syntax.