PEP 677: Runtime Behavior Specification (#2237)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This commit is contained in:
Steven Troxler 2022-01-18 10:34:53 -08:00 committed by GitHub
parent 82b633e160
commit 4a3a9932cb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 158 additions and 21 deletions

View File

@ -635,24 +635,120 @@ callable types and ``=>`` for lambdas.
Runtime Behavior Runtime Behavior
---------------- ----------------
Our tentative plan is that: The new AST nodes need to evaluate to runtime types, and we have two goals for the
behavior of these runtime types:
- The ``__repr__`` will show an arrow syntax literal. - They should expose a structured API that is descriptive and powerful
- We will provide a new API where the runtime data structure can be enough to be compatible with extending the type to include new features
accessed in the same manner as the AST data structure. like named and variadic arguments.
- We will ensure that we provide an API that is backward-compatible - They should also expose an API that is backward-compatible with
with ``typing.Callable`` and ``typing.Concatenate``, specifically ``typing.Callable``.
the behavior of ``__args__`` and ``__parameters__``.
Because these details are still under debate we are currently Evaluation and Structured API
maintaining `a separate doc ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit>`_
with details about the new builtins, the evaluation model, how to We intend to create new builtin types to which the new AST nodes will
provide both a backward-compatible and more structured API, and evaluate, exposing them in the ``types`` module.
possible alternatives to the current plan.
Our plan is to expose a structured API as if they were defined as follows::
class CallableType:
is_async: bool
arguments: Ellipsis | tuple[CallableTypeArgument]
return_type: object
class CallableTypeArgument:
kind: CallableTypeArgumentKind
annotation: object
@enum.global_enum
class CallableTypeArgumentKind(enum.IntEnum):
POSITIONAL_ONLY: int = ...
PARAM_SPEC: int = ...
The evaluation rules are expressed in terms of the following
pseudocode::
def evaluate_callable_type(
callable_type: ast.CallableType | ast.AsyncCallableType:
) -> CallableType:
return CallableType(
is_async=isinstance(callable_type, ast.AsyncCallableType),
arguments=_evaluate_arguments(callable_type.arguments),
return_type=evaluate_expression(callable_type.returns),
)
def _evaluate_arguments(arguments):
match arguments:
case ast.AnyArguments():
return Ellipsis
case ast.ArgumentsList(posonlyargs):
return tuple(
_evaluate_arg(arg) for arg in args
)
case ast.ArgumentsListConcatenation(posonlyargs, param_spec):
return tuple(
*(evaluate_arg(arg) for arg in args),
_evaluate_arg(arg=param_spec, kind=PARAM_SPEC)
)
if isinstance(arguments, Any
return Ellipsis
def _evaluate_arg(arg, kind=POSITIONAL_ONLY):
return CallableTypeArgument(
kind=POSITIONAL_ONLY,
annotation=evaluate_expression(value)
)
Backward-Compatible API
~~~~~~~~~~~~~~~~~~~~~~~
To get backward compatibility with the existing ``types.Callable`` API,
which relies on fields ``__args__`` and ``__parameters__``, we can define
them as if they were written in terms of the following::
import itertools
import typing
def get_args(t: CallableType) -> tuple[object]:
return_type_arg = (
typing.Awaitable[t.return_type]
if t.is_async
else t.return_type
)
arguments = t.arguments
if isinstance(arguments, Ellipsis):
argument_args = (Ellipsis,)
else:
argument_args = (arg.annotation for arg in arguments)
return (
*arguments_args,
return_type_arg
)
def get_parameters(t: CallableType) -> tuple[object]:
out = []
for arg in get_args(t):
if isinstance(arg, typing.ParamSpec):
out.append(t)
else:
out.extend(arg.__parameters__)
return tuple(out)
Additional Behaviors of ``types.CallableType``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As with the ``A | B`` syntax for unions introduced in PEP 604:
- The ``__eq__`` method should treat equivalent ``typing.Callable``
values as equal to values constructed using the builtin syntax, and
otherwise should behave like the ``__eq__`` of ``typing.Callable``.
- The ``__repr__`` method should produce an arrow syntax representation that,
when evaluated, gives us back an equal ``types.CallableType`` instance.
Once the plan is finalized we will include a full specification of
runtime behavior in this section of the PEP.
Rejected Alternatives Rejected Alternatives
===================== =====================
@ -908,9 +1004,9 @@ We rejected this change because:
syntax errors. syntax errors.
- Moreover, if a type is complicated enough that readability is a concern - Moreover, if a type is complicated enough that readability is a concern
we can always use type aliases, for example:: we can always use type aliases, for example::
IntToIntFunction: (int) -> int IntToIntFunction: (int) -> int
def make_adder() -> IntToIntFunction: def make_adder() -> IntToIntFunction:
return lambda x: x + 1 return lambda x: x + 1
@ -991,6 +1087,46 @@ Moreover, none of these ideas help as much with reducing verbosity
as the current proposal, nor do they introduce as strong a visual cue as the current proposal, nor do they introduce as strong a visual cue
as the ``->`` between the parameter types and the return type. as the ``->`` between the parameter types and the return type.
Alternative Runtime Behaviors
-----------------------------
The hard requirements on our runtime API are that:
- It must preserve backward compatibility with ``typing.Callable`` via
``__args__`` and ``__params__``.
- It must provide a structured API, which should be extensible if
in the future we try to support named and variadic arguments.
Alternative APIs
~~~~~~~~~~~~~~~~
We considered having the runtime data ``types.CallableType`` use a
more structured API where there would be separate fields for
``posonlyargs`` and ``param_spec``. The current proposal was
was inspired by the ``inspect.Signature`` type.
We use "argument" in our field and type names, unlike "parameter"
as in ``inspect.Signature``, in order to avoid confusion with
the ``callable_type.__parameters__`` field from the legacy API
that refers to type parameters rather than callable parameters.
Using the plain return type in ``__args__`` for async types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is debatable whether we are required to preserve backward compatiblity
of ``__args__`` for async callable types like ``async (int) -> str``. The
reason is that one could argue they are not expressible directly
using ``typing.Callable``, and therefore it would be fine to set
``__args__`` as ``(int, int)`` rather than ``(int, typing.Awaitable[int])``.
But we believe this would be problematic. By preserving the appearance
of a backward-compatible API while actually breaking its semantics on
async types, we would cause runtime type libraries that attempt to
interpret ``Callable`` using ``__args__`` to fail silently.
It is for this reason that we automatically wrap the return type in
``Awaitable``.
Backward Compatibility Backward Compatibility
====================== ======================
@ -1033,10 +1169,11 @@ Open Issues
Details of the Runtime API Details of the Runtime API
-------------------------- --------------------------
Once we have finalized all details of the runtime behavior, we We have attempted to provide a complete behavior specification in
will need to add a full specification of the behavior to the the `Runtime Behavior`_ section of this PEP.
`Runtime Behavior`_ section of this PEP as well as include that
behavior in our reference implementation. But there are probably more details that we will not realize we
need to define until we build a full reference implementation.
Optimizing ``SyntaxError`` messages Optimizing ``SyntaxError`` messages
----------------------------------- -----------------------------------