PEP 647: User-defined Type Guards (#1748)
This commit is contained in:
Eric Traut
GitHub
parent
3df12e5446
commit
a2269ff5b8
|
|
@ -0,0 +1,327 @@
|
|||
PEP: 647
|
||||
Title: User-Defined Type Guards
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Eric Traut <erictr at microsoft.com>
|
||||
Sponsor: Guido van Rossum <guido@python.org>
|
||||
Discussions-To: Python-Dev <typing-sig@python.org>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 07-Oct-2020
|
||||
Python-Version: 3.10
|
||||
Post-History: 28-Dec-2020
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP specifies a way for programs to influence conditional type narrowing
|
||||
employed by a type checker based on runtime checks.
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
||||
Static type checkers commonly employ a technique called "type narrowing" to
|
||||
determine a more precise type of an expression within a program's code flow.
|
||||
When type narrowing is applied within a block of code based on a conditional
|
||||
code flow statement (such as ``if`` and ``while`` statements), the conditional
|
||||
expression is sometimes referred to as a "type guard". Python type checkers
|
||||
typically support various forms of type guards expressions.
|
||||
|
||||
::
|
||||
|
||||
def func(val: Optional[str]):
|
||||
# "is None" type guard
|
||||
if val is not None:
|
||||
# Type of val is narrowed to str
|
||||
...
|
||||
else:
|
||||
# Type of val is narrowed to None
|
||||
...
|
||||
|
||||
def func(val: Optional[str]):
|
||||
# Truthy type guard
|
||||
if val:
|
||||
# Type of val is narrowed to str
|
||||
...
|
||||
else:
|
||||
# Type of val remains Optional[str]
|
||||
...
|
||||
|
||||
def func(val: Union[str, float]):
|
||||
# "isinstance" type guard
|
||||
if isinstance(val, str):
|
||||
# Type of val is narrowed to str
|
||||
...
|
||||
else:
|
||||
# Type of val is narrowed to float
|
||||
...
|
||||
|
||||
def func(val: Literal[1, 2]):
|
||||
# Comparison type guard
|
||||
if val == 1:
|
||||
# Type of val is narrowed to Literal[1]
|
||||
...
|
||||
else:
|
||||
# Type of val is narrowed to Literal[2]
|
||||
...
|
||||
|
||||
There are cases where type narrowing cannot be applied based on static
|
||||
information only. Consider the following example:
|
||||
|
||||
::
|
||||
|
||||
def is_str_list(val: List[object]) -> bool:
|
||||
"""Determines whether all objects in the list are strings"""
|
||||
return all(isinstance(x, str) for x in val)
|
||||
|
||||
def func1(val: List[object]):
|
||||
if is_str_list(val):
|
||||
print(" ".join(val)) # Error: invalid type
|
||||
|
||||
|
||||
This code is correct, but a type checker will report a type error because
|
||||
the value ``val`` passed to the ``join`` method is understood to be of type
|
||||
``List[object]``. The type checker does not have enough information to
|
||||
statically verify that the type of ``val`` is ``List[str]`` at this point.
|
||||
|
||||
This PEP introduces a way for a function like ``is_str_list`` to be defined as
|
||||
a "user-defined type guard". This allows code to extend the type guards that
|
||||
are supported by type checkers.
|
||||
|
||||
Using this new mechanism, the ``is_str_list`` function in the above example
|
||||
would be modified slightly. Its return type would be changed from ``bool``
|
||||
to ``TypeGuard[List[str]]``.
|
||||
|
||||
::
|
||||
|
||||
from typing import TypeGuard
|
||||
|
||||
def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
|
||||
"""Determines whether all objects in the list are strings"""
|
||||
return all(isinstance(x, str) for x in val)
|
||||
|
||||
|
||||
User-defined type guards can also be used to determine whether a dictionary
|
||||
conforms to the type requirements of a TypedDict.
|
||||
|
||||
::
|
||||
|
||||
class Person(TypedDict):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
def is_person(val: dict) -> "TypeGuard[Person]":
|
||||
try:
|
||||
return isinstance(val["name"], str) and isinstance(val["age"], int)
|
||||
except KeyError:
|
||||
return False
|
||||
|
||||
def print_age(val: dict):
|
||||
if is_person(val):
|
||||
print(f"Age: {val['age']}")
|
||||
else:
|
||||
print("Not a person!")
|
||||
|
||||
|
||||
|
||||
Specification
|
||||
=============
|
||||
|
||||
TypeGuard Type
|
||||
--------------
|
||||
|
||||
This PEP introduces the symbol ``TypeGuard`` exported from the ``typing``
|
||||
module. ``TypeGuard`` is a type alias for the built-in `bool` type, but it
|
||||
allows for a single type argument. It is meant to be used to annotate the
|
||||
return type of a function or method. When it is used in other contexts, it
|
||||
is treated as a ``bool``.
|
||||
|
||||
When ``TypeGuard`` is used to annotate the return type of a function or
|
||||
method that accepts at least one parameter, that function or method is
|
||||
assumed by type checkers to be a user-defined type guard. The type argument
|
||||
provided for ``TypeGuard`` indicates the type that has been validated by
|
||||
the function.
|
||||
|
||||
User-defined type guards can be generic functions, as shown in this example:
|
||||
|
||||
::
|
||||
|
||||
_T = TypeVar("_T")
|
||||
|
||||
def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
|
||||
return len(val) == 2
|
||||
|
||||
def func(names: Tuple[str, ...]):
|
||||
if is_two_element_tuple(names):
|
||||
reveal_type(names) # Tuple[str, str]
|
||||
else:
|
||||
reveal_type(names) # Tuple[str, ...]
|
||||
|
||||
|
||||
Type checkers should assume that type narrowing should be applied to the
|
||||
expression that is passed as the first explicit argument to a user-defined
|
||||
type guard. If the type guard function accepts more than one argument, no
|
||||
type narrowing is applied to those additional argument expressions.
|
||||
|
||||
If a type guard function is implemented as an instance method or class method,
|
||||
the first explicit argument maps to the second parameter (after "self" or "cls").
|
||||
|
||||
Here are some examples of user-defined type guard functions that accept more
|
||||
than one argument:
|
||||
|
||||
::
|
||||
|
||||
def is_str_list(val: List[object], allow_empty: bool) -> TypeGuard[List[str]]:
|
||||
if len(val) == 0:
|
||||
return allow_empty
|
||||
return all(isinstance(x, str) for x in val)
|
||||
|
||||
_T = TypeVar("_T")
|
||||
|
||||
def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]:
|
||||
return all(isinstance(x, type) for x in val)
|
||||
|
||||
|
||||
The return type of a user-defined type guard function will normally refer to
|
||||
a type that is strictly "narrower" than the type of the first argument (that
|
||||
is, it's a more specific type that can be assigned to the more general type).
|
||||
However, it is not required that the return type be strictly narrower. This
|
||||
allows for cases like the example above where ``List[str]`` is not assignable
|
||||
to ``List[object]``.
|
||||
|
||||
Some built-in type guards provide narrowing for both positive and negative
|
||||
tests (in both the ``if`` and ``else`` clauses). For example, consider the
|
||||
type guard for an expression of the form `x is None`. If `x` has a type that
|
||||
is a union of None and some other type, it will be narrowed to `None` in the
|
||||
positive case and the other type in the negative case. User-defined type
|
||||
guards apply narrowing only in the positive case (the ``if`` clause). The type
|
||||
is not narrowed in the negative case.
|
||||
|
||||
::
|
||||
|
||||
OneOrTwoStrs = Union[Tuple[str], Tuple[str, str]]
|
||||
def func(val: OneOrTwoStrs):
|
||||
if is_two_element_tuple(val):
|
||||
reveal_type(val) # Tuple[str, str]
|
||||
...
|
||||
else:
|
||||
reveal_type(val) # OneOrTwoStrs
|
||||
...
|
||||
|
||||
if not is_two_element_tuple(val):
|
||||
reveal_type(val) # OneOrTwoStrs
|
||||
...
|
||||
else:
|
||||
reveal_type(val) # Tuple[str, str]
|
||||
...
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
Existing code that does not use this new functionality will be unaffected.
|
||||
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
The Pyright type checker supports the behavior described in this PEP.
|
||||
|
||||
|
||||
Rejected Ideas
|
||||
==============
|
||||
|
||||
Decorator Syntax
|
||||
----------------
|
||||
|
||||
The use of a decorator was considered for defining type guards.
|
||||
|
||||
::
|
||||
|
||||
@type_guard(List[str])
|
||||
def is_str_list(val: List[object]) -> bool: ...
|
||||
|
||||
|
||||
The decorator approach is inferior because it requires runtime evaluation of
|
||||
the type, precluding forward references. The proposed approach was also deemed
|
||||
to be easier to understand and simpler to implement.
|
||||
|
||||
|
||||
Enforcing Strict Narrowing
|
||||
--------------------------
|
||||
|
||||
Strict type narrowing enforcement (requiring that the type specified
|
||||
in the TypeGuard type argument is a narrower form of the type specified
|
||||
for the first parameter) was considered, but this eliminates valuable
|
||||
use cases for this functionality. For instance, the ``is_str_list`` example
|
||||
above would be considered invalid because ``List[str]`` is not a subtype of
|
||||
``List[object]`` because of invariance rules.
|
||||
|
||||
One variation that was considered was to require a strict narrowing requirement
|
||||
by default but allow the type guard function to specify some flag to
|
||||
indicate that it is not following this requirement. This was rejected because
|
||||
it was deemed cumbersome and unnecessary.
|
||||
|
||||
Another consideration was to define some less-strict check that ensures that
|
||||
there is some overlap between the value type and the narrowed type specified
|
||||
in the TypeGuard. The problem with this proposal is that the rules for type
|
||||
compatibility are already very complex when considering unions, protocols,
|
||||
type variables, generics, etc. Defining a variant of these rules that relaxes
|
||||
some of these constraints just for the purpose of this feature would require
|
||||
that we articulate all of the subtle ways in which the rules differ and under
|
||||
what specific circumstances the constrains are relaxed. For this reason,
|
||||
it was decided to omit all checks.
|
||||
|
||||
It was noted that without enforcing strict narrowing, it would be possible to
|
||||
break type safety. A poorly-written type guard function could produce unsafe or
|
||||
even nonsensical results. For example:
|
||||
|
||||
::
|
||||
|
||||
def f(value: int) -> TypeGuard[str]:
|
||||
return True
|
||||
|
||||
However, there are many ways a determined or uninformed developer can subvert
|
||||
type safety -- most commonly by using ``cast`` or ``Any``. If a Python
|
||||
developer takes the time to learn about and implement user-defined
|
||||
type guards within their code, it is safe to assume that they are interested
|
||||
in type safety and will not write their type guard functions in a way that will
|
||||
undermine type safety or produce nonsensical results.
|
||||
|
||||
|
||||
Narrowing of Arbitrary Parameters
|
||||
---------------------------------
|
||||
|
||||
TypeScript's formulation of user-defined type guards allows for any input
|
||||
parameter to be used as the value tested for narrowing. The TypeScript language
|
||||
authors could not recall any real-world examples in TypeScript where the
|
||||
parameter being tested was not the first parameter. For this reason, it was
|
||||
decided unnecessary to burden the Python implementation of user-defined type
|
||||
guards with additional complexity to support a contrived use case. If such
|
||||
use cases are identified in the future, there are ways the TypeGuard mechanism
|
||||
could be extended. This could involve the use of keyword indexing, as proposed
|
||||
in PEP 637.
|
||||
|
||||
|
||||
Narrowing of Implicit "self" and "cls" Parameters
|
||||
-------------------------------------------------
|
||||
|
||||
The proposal states that the first explicit argument is assumed to be the
|
||||
value that is tested for narrowing. If the type guard function is implemented
|
||||
as an instance or class method, an implicit ``self`` or ``cls`` argument will
|
||||
also be passed to the function. A concern was raised that there may be
|
||||
cases where it is desired to apply the narrowing logic on ``self`` and ``cls``.
|
||||
This is an unusual use case, and accommodating it would significantly
|
||||
complicate the implementation of user-defined type guards. It was therefore
|
||||
decided that no special provision would be made for it. If narrowing
|
||||
of ``self`` or ``cls`` is required, the value can be passed as an explicit
|
||||
argument to a type guard function.
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document is placed in the public domain or under the
|
||||
CC0-1.0-Universal license, whichever is more permissive.
|
||||
Loading…
Reference in New Issue