363 lines
13 KiB
ReStructuredText
363 lines
13 KiB
ReStructuredText
PEP: 647
|
|
Title: User-Defined Type Guards
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Eric Traut <erictr at microsoft.com>
|
|
Sponsor: Guido van Rossum <guido@python.org>
|
|
Discussions-To: typing-sig@python.org
|
|
Status: Accepted
|
|
Type: Standards Track
|
|
Topic: Typing
|
|
Content-Type: text/x-rst
|
|
Created: 07-Oct-2020
|
|
Python-Version: 3.10
|
|
Post-History: 28-Dec-2020, 09-Apr-2021
|
|
Resolution: https://mail.python.org/archives/list/python-dev@python.org/thread/2ME6F6YUVKHOQYKSHTVQQU5WD4CVAZU4/
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP specifies a way for programs to influence conditional type narrowing
|
|
employed by a type checker based on runtime checks.
|
|
|
|
|
|
Motivation
|
|
==========
|
|
|
|
Static type checkers commonly employ a technique called "type narrowing" to
|
|
determine a more precise type of an expression within a program's code flow.
|
|
When type narrowing is applied within a block of code based on a conditional
|
|
code flow statement (such as ``if`` and ``while`` statements), the conditional
|
|
expression is sometimes referred to as a "type guard". Python type checkers
|
|
typically support various forms of type guards expressions.
|
|
|
|
::
|
|
|
|
def func(val: Optional[str]):
|
|
# "is None" type guard
|
|
if val is not None:
|
|
# Type of val is narrowed to str
|
|
...
|
|
else:
|
|
# Type of val is narrowed to None
|
|
...
|
|
|
|
def func(val: Optional[str]):
|
|
# Truthy type guard
|
|
if val:
|
|
# Type of val is narrowed to str
|
|
...
|
|
else:
|
|
# Type of val remains Optional[str]
|
|
...
|
|
|
|
def func(val: Union[str, float]):
|
|
# "isinstance" type guard
|
|
if isinstance(val, str):
|
|
# Type of val is narrowed to str
|
|
...
|
|
else:
|
|
# Type of val is narrowed to float
|
|
...
|
|
|
|
def func(val: Literal[1, 2]):
|
|
# Comparison type guard
|
|
if val == 1:
|
|
# Type of val is narrowed to Literal[1]
|
|
...
|
|
else:
|
|
# Type of val is narrowed to Literal[2]
|
|
...
|
|
|
|
There are cases where type narrowing cannot be applied based on static
|
|
information only. Consider the following example:
|
|
|
|
::
|
|
|
|
def is_str_list(val: List[object]) -> bool:
|
|
"""Determines whether all objects in the list are strings"""
|
|
return all(isinstance(x, str) for x in val)
|
|
|
|
def func1(val: List[object]):
|
|
if is_str_list(val):
|
|
print(" ".join(val)) # Error: invalid type
|
|
|
|
|
|
This code is correct, but a type checker will report a type error because
|
|
the value ``val`` passed to the ``join`` method is understood to be of type
|
|
``List[object]``. The type checker does not have enough information to
|
|
statically verify that the type of ``val`` is ``List[str]`` at this point.
|
|
|
|
This PEP introduces a way for a function like ``is_str_list`` to be defined as
|
|
a "user-defined type guard". This allows code to extend the type guards that
|
|
are supported by type checkers.
|
|
|
|
Using this new mechanism, the ``is_str_list`` function in the above example
|
|
would be modified slightly. Its return type would be changed from ``bool``
|
|
to ``TypeGuard[List[str]]``. This promises not merely that the return value
|
|
is boolean, but that a true indicates the input to the function was of the
|
|
specified type.
|
|
|
|
::
|
|
|
|
from typing import TypeGuard
|
|
|
|
def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
|
|
"""Determines whether all objects in the list are strings"""
|
|
return all(isinstance(x, str) for x in val)
|
|
|
|
|
|
User-defined type guards can also be used to determine whether a dictionary
|
|
conforms to the type requirements of a TypedDict.
|
|
|
|
::
|
|
|
|
class Person(TypedDict):
|
|
name: str
|
|
age: int
|
|
|
|
def is_person(val: dict) -> "TypeGuard[Person]":
|
|
try:
|
|
return isinstance(val["name"], str) and isinstance(val["age"], int)
|
|
except KeyError:
|
|
return False
|
|
|
|
def print_age(val: dict):
|
|
if is_person(val):
|
|
print(f"Age: {val['age']}")
|
|
else:
|
|
print("Not a person!")
|
|
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
TypeGuard Type
|
|
--------------
|
|
|
|
This PEP introduces the symbol ``TypeGuard`` exported from the ``typing``
|
|
module. ``TypeGuard`` is a special form that accepts a single type argument.
|
|
It is used to annotate the return type of a user-defined type guard function.
|
|
Return statements within a type guard function should return bool values,
|
|
and type checkers should verify that all return paths return a bool.
|
|
|
|
In all other respects, TypeGuard is a distinct type from bool. It is not a
|
|
subtype of bool. Therefore, ``Callable[..., TypeGuard[int]]`` is not assignable
|
|
to ``Callable[..., bool]``.
|
|
|
|
When ``TypeGuard`` is used to annotate the return type of a function or
|
|
method that accepts at least one parameter, that function or method is
|
|
treated by type checkers as a user-defined type guard. The type argument
|
|
provided for ``TypeGuard`` indicates the type that has been validated by
|
|
the function.
|
|
|
|
User-defined type guards can be generic functions, as shown in this example:
|
|
|
|
::
|
|
|
|
_T = TypeVar("_T")
|
|
|
|
def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
|
|
return len(val) == 2
|
|
|
|
def func(names: Tuple[str, ...]):
|
|
if is_two_element_tuple(names):
|
|
reveal_type(names) # Tuple[str, str]
|
|
else:
|
|
reveal_type(names) # Tuple[str, ...]
|
|
|
|
|
|
Type checkers should assume that type narrowing should be applied to the
|
|
expression that is passed as the first positional argument to a user-defined
|
|
type guard. If the type guard function accepts more than one argument, no
|
|
type narrowing is applied to those additional argument expressions.
|
|
|
|
If a type guard function is implemented as an instance method or class method,
|
|
the first positional argument maps to the second parameter (after "self" or
|
|
"cls").
|
|
|
|
Here are some examples of user-defined type guard functions that accept more
|
|
than one argument:
|
|
|
|
::
|
|
|
|
def is_str_list(val: List[object], allow_empty: bool) -> TypeGuard[List[str]]:
|
|
if len(val) == 0:
|
|
return allow_empty
|
|
return all(isinstance(x, str) for x in val)
|
|
|
|
_T = TypeVar("_T")
|
|
|
|
def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]:
|
|
return all(isinstance(x, type) for x in val)
|
|
|
|
|
|
The return type of a user-defined type guard function will normally refer to
|
|
a type that is strictly "narrower" than the type of the first argument (that
|
|
is, it's a more specific type that can be assigned to the more general type).
|
|
However, it is not required that the return type be strictly narrower. This
|
|
allows for cases like the example above where ``List[str]`` is not assignable
|
|
to ``List[object]``.
|
|
|
|
When a conditional statement includes a call to a user-defined type guard
|
|
function, and that function returns true, the expression passed as the first
|
|
positional argument to the type guard function should be assumed by a static
|
|
type checker to take on the type specified in the TypeGuard return type,
|
|
unless and until it is further narrowed within the conditional code block.
|
|
|
|
Some built-in type guards provide narrowing for both positive and negative
|
|
tests (in both the ``if`` and ``else`` clauses). For example, consider the
|
|
type guard for an expression of the form ``x is None``. If ``x`` has a type that
|
|
is a union of None and some other type, it will be narrowed to ``None`` in the
|
|
positive case and the other type in the negative case. User-defined type
|
|
guards apply narrowing only in the positive case (the ``if`` clause). The type
|
|
is not narrowed in the negative case.
|
|
|
|
::
|
|
|
|
OneOrTwoStrs = Union[Tuple[str], Tuple[str, str]]
|
|
def func(val: OneOrTwoStrs):
|
|
if is_two_element_tuple(val):
|
|
reveal_type(val) # Tuple[str, str]
|
|
...
|
|
else:
|
|
reveal_type(val) # OneOrTwoStrs
|
|
...
|
|
|
|
if not is_two_element_tuple(val):
|
|
reveal_type(val) # OneOrTwoStrs
|
|
...
|
|
else:
|
|
reveal_type(val) # Tuple[str, str]
|
|
...
|
|
|
|
|
|
Backwards Compatibility
|
|
=======================
|
|
Existing code that does not use this new functionality will be unaffected.
|
|
|
|
Notably, code which uses annotations in a manner incompatible with the
|
|
stdlib typing library should simply not import TypeGuard.
|
|
|
|
|
|
Reference Implementation
|
|
========================
|
|
|
|
The Pyright type checker supports the behavior described in this PEP.
|
|
|
|
|
|
Rejected Ideas
|
|
==============
|
|
|
|
Decorator Syntax
|
|
----------------
|
|
|
|
The use of a decorator was considered for defining type guards.
|
|
|
|
::
|
|
|
|
@type_guard(List[str])
|
|
def is_str_list(val: List[object]) -> bool: ...
|
|
|
|
|
|
The decorator approach is inferior because it requires runtime evaluation of
|
|
the type, precluding forward references. The proposed approach was also deemed
|
|
to be easier to understand and simpler to implement.
|
|
|
|
|
|
Enforcing Strict Narrowing
|
|
--------------------------
|
|
|
|
Strict type narrowing enforcement (requiring that the type specified
|
|
in the TypeGuard type argument is a narrower form of the type specified
|
|
for the first parameter) was considered, but this eliminates valuable
|
|
use cases for this functionality. For instance, the ``is_str_list`` example
|
|
above would be considered invalid because ``List[str]`` is not a subtype of
|
|
``List[object]`` because of invariance rules.
|
|
|
|
One variation that was considered was to require a strict narrowing requirement
|
|
by default but allow the type guard function to specify some flag to
|
|
indicate that it is not following this requirement. This was rejected because
|
|
it was deemed cumbersome and unnecessary.
|
|
|
|
Another consideration was to define some less-strict check that ensures that
|
|
there is some overlap between the value type and the narrowed type specified
|
|
in the TypeGuard. The problem with this proposal is that the rules for type
|
|
compatibility are already very complex when considering unions, protocols,
|
|
type variables, generics, etc. Defining a variant of these rules that relaxes
|
|
some of these constraints just for the purpose of this feature would require
|
|
that we articulate all of the subtle ways in which the rules differ and under
|
|
what specific circumstances the constrains are relaxed. For this reason,
|
|
it was decided to omit all checks.
|
|
|
|
It was noted that without enforcing strict narrowing, it would be possible to
|
|
break type safety. A poorly-written type guard function could produce unsafe or
|
|
even nonsensical results. For example:
|
|
|
|
::
|
|
|
|
def f(value: int) -> TypeGuard[str]:
|
|
return True
|
|
|
|
However, there are many ways a determined or uninformed developer can subvert
|
|
type safety -- most commonly by using ``cast`` or ``Any``. If a Python
|
|
developer takes the time to learn about and implement user-defined
|
|
type guards within their code, it is safe to assume that they are interested
|
|
in type safety and will not write their type guard functions in a way that will
|
|
undermine type safety or produce nonsensical results.
|
|
|
|
|
|
Conditionally Applying TypeGuard Type
|
|
-------------------------------------
|
|
|
|
It was suggested that the expression passed as the first argument to a type
|
|
guard function should retain its existing type if the type of the expression was
|
|
a proper subtype of the type specified in the TypeGuard return type.
|
|
For example, if the type guard function is ``def f(value: object) ->
|
|
TypeGuard[float]`` and the expression passed to this function is of type
|
|
``int``, it would retain the ``int`` type rather than take on the
|
|
``float`` type indicated by the TypeGuard return type. This proposal was
|
|
rejected because it added complexity, inconsistency, and opened up additional
|
|
questions about the proper behavior if the type of the expression was of
|
|
composite types like unions or type variables with multiple constraints. It was
|
|
decided that the added complexity and inconsistency was not justified given
|
|
that it would provide little or no added value.
|
|
|
|
|
|
Narrowing of Arbitrary Parameters
|
|
---------------------------------
|
|
|
|
TypeScript's formulation of user-defined type guards allows for any input
|
|
parameter to be used as the value tested for narrowing. The TypeScript language
|
|
authors could not recall any real-world examples in TypeScript where the
|
|
parameter being tested was not the first parameter. For this reason, it was
|
|
decided unnecessary to burden the Python implementation of user-defined type
|
|
guards with additional complexity to support a contrived use case. If such
|
|
use cases are identified in the future, there are ways the TypeGuard mechanism
|
|
could be extended. This could involve the use of keyword indexing, as proposed
|
|
in :pep:`637`.
|
|
|
|
|
|
Narrowing of Implicit "self" and "cls" Parameters
|
|
-------------------------------------------------
|
|
|
|
The proposal states that the first positional argument is assumed to be the
|
|
value that is tested for narrowing. If the type guard function is implemented
|
|
as an instance or class method, an implicit ``self`` or ``cls`` argument will
|
|
also be passed to the function. A concern was raised that there may be
|
|
cases where it is desired to apply the narrowing logic on ``self`` and ``cls``.
|
|
This is an unusual use case, and accommodating it would significantly
|
|
complicate the implementation of user-defined type guards. It was therefore
|
|
decided that no special provision would be made for it. If narrowing
|
|
of ``self`` or ``cls`` is required, the value can be passed as an explicit
|
|
argument to a type guard function.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|