1679 lines
58 KiB
Plaintext
1679 lines
58 KiB
Plaintext
PEP: 484
|
||
Title: Type Hints
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Guido van Rossum <guido@python.org>, Jukka Lehtosalo <jukka.lehtosalo@iki.fi>, Łukasz Langa <lukasz@langa.pl>
|
||
BDFL-Delegate: Mark Shannon
|
||
Discussions-To: Python-Dev <python-dev@python.org>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 29-Sep-2014
|
||
Post-History: 16-Jan-2015,20-Mar-2015,17-Apr-2015,20-May-2015,22-May-2015
|
||
Resolution:
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
PEP 3107 introduced syntax for function annotations, but the semantics
|
||
were deliberately left undefined. There has now been enough 3rd party
|
||
usage for static type analysis that the community would benefit from
|
||
a standard vocabulary and baseline tools within the standard library.
|
||
|
||
This PEP introduces a provisional module to provide these standard
|
||
definitions and tools, along with some conventions for situations
|
||
where annotations are not available.
|
||
|
||
Note that this PEP still explicitly does NOT prevent other uses of
|
||
annotations, nor does it require (or forbid) any particular processing
|
||
of annotations, even when they conform to this specification. It
|
||
simply enables better coordination, as PEP 333 did for web frameworks.
|
||
|
||
For example, here is a simple function whose argument and return type
|
||
are declared in the annotations::
|
||
|
||
def greeting(name: str) -> str:
|
||
return 'Hello ' + name
|
||
|
||
While these annotations are available at runtime through the usual
|
||
``__annotations__`` attribute, *no type checking happens at runtime*.
|
||
Instead, the proposal assumes the existence of a separate off-line
|
||
type checker which users can run over their source code voluntarily.
|
||
Essentially, such a type checker acts as a very powerful linter.
|
||
(While it would of course be possible for individual users to employ
|
||
a similar checker at run time for Design By Contract enforcement or
|
||
JIT optimization, those tools are not yet as mature.)
|
||
|
||
The proposal is strongly inspired by mypy [mypy]_. For example, the
|
||
type "sequence of integers" can be written as ``Sequence[int]``. The
|
||
square brackets mean that no new syntax needs to be added to the
|
||
language. The example here uses a custom type ``Sequence``, imported
|
||
from a pure-Python module ``typing``. The ``Sequence[int]`` notation
|
||
works at runtime by implementing ``__getitem__()`` in the metaclass
|
||
(but its significance is primarily to an offline type checker).
|
||
|
||
The type system supports unions, generic types, and a special type
|
||
named ``Any`` which is consistent with (i.e. assignable to and from) all
|
||
types. This latter feature is taken from the idea of gradual typing.
|
||
Gradual typing and the full type system are explained in PEP 483.
|
||
|
||
Other approaches from which we have borrowed or to which ours can be
|
||
compared and contrasted are described in PEP 482.
|
||
|
||
|
||
Rationale and Goals
|
||
===================
|
||
|
||
PEP 3107 added support for arbitrary annotations on parts of a
|
||
function definition. Although no meaning was assigned to annotations
|
||
then, there has always been an implicit goal to use them for type
|
||
hinting [gvr-artima]_, which is listed as the first possible use case
|
||
in said PEP.
|
||
|
||
This PEP aims to provide a standard syntax for type annotations,
|
||
opening up Python code to easier static analysis and refactoring,
|
||
potential runtime type checking, and (perhaps, in some contexts)
|
||
code generation utilizing type information.
|
||
|
||
Of these goals, static analysis is the most important. This includes
|
||
support for off-line type checkers such as mypy, as well as providing
|
||
a standard notation that can be used by IDEs for code completion and
|
||
refactoring.
|
||
|
||
Non-goals
|
||
---------
|
||
|
||
While the proposed typing module will contain some building blocks for
|
||
runtime type checking -- in particular a useful ``isinstance()``
|
||
implementation -- third party packages would have to be developed to
|
||
implement specific runtime type checking functionality, for example
|
||
using decorators or metaclasses. Using type hints for performance
|
||
optimizations is left as an exercise for the reader.
|
||
|
||
It should also be emphasized that **Python will remain a dynamically
|
||
typed language, and the authors have no desire to ever make type hints
|
||
mandatory, even by convention.**
|
||
|
||
|
||
The meaning of annotations
|
||
==========================
|
||
|
||
Any function without annotations should be treated as having the most
|
||
general type possible, or ignored, by any type checker. Functions
|
||
with the ``@no_type_check`` decorator or with a ``# type: ignore``
|
||
comment should be treated as having no annotations.
|
||
|
||
It is recommended but not required that checked functions have
|
||
annotations for all arguments and the return type. For a checked
|
||
function, the default annotation for arguments and for the return type
|
||
is ``Any``. An exception is that the first argument of instance and
|
||
class methods does not need to be annotated; it is assumed to have the
|
||
type of the containing class for instance methods, and a type object
|
||
type corresponding to the containing class object for class methods.
|
||
For example, in class ``A`` the first argument of an instance method
|
||
has the implicit type ``A``. In a class method, the precise type of
|
||
the first argument cannot be represented using the available type
|
||
notation.
|
||
|
||
(Note that the return type of ``__init__`` ought to be annotated with
|
||
``-> None``. The reason for this is subtle. If ``__init__`` assumed
|
||
a return annotation of ``-> None``, would that mean that an
|
||
argument-less, un-annotated ``__init__`` method should still be
|
||
type-checked? Rather than leaving this ambiguous or introducing an
|
||
exception to the exception, we simply say that ``__init__`` ought to
|
||
have a return annotation; the default behavior is thus the same as for
|
||
other methods.)
|
||
|
||
A type checker is expected to check the body of a checked function for
|
||
consistency with the given annotations. The annotations may also used
|
||
to check correctness of calls appearing in other checked functions.
|
||
|
||
Type checkers are expected to attempt to infer as much information as
|
||
necessary. The minimum requirement is to handle the builtin
|
||
decorators ``@property``, ``@staticmethod`` and ``@classmethod``.
|
||
|
||
|
||
Type Definition Syntax
|
||
======================
|
||
|
||
The syntax leverages PEP 3107-style annotations with a number of
|
||
extensions described in sections below. In its basic form, type
|
||
hinting is used by filling function annotation slots with classes::
|
||
|
||
def greeting(name: str) -> str:
|
||
return 'Hello ' + name
|
||
|
||
This states that the expected type of the ``name`` argument is
|
||
``str``. Analogically, the expected return type is ``str``.
|
||
|
||
Expressions whose type is a subtype of a specific argument type are
|
||
also accepted for that argument.
|
||
|
||
|
||
Acceptable type hints
|
||
---------------------
|
||
|
||
Type hints may be built-in classes (including those defined in
|
||
standard library or third-party extension modules), abstract base
|
||
classes, types available in the ``types`` module, and user-defined
|
||
classes (including those defined in the standard library or
|
||
third-party modules).
|
||
|
||
While annotations are normally the best format for type hints,
|
||
there are times when it is more appropriate to represent them
|
||
by a special comment, or in a separately distributed stub
|
||
file. (See below for examples.)
|
||
|
||
Annotations must be valid expressions that evaluate without raising
|
||
exceptions at the time the function is defined (but see below for
|
||
forward references).
|
||
|
||
Annotations should be kept simple or static analysis tools may not be
|
||
able to interpret the values. For example, dynamically computed types
|
||
are unlikely to be understood. (This is an
|
||
intentionally somewhat vague requirement, specific inclusions and
|
||
exclusions may be added to future versions of this PEP as warranted by
|
||
the discussion.)
|
||
|
||
In addition to the above, the following special constructs defined
|
||
below may be used: ``None``, ``Any``, ``Union``, ``Tuple``,
|
||
``Callable``, all ABCs and stand-ins for concrete classes exported
|
||
from ``typing`` (e.g. ``Sequence`` and ``Dict``), type variables, and
|
||
type aliases.
|
||
|
||
All newly introduced names used to support features described in
|
||
following sections (such as ``Any`` and ``Union``) are available in
|
||
the ``typing`` module.
|
||
|
||
|
||
Using None
|
||
----------
|
||
|
||
When used in a type hint, the expression ``None`` is considered
|
||
equivalent to ``type(None)``.
|
||
|
||
|
||
Type aliases
|
||
------------
|
||
|
||
Type aliases are defined by simple variable assignments::
|
||
|
||
Url = str
|
||
|
||
def retry(url: Url, retry_count: int) -> None: ...
|
||
|
||
Note that we recommend capitalizing alias names, since they represent
|
||
user-defined types, which (like user-defined classes) are typically
|
||
spelled that way.
|
||
|
||
Type aliases may be as complex as type hints in annotations --
|
||
anything that is acceptable as a type hint is acceptable in a type
|
||
alias::
|
||
|
||
from typing import TypeVar, Iterable, Tuple
|
||
|
||
T = TypeVar('T', int, float, complex)
|
||
Vector = Iterable[Tuple[T, T]]
|
||
|
||
def inproduct(v: Vector) -> T:
|
||
return sum(x*y for x, y in v)
|
||
|
||
This is equivalent to::
|
||
|
||
from typing import TypeVar, Iterable, Tuple
|
||
|
||
T = TypeVar('T', int, float, complex)
|
||
|
||
def inproduct(v: Iterable[Tuple[T, T]]) -> T:
|
||
return sum(x*y for x, y in v)
|
||
|
||
|
||
Callable
|
||
--------
|
||
|
||
Frameworks expecting callback functions of specific signatures might be
|
||
type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``.
|
||
Examples::
|
||
|
||
from typing import Callable
|
||
|
||
def feeder(get_next_item: Callable[[], str]) -> None:
|
||
# Body
|
||
|
||
def async_query(on_success: Callable[[int], None],
|
||
on_error: Callable[[int, Exception], None]) -> None:
|
||
# Body
|
||
|
||
It is possible to declare the return type of a callable without
|
||
specifying the call signature by substituting a literal ellipsis
|
||
(three dots) for the list of arguments::
|
||
|
||
def partial(func: Callable[..., str], *args) -> Callable[..., str]:
|
||
# Body
|
||
|
||
Note that there are no square brackets around the ellipsis. The
|
||
arguments of the callback are completely unconstrained in this case
|
||
(and keyword arguments are acceptable).
|
||
|
||
Since using callbacks with keyword arguments is not perceived as a
|
||
common use case, there is currently no support for specifying keyword
|
||
arguments with ``Callable``. Similarly, there is no support for
|
||
specifying callback signatures with a variable number of argument of a
|
||
specific type.
|
||
|
||
|
||
Generics
|
||
--------
|
||
|
||
Since type information about objects kept in containers cannot be
|
||
statically inferred in a generic way, abstract base classes have been
|
||
extended to support subscription to denote expected types for container
|
||
elements. Example::
|
||
|
||
from typing import Mapping, Set
|
||
|
||
def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]) -> None: ...
|
||
|
||
Generics can be parametrized by using a new factory available in
|
||
``typing`` called ``TypeVar``. Example::
|
||
|
||
from typing import Sequence, TypeVar
|
||
|
||
T = TypeVar('T') # Declare type variable
|
||
|
||
def first(l: Sequence[T]) -> T: # Generic function
|
||
return l[0]
|
||
|
||
In this case the contract is that the returned value is consistent with
|
||
the elements held by the collection.
|
||
|
||
A ``TypeVar()`` expression must always directly be assigned to a
|
||
variable (it should not be used as part of a larger expression). The
|
||
argument to ``TypeVar()`` must be a string equal to the variable name
|
||
to which it is assigned. Type variables must not be redefined.
|
||
|
||
``TypeVar`` supports constraining parametric types to a fixed set of
|
||
possible types. For example, we can define a type variable that ranges
|
||
over just ``str`` and ``bytes``. By default, a type variable ranges
|
||
over all possible types. Example of constraining a type variable::
|
||
|
||
from typing import TypeVar
|
||
|
||
AnyStr = TypeVar('AnyStr', str, bytes)
|
||
|
||
def concat(x: AnyStr, y: AnyStr) -> AnyStr:
|
||
return x + y
|
||
|
||
The function ``concat`` can be called with either two ``str`` arguments
|
||
or two ``bytes`` arguments, but not with a mix of ``str`` and ``bytes``
|
||
arguments.
|
||
|
||
There should be at least two constraints, if any; specifying a single
|
||
constraint is disallowed.
|
||
|
||
Subtypes of types constrained by a type variable should be treated
|
||
as their respective explicitly listed base types in the context of the
|
||
type variable. Consider this example::
|
||
|
||
class MyStr(str): ...
|
||
|
||
x = concat(MyStr('apple'), MyStr('pie'))
|
||
|
||
The call is valid but the type variable ``AnyStr`` will be set to
|
||
``str`` and not ``MyStr``. In effect, the inferred type of the return
|
||
value assigned to ``x`` will also be ``str``.
|
||
|
||
Additionally, ``Any`` is a valid value for every type variable.
|
||
Consider the following::
|
||
|
||
def count_truthy(elements: List[Any]) -> int:
|
||
return sum(1 for elem in elements if element)
|
||
|
||
This is equivalent to omitting the generic notation and just saying
|
||
``elements: List``.
|
||
|
||
|
||
User-defined generic types
|
||
--------------------------
|
||
|
||
You can include a ``Generic`` base class to define a user-defined class
|
||
as generic. Example::
|
||
|
||
from typing import TypeVar, Generic
|
||
|
||
T = TypeVar('T')
|
||
|
||
class LoggedVar(Generic[T]):
|
||
def __init__(self, value: T, name: str, logger: Logger) -> None:
|
||
self.name = name
|
||
self.logger = logger
|
||
self.value = value
|
||
|
||
def set(self, new: T) -> None:
|
||
self.log('Set ' + repr(self.value))
|
||
self.value = new
|
||
|
||
def get(self) -> T:
|
||
self.log('Get ' + repr(self.value))
|
||
return self.value
|
||
|
||
def log(self, message: str) -> None:
|
||
self.logger.info('{}: {}'.format(self.name message))
|
||
|
||
``Generic[T]`` as a base class defines that the class ``LoggedVar``
|
||
takes a single type parameter ``T``. This also makes ``T`` valid as
|
||
a type within the class body.
|
||
|
||
The ``Generic`` base class uses a metaclass that defines ``__getitem__``
|
||
so that ``LoggedVar[t]`` is valid as a type::
|
||
|
||
from typing import Iterable
|
||
|
||
def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
|
||
for var in vars:
|
||
var.set(0)
|
||
|
||
A generic type can have any number of type variables, and type variables
|
||
may be constrained. This is valid::
|
||
|
||
from typing import TypeVar, Generic
|
||
...
|
||
|
||
T = TypeVar('T')
|
||
S = TypeVar('S')
|
||
|
||
class Pair(Generic[T, S]):
|
||
...
|
||
|
||
Each type variable argument to ``Generic`` must be distinct. This is
|
||
thus invalid::
|
||
|
||
from typing import TypeVar, Generic
|
||
...
|
||
|
||
T = TypeVar('T')
|
||
|
||
class Pair(Generic[T, T]): # INVALID
|
||
...
|
||
|
||
You can use multiple inheritance with ``Generic``::
|
||
|
||
from typing import TypeVar, Generic, Sized
|
||
|
||
T = TypeVar('T')
|
||
|
||
class LinkedList(Sized, Generic[T]):
|
||
...
|
||
|
||
Subclassing a generic class without specifying type parameters assumes
|
||
``Any`` for each position. In the following example, ``MyIterable``
|
||
is not generic but implicitly inherits from ``Iterable[Any]``:
|
||
|
||
from typing import Iterable
|
||
|
||
class MyIterable(Iterable): # Same as Iterable[Any]
|
||
...
|
||
|
||
Generic metaclasses are not supported.
|
||
|
||
|
||
Instantiating generic classes and type erasure
|
||
----------------------------------------------
|
||
|
||
Generic types like ``List`` or ``Sequence`` cannot be instantiated.
|
||
However, user-defined classes derived from them can be instantiated.
|
||
Suppose we write a ``Node`` class inheriting from ``Generic[T]``::
|
||
|
||
from typing import TypeVar, Generic
|
||
|
||
T = TypeVar('T')
|
||
|
||
class Node(Generic[T]):
|
||
...
|
||
|
||
Now there are two ways we can instantiate this class; the type
|
||
inferred by a type checker may be different depending on the form we
|
||
use. The first way is to give the value of the type parameter
|
||
explicitly -- this overrides whatever type inference the type
|
||
checker would otherwise perform:
|
||
|
||
x = Node[T]() # The type inferred for x is Node[T].
|
||
|
||
y = Node[int]() # The type inferred for y is Node[int].
|
||
|
||
If no explicit types are given, the type checker is given some
|
||
freedom. Consider this code:
|
||
|
||
x = Node()
|
||
|
||
The inferred type could be ``Node[Any]``, as there isn't enough
|
||
context to infer a more precise type. Alternatively, a type checker
|
||
may reject the line and require an explicit annotation, like this:
|
||
|
||
x = Node() # type: Node[int] # Inferred type is Node[int].
|
||
|
||
A type checker with more powerful type inference could look at how
|
||
``x`` is used elsewhere in the file and try to infer a more precise
|
||
type such as ``Node[int]`` even without an explicit type annotation.
|
||
However, it is probably impossible to make such type inference work
|
||
well in all cases, since Python programs can be very dynamic.
|
||
|
||
This PEP doesn't specify the details of how type inference should
|
||
work. We allow different tools to experiment with various approaches.
|
||
We may give more explicit rules in future revisions.
|
||
|
||
At runtime the type is not preserved, and the class of ``x`` is just
|
||
``Node`` in all cases. This behavior is called "type erasure"; it is
|
||
common practice in languages with generics (e.g. Java, TypeScript).
|
||
|
||
|
||
Arbitrary generic types as base classes
|
||
---------------------------------------
|
||
|
||
``Generic[T]`` is only valid as a base class -- it's not a proper type.
|
||
However, user-defined generic types such as ``LinkedList[T]`` from the
|
||
above example and built-in generic types and ABCs such as ``List[T]``
|
||
and ``Iterable[T]`` are valid both as types and as base classes. For
|
||
example, we can define a subclass of ``Dict`` that specializes type
|
||
arguments::
|
||
|
||
from typing import Dict, List, Optional
|
||
|
||
class Node:
|
||
...
|
||
|
||
class SymbolTable(Dict[str, List[Node]]):
|
||
def push(self, name: str, node: Node) -> None:
|
||
self.setdefault(name, []).append(node)
|
||
|
||
def pop(self, name: str) -> Node:
|
||
return self[name].pop()
|
||
|
||
def lookup(self, name: str) -> Optional[Node]:
|
||
nodes = self.get(name)
|
||
if nodes:
|
||
return nodes[-1]
|
||
return None
|
||
|
||
``SymbolTable`` is a subclass of ``dict`` and a subtype of ``Dict[str,
|
||
List[Node]]``.
|
||
|
||
If a generic base class has a type variable as a type argument, this
|
||
makes the defined class generic. For example, we can define a generic
|
||
``LinkedList`` class that is iterable and a container::
|
||
|
||
from typing import TypeVar, Iterable, Container
|
||
|
||
T = TypeVar('T')
|
||
|
||
class LinkedList(Iterable[T], Container[T]):
|
||
...
|
||
|
||
Now ``LinkedList[int]`` is a valid type. Note that we can use ``T``
|
||
multiple times in the base class list, as long as we don't use the
|
||
same type variable ``T`` multiple times within ``Generic[...]``.
|
||
|
||
Also consider the following example::
|
||
|
||
from typing import TypeVar, Mapping
|
||
|
||
T = TypeVar('T')
|
||
|
||
class MyDict(Mapping[str, T]):
|
||
...
|
||
|
||
In this case MyDict has a single parameter, T.
|
||
|
||
|
||
Abstract generic types
|
||
----------------------
|
||
|
||
The metaclass used by ``Generic`` is a subclass of ``abc.ABCMeta``.
|
||
A generic class can be an ABC by including abstract methods
|
||
or properties, and generic classes can also have ABCs as base
|
||
classes without a metaclass conflict.
|
||
|
||
|
||
Type variables with an upper bound
|
||
----------------------------------
|
||
|
||
A type variable may specify an upper bound using ``bound=<type>``.
|
||
This means that an actual type substituted (explicitly or implictly)
|
||
for the type variable must be a subclass of the boundary type. A
|
||
common example is the definition of a Comparable type that works well
|
||
enough to catch the most common errors::
|
||
|
||
from typing import TypeVar
|
||
|
||
class Comparable(metaclass=ABCMeta):
|
||
@abstractmethod
|
||
def __lt__(self, other: Any) -> bool: ...
|
||
... # __gt__ etc. as well
|
||
|
||
CT = TypeVar('CT', bound=Comparable)
|
||
|
||
def min(x: CT, y: CT) -> CT:
|
||
if x < y:
|
||
return x
|
||
else:
|
||
return y
|
||
|
||
min(1, 2) # ok, return type int
|
||
min('x', 'y') # ok, return type str
|
||
|
||
(Note that this is not ideal -- for example ``min('x', 1)`` is invalid
|
||
at runtime but a type checker would simply infer the return type
|
||
``Comparable``. Unfortunately, addressing this would require
|
||
introducing a much more powerful and also much more complicated
|
||
concept, F-bounded polymorphism. We may revisit this in the future.)
|
||
|
||
An upper bound cannot be combined with type constraints (as in used
|
||
``AnyStr``, see the example earlier); type constraints cause the
|
||
inferred type to be _exactly_ one of the constraint types, while an
|
||
upper bound just requires that the actual type is a subclass of the
|
||
boundary type.
|
||
|
||
|
||
Covariance and contravariance
|
||
-----------------------------
|
||
|
||
Consider a class ``Employee`` with a subclass ``Manager``. Now
|
||
suppose we have a function with an argument annotated with
|
||
``List[Employee]``. Should we be allowed to call this function with a
|
||
variable of type ``List[Manager]`` as its argument? Many people would
|
||
answer "yes, of course" without even considering the consequences.
|
||
But unless we know more about the function, a type checker should
|
||
reject such a call: the function might append an ``Employee`` instance
|
||
to the list, which would violate the variable's type in the caller.
|
||
|
||
It turns out such an argument acts _contravariantly_, whereas the
|
||
intuitive answer (which is correct in case the function doesn't mutate
|
||
its argument!) requires the argument to act _covariantly_. A longer
|
||
introduction to these concepts can be found on Wikipedia
|
||
[wiki-variance]_; here we just show how to control a type checker's
|
||
behavior.
|
||
|
||
By default type variables are considered _invariant_, which means that
|
||
arguments for arguments annotated with types like ``List[Employee]``
|
||
must exactly match the type annotation -- no subclasses or
|
||
superclasses of the type parameter (in this example ``Employee``) are
|
||
allowed.
|
||
|
||
To facilitate the declaration of container types where covariant type
|
||
checking is acceptable, a type variable can be declared using
|
||
``covariant=True``. For the (rare) case where contravariant behavior
|
||
is desirable, pass ``contravariant=True``. At most one of these may
|
||
be passed.
|
||
|
||
A typical example involves defining an immutable (or read-only)
|
||
container class::
|
||
|
||
from typing import TypeVar, Generic, Iterable, Iterator
|
||
|
||
T = TypeVar('T', covariant=True)
|
||
|
||
class ImmutableList(Generic[T]):
|
||
def __init__(self, items: Iterable[T]) -> None: ...
|
||
def __iter__(self) -> Iterator[T]: ...
|
||
...
|
||
|
||
class Employee: ...
|
||
|
||
class Manager(Employee): ...
|
||
|
||
def dump_employees(emps: ImmutableList[Employee]) -> None:
|
||
for emp in emps:
|
||
...
|
||
|
||
mgrs = ImmutableList([Manager()]) # type: ImmutableList[Manager]
|
||
dump_employees(mgrs) # OK
|
||
|
||
The read-only collection classes in ``typing`` are all defined using a
|
||
covariant type variable (e.g. ``Mapping`` and ``Sequence``). The
|
||
mutable collection classes (e.g. ``MutableMapping`` and
|
||
``MutableSequence``) are defined using regular invariant type
|
||
variables. The one example of a contravariant type variable is the
|
||
``Generator`` type, which is contravariant in the ``send()`` argument
|
||
type (see below).
|
||
|
||
Note: variance affects type parameters for generic types -- it does
|
||
not affect regular parameters. For example, the following example is
|
||
fine::
|
||
|
||
from typing import TypeVar
|
||
|
||
class Employee: ...
|
||
|
||
class Manager(Employee): ...
|
||
|
||
E = TypeVar('E', bound=Employee) # Invariant
|
||
|
||
def dump_employee(e: E) -> None: ...
|
||
|
||
dump_employee(Manager()) # OK
|
||
|
||
|
||
The numeric tower
|
||
-----------------
|
||
|
||
PEP 3141 defines Python's numeric tower, and the stdlib module
|
||
``numbers`` implements the corresponding ABCs (``Number``,
|
||
``Complex``, ``Real``, ``Rational`` and ``Integral``). There are some
|
||
issues with these ABCs, but the built-in concrete numeric classes
|
||
``complex``, ``float`` and ``int`` are ubiquitous (especially the
|
||
latter two :-).
|
||
|
||
Rather than requiring that users write ``import numbers`` and then use
|
||
``numbers.Float`` etc., this PEP proposes a straightforward shortcut
|
||
that is almost as effective: when an argument is annotated as having
|
||
type ``float``, an argument of type ``int`` is acceptable; similar,
|
||
for an argument annotated as having type ``complex``, arguments of
|
||
type ``float`` or ``int`` are acceptable. This does not handle
|
||
classes implementing the corresponding ABCs or the
|
||
``fractions.Fraction`` class, but we believe those use cases are
|
||
exceedingly rare.
|
||
|
||
|
||
The bytes types
|
||
---------------
|
||
|
||
There are three different builtin classes used for arrays of bytes
|
||
(not counting the classes available in the ``array`` module):
|
||
``bytes``, ``bytearray`` and ``memoryview``. Of these, ``bytes`` and
|
||
``bytearray`` have many behaviors in common (though not all --
|
||
``bytearray`` is mutable).
|
||
|
||
While there is an ABC ``ByteString`` defined in ``collections.abc``
|
||
and a corresponding type in ``typing``, functions accepting bytes (of
|
||
some form) are so common that it would be cumbersome to have to write
|
||
``typing.ByteString`` everywhere. So, as a shortcut similar to that
|
||
for the builtin numeric classes, when an argument is annotated as
|
||
having type ``bytes``, arguments of type ``bytearray`` or
|
||
``memoryview`` are acceptable. (Again, there are situations where
|
||
this isn't sound, but we believe those are exceedingly rare in
|
||
practice.)
|
||
|
||
|
||
Forward references
|
||
------------------
|
||
|
||
When a type hint contains names that have not been defined yet, that
|
||
definition may be expressed as a string literal, to be resolved later.
|
||
|
||
A situation where this occurs commonly is the definition of a
|
||
container class, where the class being defined occurs in the signature
|
||
of some of the methods. For example, the following code (the start of
|
||
a simple binary tree implementation) does not work::
|
||
|
||
class Tree:
|
||
def __init__(self, left: Tree, right: Tree):
|
||
self.left = left
|
||
self.right = right
|
||
|
||
To address this, we write::
|
||
|
||
class Tree:
|
||
def __init__(self, left: 'Tree', right: 'Tree'):
|
||
self.left = left
|
||
self.right = right
|
||
|
||
The string literal should contain a valid Python expression (i.e.,
|
||
``compile(lit, '', 'eval')`` should be a valid code object) and it
|
||
should evaluate without errors once the module has been fully loaded.
|
||
The local and global namespace in which it is evaluated should be the
|
||
same namespaces in which default arguments to the same function would
|
||
be evaluated.
|
||
|
||
Moreover, the expression should be parseable as a valid type hint, i.e.,
|
||
it is constrained by the rules from the section `Acceptable type hints`_
|
||
above.
|
||
|
||
It is allowable to use string literals as *part* of a type hint, for
|
||
example::
|
||
|
||
class Tree:
|
||
...
|
||
def leaves(self) -> List['Tree']:
|
||
...
|
||
|
||
A common use for forward references is when e.g. Django models are
|
||
needed in the signatures. Typically, each model is in a separate
|
||
file, and has methods that arguments whose type involves other models.
|
||
Because of the way circular imports work in Python, it is often not
|
||
possible to import all the needed models directly::
|
||
|
||
# File models/a.py
|
||
from models.b import B
|
||
class A(Model):
|
||
def foo(self, b: B): ...
|
||
|
||
# File models/b.py
|
||
from models.a import A
|
||
class B(Model):
|
||
def bar(self, a: A): ...
|
||
|
||
# File main.py
|
||
from models.a import A
|
||
from models.b import B
|
||
|
||
Assuming main is imported first, this will fail with an ImportError at
|
||
the line ``from models.a import A`` in models/b.py, which is being
|
||
imported from models/a.py before a has defined class A. The solution
|
||
is to switch to module-only imports and reference the models by their
|
||
_module_._class_ name::
|
||
|
||
# File models/a.py
|
||
from models import b
|
||
class A(Model):
|
||
def foo(self, b: 'b.B'): ...
|
||
|
||
# File models/b.py
|
||
from models import a
|
||
class B(Model):
|
||
def bar(self, a: 'a.A'): ...
|
||
|
||
# File main.py
|
||
from models.a import A
|
||
from models.b import B
|
||
|
||
|
||
Union types
|
||
-----------
|
||
|
||
Since accepting a small, limited set of expected types for a single
|
||
argument is common, there is a new special factory called ``Union``.
|
||
Example::
|
||
|
||
from typing import Union
|
||
|
||
def handle_employees(e: Union[Employee, Sequence[Employee]]) -> None:
|
||
if isinstance(e, Employee):
|
||
e = [e]
|
||
...
|
||
|
||
A type factored by ``Union[T1, T2, ...]`` responds ``True`` to
|
||
``issubclass`` checks for ``T1`` and any of its subtypes, ``T2`` and
|
||
any of its subtypes, and so on.
|
||
|
||
One common case of union types are *optional* types. By default,
|
||
``None`` is an invalid value for any type, unless a default value of
|
||
``None`` has been provided in the function definition. Examples::
|
||
|
||
def handle_employee(e: Union[Employee, None]) -> None: ...
|
||
|
||
As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``;
|
||
for example, the above is equivalent to::
|
||
|
||
from typing import Optional
|
||
|
||
def handle_employee(e: Optional[Employee]) -> None: ...
|
||
|
||
An optional type is also automatically assumed when the default value is
|
||
``None``, for example::
|
||
|
||
def handle_employee(e: Employee = None): ...
|
||
|
||
This is equivalent to::
|
||
|
||
def handle_employee(e: Optional[Employee] = None) -> None: ...
|
||
|
||
The ``Any`` type
|
||
----------------
|
||
|
||
A special kind of type is ``Any``. Every type is a subtype of
|
||
``Any``. This is also true for the builtin type ``object``.
|
||
However, to the static type checker these are completely different.
|
||
|
||
When the type of a value is ``object``, the type checker will reject
|
||
almost all operations on it, and assigning it to a variable (or using
|
||
it as a return value) of a more specialized type is a type error. On
|
||
the other hand, when a value has type ``Any``, the type checker will
|
||
allow all operations on it, and a value of type ``Any`` can be assigned
|
||
to a variable (or used as a return value) of a more constrained type.
|
||
|
||
|
||
Version and platform checking
|
||
-----------------------------
|
||
|
||
Type checkers are expected to understand simple version and platform
|
||
checks, e.g.::
|
||
|
||
import sys
|
||
|
||
if sys.version_info[0] >= 3:
|
||
# Python 3 specific definitions
|
||
else:
|
||
# Python 2 specific definitions
|
||
|
||
if sys.platform == 'win32':
|
||
# Windows specific definitions
|
||
else:
|
||
# Posix specific definitions
|
||
|
||
Don't expect a checker to understand obfuscations like
|
||
``"".join(reversed(sys.platform)) == "xunil"``.
|
||
|
||
|
||
Default argument values
|
||
-----------------------
|
||
|
||
In stubs it may be useful to declare an argument as having a default
|
||
without specifying the actual default value. For example::
|
||
|
||
def foo(x: AnyStr, y: AnyStr = ...) -> AnyStr: ...
|
||
|
||
What should the default value look like? Any of the options ``""``,
|
||
``b""`` or ``None`` fails to satisfy the type constraint (actually,
|
||
``None`` will *modify* the type to become ``Optional[AnyStr]``).
|
||
|
||
In such cases the default value may be specified as a literal
|
||
ellipsis, i.e. the above example is literally what you would write.
|
||
|
||
|
||
Compatibility with other uses of function annotations
|
||
=====================================================
|
||
|
||
A number of existing or potential use cases for function annotations
|
||
exist, which are incompatible with type hinting. These may confuse
|
||
a static type checker. However, since type hinting annotations have no
|
||
runtime behavior (other than evaluation of the annotation expression and
|
||
storing annotations in the ``__annotations__`` attribute of the function
|
||
object), this does not make the program incorrect -- it just may cause
|
||
a type checker to emit spurious warnings or errors.
|
||
|
||
To mark portions of the program that should not be covered by type
|
||
hinting, you can use one or more of the following:
|
||
|
||
* a ``# type: ignore`` comment;
|
||
|
||
* a ``@no_type_check`` decorator on a class or function;
|
||
|
||
* a custom class or function decorator marked with
|
||
``@no_type_check_decorator``.
|
||
|
||
For more details see later sections.
|
||
|
||
In order for maximal compatibility with offline type checking it may
|
||
eventually be a good idea to change interfaces that rely on annotations
|
||
to switch to a different mechanism, for example a decorator. In Python
|
||
3.5 there is no pressure to do this, however. See also the longer
|
||
discussion under `Rejected alternatives`_ below.
|
||
|
||
|
||
Type comments
|
||
=============
|
||
|
||
No first-class syntax support for explicitly marking variables as being
|
||
of a specific type is added by this PEP. To help with type inference in
|
||
complex cases, a comment of the following format may be used::
|
||
|
||
x = [] # type: List[Employee]
|
||
x, y, z = [], [], [] # type: List[int], List[int], List[str]
|
||
x, y, z = [], [], [] # type: (List[int], List[int], List[str])
|
||
x = [
|
||
1,
|
||
2,
|
||
] # type: List[int]
|
||
|
||
Type comments should be put on the last line of the statement that
|
||
contains the variable definition. They can also be placed on
|
||
``with`` statements and ``for`` statements, right after the colon.
|
||
|
||
Examples of type comments on ``with`` and ``for`` statements::
|
||
|
||
with frobnicate() as foo: # type: int
|
||
# Here foo is an int
|
||
...
|
||
|
||
for x, y in points: # type: float, float
|
||
# Here x and y are floats
|
||
...
|
||
|
||
In stubs it may be useful to declare the existence of a variable
|
||
without giving it an initial value. This can be done using a literal
|
||
ellipsis::
|
||
|
||
from typing import IO
|
||
|
||
stream = ... # type: IO[str]
|
||
|
||
In non-stub code, there is a similar special case:
|
||
|
||
from typing import IO
|
||
|
||
stream = None # type: IO[str]
|
||
|
||
Type checkers should not complain about this (despite the value
|
||
``None`` not matching the given type), nor should they change the
|
||
inferred type to ``Optional[...]`` (despite the rule that does this
|
||
for annotated arguments with a default value of ``None``). The
|
||
assumption here is that other code will ensure that the variable is
|
||
given a value of the proper type, and all uses can assume that the
|
||
variable has the given type.
|
||
|
||
The ``# type: ignore`` comment should be put on the line that the
|
||
error refers to::
|
||
|
||
import http.client
|
||
errors = {
|
||
'not_found': http.client.NOT_FOUND # type: ignore
|
||
}
|
||
|
||
A ``# type: ignore`` comment on a line by itself disables all type
|
||
checking for the rest of the file.
|
||
|
||
If type hinting proves useful in general, a syntax for typing variables
|
||
may be provided in a future Python version.
|
||
|
||
Casts
|
||
=====
|
||
|
||
Occasionally the type checker may need a different kind of hint: the
|
||
programmer may know that an expression is of a more constrained type
|
||
than a type checker may be able to infer. For example::
|
||
|
||
from typing import List, cast
|
||
|
||
def find_first_str(a: List[object]) -> str:
|
||
index = next(i for i, x in enumerate(a) if isinstance(x, str))
|
||
# We only get here if there's at least one string in a
|
||
return cast(str, a[index])
|
||
|
||
Some type checkers may not be able to infer that the type of
|
||
``a[index]`` is ``str`` and only infer ``object`` or ``Any``", but we
|
||
know that (if the code gets to that point) it must be a string. The
|
||
``cast(t, x)`` call tells the type checker that we are confident that
|
||
the type of ``x`` is ``t``. At runtime a cast always returns the
|
||
expression unchanged -- it does not check the type, and it does not
|
||
convert or coerce the value.
|
||
|
||
Casts differ from type comments (see the previous section). When using
|
||
a type comment, the type checker should still verify that the inferred
|
||
type is consistent with the stated type. When using a cast, the type
|
||
checker should blindly believe the programmer. Also, casts can be used
|
||
in expressions, while type comments only apply to assignments.
|
||
|
||
|
||
Stub Files
|
||
==========
|
||
|
||
Stub files are files containing type hints that are only for use by
|
||
the type checker, not at runtime. There are several use cases for
|
||
stub files:
|
||
|
||
* Extension modules
|
||
|
||
* Third-party modules whose authors have not yet added type hints
|
||
|
||
* Standard library modules for which type hints have not yet been
|
||
written
|
||
|
||
* Modules that must be compatible with Python 2 and 3
|
||
|
||
* Modules that use annotations for other purposes
|
||
|
||
Stub files have the same syntax as regular Python modules. There is one
|
||
feature of the ``typing`` module that may only be used in stub files:
|
||
the ``@overload`` decorator described below.
|
||
|
||
The type checker should only check function signatures in stub files;
|
||
It is recommended that function bodies in stub files just be a single
|
||
ellipsis (``...``).
|
||
|
||
The type checker should have a configurable search path for stub files.
|
||
If a stub file is found the type checker should not read the
|
||
corresponding "real" module.
|
||
|
||
While stub files are syntactically valid Python modules, they use the
|
||
``.pyi`` extension to make it possible to maintain stub files in the
|
||
same directory as the corresponding real module. This also reinforces
|
||
the notion that no runtime behavior should be expected of stub files.
|
||
|
||
Additional notes on stub files:
|
||
|
||
* Modules and variables imported into the stub are not considered
|
||
exported from the stub unless the import uses the ``import ... as
|
||
...`` form.
|
||
|
||
Function overloading
|
||
--------------------
|
||
|
||
The ``@overload`` decorator allows describing functions that support
|
||
multiple different combinations of argument types. This pattern is
|
||
used frequently in builtin modules and types. For example, the
|
||
``__getitem__()`` method of the ``bytes`` type can be described as
|
||
follows::
|
||
|
||
from typing import overload
|
||
|
||
class bytes:
|
||
...
|
||
@overload
|
||
def __getitem__(self, i: int) -> int: ...
|
||
@overload
|
||
def __getitem__(self, s: slice) -> bytes: ...
|
||
|
||
This description is more precise than would be possible using unions
|
||
(which cannot express the relationship between the argument and return
|
||
types)::
|
||
|
||
from typing import Union
|
||
|
||
class bytes:
|
||
...
|
||
def __getitem__(self, a: Union[int, slice]) -> Union[int, bytes]: ...
|
||
|
||
Another example where ``@overload`` comes in handy is the type of the
|
||
builtin ``map()`` function, which takes a different number of
|
||
arguments depending on the type of the callable::
|
||
|
||
from typing import Callable, Iterable, Iterator, Tuple, TypeVar, overload
|
||
|
||
T1 = TypeVar('T1')
|
||
T2 = TypeVar('T2)
|
||
S = TypeVar('S')
|
||
|
||
@overload
|
||
def map(func: Callable[[T1], S], iter1: Iterable[T1]) -> Iterator[S]: ...
|
||
@overload
|
||
def map(func: Callable[[T1, T2], S],
|
||
iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterator[S]: ...
|
||
# ... and we could add more items to support more than two iterables
|
||
|
||
Note that we could also easily add items to support ``map(None, ...)``::
|
||
|
||
@overload
|
||
def map(func: None, iter1: Iterable[T1]) -> Iterable[T1]: ...
|
||
@overload
|
||
def map(func: None,
|
||
iter1: Iterable[T1],
|
||
iter2: Iterable[T2]) -> Iterable[Tuple[T1, T2]]: ...
|
||
|
||
The ``@overload`` decorator may only be used in stub files. While it
|
||
would be possible to provide a multiple dispatch implementation using
|
||
this syntax, its implementation would require using
|
||
``sys._getframe()``, which is frowned upon. Also, designing and
|
||
implementing an efficient multiple dispatch mechanism is hard, which
|
||
is why previous attempts were abandoned in favor of
|
||
``functools.singledispatch()``. (See PEP 443, especially its section
|
||
"Alternative approaches".) In the future we may come up with a
|
||
satisfactory multiple dispatch design, but we don't want such a design
|
||
to be constrained by the overloading syntax defined for type hints in
|
||
stub files. In the meantime, using the ``@overload`` decorator or
|
||
calling ``overload()`` directly raises ``RuntimeError``.
|
||
|
||
A constrained ``TypeVar`` type can often be used instead of using the
|
||
``@overload`` decorator. For example, the definitions of ``concat1``
|
||
and ``concat2`` in this stub file are equivalent:
|
||
|
||
from typing import TypeVar
|
||
|
||
AnyStr = TypeVar('AnyStr', str, bytes)
|
||
|
||
def concat1(x: AnyStr, y: AnyStr) -> AnyStr: ...
|
||
|
||
@overload
|
||
def concat2(x: str, y: str) -> str: ...
|
||
@overload
|
||
def concat2(x: bytes, y: bytes) -> bytes: ...
|
||
|
||
Some functions, such as ``map`` or ``bytes.__getitem__`` above, can't
|
||
be represented precisely using type variables. However, unlike
|
||
``@overload``, type variables can also be used outside stub files. We
|
||
recommend that ``@overload`` is only used in cases where a type
|
||
variable is not sufficient, due to its special stub-only status.
|
||
|
||
Another important difference between type variables such as ``AnyStr``
|
||
and using ``@overload`` is that the prior can also be used to define
|
||
constraints for generic class type parameters. For example, the type
|
||
parameter of the generic class ``typing.IO`` is constrained (only
|
||
``IO[str]``, ``IO[bytes]`` and ``IO[Any]`` are valid):
|
||
|
||
class IO(Generic[AnyStr]): ...
|
||
|
||
Storing and distributing stub files
|
||
-----------------------------------
|
||
|
||
The easiest form of stub file storage and distribution is to put them
|
||
alongside Python modules in the same directory. This makes them easy to
|
||
find by both programmers and the tools. However, since package
|
||
maintainers are free not to add type hinting to their packages,
|
||
third-party stubs installable by ``pip`` from PyPI are also supported.
|
||
In this case we have to consider three issues: naming, versioning,
|
||
installation path.
|
||
|
||
This PEP does not provide a recommendation on a naming scheme that
|
||
should be used for third-party stub file packages. Discoverability will
|
||
hopefully be based on package popularity, like with Django packages for
|
||
example.
|
||
|
||
Third-party stubs have to be versioned using the lowest version of the
|
||
source package that is compatible. Example: FooPackage has versions
|
||
1.0, 1.1, 1.2, 1.3, 2.0, 2.1, 2.2. There are API changes in versions
|
||
1.1, 2.0 and 2.2. The stub file package maintainer is free to release
|
||
stubs for all versions but at least 1.0, 1.1, 2.0 and 2.2 are needed
|
||
to enable the end user type check all versions. This is because the
|
||
user knows that the closest *lower or equal* version of stubs is
|
||
compatible. In the provided example, for FooPackage 1.3 the user would
|
||
choose stubs version 1.1.
|
||
|
||
Note that if the user decides to use the "latest" available source
|
||
package, using the "latest" stub files should generally also work if
|
||
they're updated often.
|
||
|
||
Third-party stub packages can use any location for stub storage. Type
|
||
checkers should search for them using PYTHONPATH. A default fallback
|
||
directory that is always checked is ``shared/typehints/python3.5/`` (or
|
||
3.6, etc.). Since there can only be one package installed for a given
|
||
Python version per environment, no additional versioning is performed
|
||
under that directory (just like bare directory installs by ``pip`` in
|
||
site-packages). Stub file package authors might use the following
|
||
snippet in ``setup.py``::
|
||
|
||
...
|
||
data_files=[
|
||
(
|
||
'shared/typehints/python{}.{}'.format(*sys.version_info[:2]),
|
||
pathlib.Path(SRC_PATH).glob('**/*.pyi'),
|
||
),
|
||
],
|
||
...
|
||
|
||
The Typeshed Repo
|
||
-----------------
|
||
|
||
There is a shared repository where useful stubs are being collected
|
||
[typeshed]_. Note that stubs for a given package will not be included
|
||
here without the explicit consent of the package owner. Further
|
||
policies regarding the stubs collected here will be decided at a later
|
||
time, after discussion on python-dev, and reported in the typeshed
|
||
repo's README.
|
||
|
||
|
||
Exceptions
|
||
==========
|
||
|
||
No syntax for listing explicitly raised exceptions is proposed.
|
||
Currently the only known use case for this feature is documentational,
|
||
in which case the recommendation is to put this information in a
|
||
docstring.
|
||
|
||
|
||
The ``typing`` Module
|
||
=====================
|
||
|
||
To open the usage of static type checking to Python 3.5 as well as older
|
||
versions, a uniform namespace is required. For this purpose, a new
|
||
module in the standard library is introduced called ``typing``.
|
||
|
||
It defines the fundamental building blocks for constructing types
|
||
(e.g. ``Any``), types representing generic variants of builtin
|
||
collections (e.g. ``List``), types representing generic
|
||
collection ABCs (e.g. ``Sequence``), and a small collection of
|
||
convenience definitions.
|
||
|
||
Fundamental building blocks:
|
||
|
||
* Any, used as ``def get(key: str) -> Any: ...``
|
||
|
||
* Union, used as ``Union[Type1, Type2, Type3]``
|
||
|
||
* Callable, used as ``Callable[[Arg1Type, Arg2Type], ReturnType]``
|
||
|
||
* Tuple, used by listing the element types, for example
|
||
``Tuple[int, int, str]``.
|
||
Arbitrary-length homogeneous tuples can be expressed
|
||
using one type and ellipsis, for example ``Tuple[int, ...]``.
|
||
(The ``...`` here are part of the syntax, a literal ellipsis.)
|
||
|
||
* TypeVar, used as ``X = TypeVar('X', Type1, Type2, Type3)`` or simply
|
||
``Y = TypeVar('Y')`` (see above for more details)
|
||
|
||
* Generic, used to create user-defined generic classes
|
||
|
||
Generic variants of builtin collections:
|
||
|
||
* Dict, used as ``Dict[key_type, value_type]``
|
||
|
||
* List, used as ``List[element_type]``
|
||
|
||
* Set, used as ``Set[element_type]``. See remark for ``AbstractSet``
|
||
below.
|
||
|
||
* FrozenSet, used as ``FrozenSet[element_type]``
|
||
|
||
Note: ``Dict``, ``List``, ``Set`` and ``FrozenSet`` are mainly useful
|
||
for annotating return values. For arguments, prefer the abstract
|
||
collection types defined below, e.g. ``Mapping``, ``Sequence`` or
|
||
``AbstractSet``.
|
||
|
||
Generic variants of container ABCs (and a few non-containers):
|
||
|
||
* ByteString
|
||
|
||
* Callable (see above, listed here for completeness)
|
||
|
||
* Container
|
||
|
||
* Generator, used as ``Generator[yield_type, send_type,
|
||
return_type]``. This represents the return value of generator
|
||
functions. It is a subtype of ``Iterable`` and it has additional
|
||
type variables for the type accepted by the ``send()`` method (which
|
||
is contravariant -- a generator that accepts sending it ``Employee``
|
||
instance is valid in a context where a generator is required that
|
||
accepts sending it ``Manager`` instances) and the return type of the
|
||
generator.
|
||
|
||
* Hashable (not generic, but present for completeness)
|
||
|
||
* ItemsView
|
||
|
||
* Iterable
|
||
|
||
* Iterator
|
||
|
||
* KeysView
|
||
|
||
* Mapping
|
||
|
||
* MappingView
|
||
|
||
* MutableMapping
|
||
|
||
* MutableSequence
|
||
|
||
* MutableSet
|
||
|
||
* Sequence
|
||
|
||
* Set, renamed to ``AbstractSet``. This name change was required
|
||
because ``Set`` in the ``typing`` module means ``set()`` with
|
||
generics.
|
||
|
||
* Sized (not generic, but present for completeness)
|
||
|
||
* ValuesView
|
||
|
||
A few one-off types are defined that test for single special methods
|
||
(similar to ``Hashable`` or ``Sized``):
|
||
|
||
* Reversible, to test for ``__reversed__``
|
||
|
||
* SupportsAbs, to test for ``__abs__``
|
||
|
||
* SupportsComplex, to test for ``__complex__``
|
||
|
||
* SupportsFloat, to test for ``__float__``
|
||
|
||
* SupportsInt, to test for ``__int__``
|
||
|
||
* SupportsRound, to test for ``__round__``
|
||
|
||
* SupportsBytes, to test for ``__bytes__``
|
||
|
||
Convenience definitions:
|
||
|
||
* Optional, defined by ``Optional[t] == Union[t, type(None)]``
|
||
|
||
* AnyStr, defined as ``TypeVar('AnyStr', str, bytes)``
|
||
|
||
* NamedTuple, used as
|
||
``NamedTuple(type_name, [(field_name, field_type), ...])``
|
||
and equivalent to
|
||
``collections.namedtuple(type_name, [field_name, ...])``.
|
||
This is useful to declare the types of the fields of a a named tuple
|
||
type.
|
||
|
||
* cast(), described earlier
|
||
|
||
* @no_type_check, a decorator to disable type checking per class or
|
||
function (see below)
|
||
|
||
* @no_type_check_decorator, a decorator to create your own decorators
|
||
with the same meaning as ``@no_type_check`` (see below)
|
||
|
||
* @overload, described earlier
|
||
|
||
* get_type_hints(), a utility function to retrieve the type hints from a
|
||
function or method. Given a function or method object, it returns
|
||
a dict with the same format as ``__annotations__``, but evaluating
|
||
forward references (which are given as string literals) as expressions
|
||
in the context of the original function or method definition.
|
||
|
||
Types available in the ``typing.io`` submodule:
|
||
|
||
* IO (generic over ``AnyStr``)
|
||
|
||
* BinaryIO (a simple subtype of ``IO[bytes]``)
|
||
|
||
* TextIO (a simple subtype of ``IO[str]``)
|
||
|
||
Types available in the ``typing.re`` submodule:
|
||
|
||
* Match and Pattern, types of ``re.match()`` and ``re.compile()``
|
||
results (generic over ``AnyStr``)
|
||
|
||
|
||
Rejected Alternatives
|
||
=====================
|
||
|
||
During discussion of earlier drafts of this PEP, various objections
|
||
were raised and alternatives were proposed. We discuss some of these
|
||
here and explain why we reject them.
|
||
|
||
Several main objections were raised.
|
||
|
||
Which brackets for generic type parameters?
|
||
-------------------------------------------
|
||
|
||
Most people are familiar with the use of angular brackets
|
||
(e.g. ``List<int>``) in languages like C++, Java, C# and Swift to
|
||
express the parametrization of generic types. The problem with these
|
||
is that they are really hard to parse, especially for a simple-minded
|
||
parser like Python. In most languages the ambiguities are usually
|
||
dealt with by only allowing angular brackets in specific syntactic
|
||
positions, where general expressions aren't allowed. (And also by
|
||
using very powerful parsing techniques that can backtrack over an
|
||
arbitrary section of code.)
|
||
|
||
But in Python, we'd like type expressions to be (syntactically) the
|
||
same as other expressions, so that we can use e.g. variable assignment
|
||
to create type aliases. Consider this simple type expression::
|
||
|
||
List<int>
|
||
|
||
From the Python parser's perspective, the expression begins with the
|
||
same four tokens (NAME, LESS, NAME, GREATER) as a chained comparison::
|
||
|
||
a < b > c # I.e., (a < b) and (b > c)
|
||
|
||
We can even make up an example that could be parsed both ways::
|
||
|
||
a < b > [ c ]
|
||
|
||
Assuming we had angular brackets in the language, this could be
|
||
interpreted as either of the following two::
|
||
|
||
(a<b>)[c] # I.e., (a<b>).__getitem__(c)
|
||
a < b > ([c]) # I.e., (a < b) and (b > [c])
|
||
|
||
It would surely be possible to come up with a rule to disambiguate
|
||
such cases, but to most users the rules would feel arbitrary and
|
||
complex. It would also require us to dramatically change the CPython
|
||
parser (and every other parser for Python). It should be noted that
|
||
Python's current parser is intentionally "dumb" -- a simple grammar is
|
||
easier for users to reason about.
|
||
|
||
For all these reasons, square brackets (e.g. ``List[int]``) are (and
|
||
have long been) the preferred syntax for generic type parameters.
|
||
They can be implemented by defining the ``__getitem__()`` method on
|
||
the metaclass, and no new syntax is required at all. This option
|
||
works in all recent versions of Python (starting with Python 2.2).
|
||
Python is not alone in this syntactic choice -- generic classes in
|
||
Scala also use square brackets.
|
||
|
||
What about existing uses of annotations?
|
||
----------------------------------------
|
||
|
||
One line of argument points out that PEP 3107 explicitly supports
|
||
the use of arbitrary expressions in function annotations. The new
|
||
proposal is then considered incompatible with the specification of PEP
|
||
3107.
|
||
|
||
Our response to this is that, first of all, the current proposal does
|
||
not introduce any direct incompatibilities, so programs using
|
||
annotations in Python 3.4 will still work correctly and without
|
||
prejudice in Python 3.5.
|
||
|
||
We do hope that type hints will eventually become the sole use for
|
||
annotations, but this will require additional discussion and a
|
||
deprecation period after the initial roll-out of the typing module
|
||
with Python 3.5. The current PEP will have provisional status (see
|
||
PEP 411) until Python 3.6 is released. The fastest conceivable scheme
|
||
would introduce silent deprecation of non-type-hint annotations in
|
||
3.6, full deprecation in 3.7, and declare type hints as the only
|
||
allowed use of annotations in Python 3.8. This should give authors of
|
||
packages that use annotations plenty of time to devise another
|
||
approach, even if type hints become an overnight success.
|
||
|
||
Another possible outcome would be that type hints will eventually
|
||
become the default meaning for annotations, but that there will always
|
||
remain an option to disable them. For this purpose the current
|
||
proposal defines a decorator ``@no_type_check`` which disables the
|
||
default interpretation of annotations as type hints in a given class
|
||
or function. It also defines a meta-decorator
|
||
``@no_type_check_decorator`` which can be used to decorate a decorator
|
||
(!), causing annotations in any function or class decorated with the
|
||
latter to be ignored by the type checker.
|
||
|
||
There are also ``# type: ignore`` comments, and static checkers should
|
||
support configuration options to disable type checking in selected
|
||
packages.
|
||
|
||
Despite all these options, proposals have been circulated to allow
|
||
type hints and other forms of annotations to coexist for individual
|
||
arguments. One proposal suggests that if an annotation for a given
|
||
argument is a dictionary literal, each key represents a different form
|
||
of annotation, and the key ``'type'`` would be use for type hints.
|
||
The problem with this idea and its variants is that the notation
|
||
becomes very "noisy" and hard to read. Also, in most cases where
|
||
existing libraries use annotations, there would be little need to
|
||
combine them with type hints. So the simpler approach of selectively
|
||
disabling type hints appears sufficient.
|
||
|
||
The problem of forward declarations
|
||
-----------------------------------
|
||
|
||
The current proposal is admittedly sub-optimal when type hints must
|
||
contain forward references. Python requires all names to be defined
|
||
by the time they are used. Apart from circular imports this is rarely
|
||
a problem: "use" here means "look up at runtime", and with most
|
||
"forward" references there is no problem in ensuring that a name is
|
||
defined before the function using it is called.
|
||
|
||
The problem with type hints is that annotations (per PEP 3107, and
|
||
similar to default values) are evaluated at the time a function is
|
||
defined, and thus any names used in an annotation must be already
|
||
defined when the function is being defined. A common scenario is a
|
||
class definition whose methods need to reference the class itself in
|
||
their annotations. (More general, it can also occur with mutually
|
||
recursive classes.) This is natural for container types, for
|
||
example::
|
||
|
||
class Node:
|
||
"""Binary tree node."""
|
||
|
||
def __init__(self, left: Node, right: None):
|
||
self.left = left
|
||
self.right = right
|
||
|
||
As written this will not work, because of the peculiarity in Python
|
||
that class names become defined once the entire body of the class has
|
||
been executed. Our solution, which isn't particularly elegant, but
|
||
gets the job done, is to allow using string literals in annotations.
|
||
Most of the time you won't have to use this though -- most *uses* of
|
||
type hints are expected to reference builtin types or types defined in
|
||
other modules.
|
||
|
||
A counterproposal would change the semantics of type hints so they
|
||
aren't evaluated at runtime at all (after all, type checking happens
|
||
off-line, so why would type hints need to be evaluated at runtime at
|
||
all). This of course would run afoul of backwards compatibility,
|
||
since the Python interpreter doesn't actually know whether a
|
||
particular annotation is meant to be a type hint or something else.
|
||
|
||
A compromise is possible where a ``__future__`` import could enable
|
||
turning *all* annotations in a given module into string literals, as
|
||
follows::
|
||
|
||
from __future__ import annotations
|
||
|
||
class ImSet:
|
||
def add(self, a: ImSet) -> List[ImSet]: ...
|
||
|
||
assert ImSet.add.__annotations__ == {'a': 'ImSet', 'return': 'List[ImSet]'}
|
||
|
||
Such a ``__future__`` import statement may be proposed in a separate
|
||
PEP.
|
||
|
||
|
||
The double colon
|
||
----------------
|
||
|
||
A few creative souls have tried to invent solutions for this problem.
|
||
For example, it was proposed to use a double colon (``::``) for type
|
||
hints, solving two problems at once: disambiguating between type hints
|
||
and other annotations, and changing the semantics to preclude runtime
|
||
evaluation. There are several things wrong with this idea, however.
|
||
|
||
* It's ugly. The single colon in Python has many uses, and all of
|
||
them look familiar because they resemble the use of the colon in
|
||
English text. This is a general rule of thumb by which Python
|
||
abides for most forms of punctuation; the exceptions are typically
|
||
well known from other programming languages. But this use of ``::``
|
||
is unheard of in English, and in other languages (e.g. C++) it is
|
||
used as a scoping operator, which is a very different beast. In
|
||
contrast, the single colon for type hints reads naturally -- and no
|
||
wonder, since it was carefully designed for this purpose (the idea
|
||
long predates PEP 3107 [gvr-artima]_). It is also used in the same
|
||
fashion in other languages from Pascal to Swift.
|
||
|
||
* What would you do for return type annotations?
|
||
|
||
* It's actually a feature that type hints are evaluated at runtime.
|
||
|
||
* Making type hints available at runtime allows runtime type
|
||
checkers to be built on top of type hints.
|
||
|
||
* It catches mistakes even when the type checker is not run. Since
|
||
it is a separate program, users may choose not to run it (or even
|
||
install it), but might still want to use type hints as a concise
|
||
form of documentation. Broken type hints are no use even for
|
||
documentation.
|
||
|
||
* Because it's new syntax, using the double colon for type hints would
|
||
limit them to code that works with Python 3.5 only. By using
|
||
existing syntax, the current proposal can easily work for older
|
||
versions of Python 3. (And in fact mypy supports Python 3.2 and
|
||
newer.)
|
||
|
||
* If type hints become successful we may well decide to add new syntax
|
||
in the future to declare the type for variables, for example
|
||
``var age: int = 42``. If we were to use a double colon for
|
||
argument type hints, for consistency we'd have to use the same
|
||
convention for future syntax, perpetuating the ugliness.
|
||
|
||
Other forms of new syntax
|
||
-------------------------
|
||
|
||
A few other forms of alternative syntax have been proposed, e.g. the
|
||
introduction of a ``where`` keyword [roberge]_, and Cobra-inspired
|
||
``requires`` clauses. But these all share a problem with the double
|
||
colon: they won't work for earlier versions of Python 3. The same
|
||
would apply to a new ``__future__`` import.
|
||
|
||
Other backwards compatible conventions
|
||
--------------------------------------
|
||
|
||
The ideas put forward include:
|
||
|
||
* A decorator, e.g. ``@typehints(name=str, returns=str)``. This could
|
||
work, but it's pretty verbose (an extra line, and the argument names
|
||
must be repeated), and a far cry in elegance from the PEP 3107
|
||
notation.
|
||
|
||
* Stub files. We do want stub files, but they are primarily useful
|
||
for adding type hints to existing code that doesn't lend itself to
|
||
adding type hints, e.g. 3rd party packages, code that needs to
|
||
support both Python 2 and Python 3, and especially extension
|
||
modules. For most situations, having the annotations in line with
|
||
the function definitions makes them much more useful.
|
||
|
||
* Docstrings. There is an existing convention for docstrings, based
|
||
on the Sphinx notation (``:type arg1: description``). This is
|
||
pretty verbose (an extra line per parameter), and not very elegant.
|
||
We could also make up something new, but the annotation syntax is
|
||
hard to beat (because it was designed for this very purpose).
|
||
|
||
It's also been proposed to simply wait another release. But what
|
||
problem would that solve? It would just be procrastination.
|
||
|
||
|
||
PEP Development Process
|
||
=======================
|
||
|
||
A live draft for this PEP lives on GitHub [github]_. There is also an
|
||
issue tracker [issues]_, where much of the technical discussion takes
|
||
place.
|
||
|
||
The draft on GitHub is updated regularly in small increments. The
|
||
official PEPS repo [peps_] is (usually) only updated when a new draft
|
||
is posted to python-dev.
|
||
|
||
|
||
Acknowledgements
|
||
================
|
||
|
||
This document could not be completed without valuable input,
|
||
encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson
|
||
Vitousek, Andrey Vlasovskikh, Radomir Dopieralski, Peter Ludemann,
|
||
and the BDFL-Delegate, Mark Shannon.
|
||
|
||
Influences include existing languages, libraries and frameworks
|
||
mentioned in PEP 482. Many thanks to their creators, in alphabetical
|
||
order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings,
|
||
Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer,
|
||
Raoul-Gabriel Urma, and Julien Verlaguet.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [mypy]
|
||
http://mypy-lang.org
|
||
|
||
.. [gvr-artima]
|
||
http://www.artima.com/weblogs/viewpost.jsp?thread=85551
|
||
|
||
.. [wiki-variance]
|
||
http://en.wikipedia.org/wiki/Covariance_and_contravariance_%28computer_science%29
|
||
|
||
.. [typeshed]
|
||
https://github.com/JukkaL/typeshed/
|
||
|
||
.. [pyflakes]
|
||
https://github.com/pyflakes/pyflakes/
|
||
|
||
.. [pylint]
|
||
http://www.pylint.org
|
||
|
||
.. [roberge]
|
||
http://aroberge.blogspot.com/2015/01/type-hinting-in-python-focus-on.html
|
||
|
||
.. [github]
|
||
https://github.com/ambv/typehinting
|
||
|
||
.. [issues]
|
||
https://github.com/ambv/typehinting/issues
|
||
|
||
.. [peps]
|
||
https://hg.python.org/peps/file/tip/pep-0484.txt
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|