PEP: 484 Title: Type Hints Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum , Jukka Lehtosalo , Łukasz Langa Discussions-To: Python-Dev Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Sep-2014 Post-History: 16-Jan-2015 Resolution: Abstract ======== This PEP introduces a standard syntax for type hints using annotations on function definitions. The proposal is strongly inspired by mypy [mypy]_. The theory behind type hints and gradual typing is explained in PEP 483. Rationale and Goals =================== PEP 3107 added support for arbitrary annotations on parts of a function definition. Although no meaning was assigned to annotations then, there has always been an implicit goal to use them for type hinting, which is listed as the first possible use case in said PEP. This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and performance optimizations utilizing type information. Type Definition Syntax ====================== The syntax leverages PEP 3107-style annotations with a number of extensions described in sections below. In its basic form, type hinting is used by filling function annotations with classes:: def greeting(name: str) -> str: return 'Hello ' + name This denotes that the expected type of the ``name`` argument is ``str``. Analogically, the expected return type is ``str``. Subclasses of a specified argument type are also accepted as valid types for that argument. Abstract base classes, types available in the ``types`` module, and user-defined classes may be used as type hints as well. Annotations must be valid expressions that evaluate without raising exceptions at the time the function is defined. In addition, the needs of static analysis require that annotations must be simple enough to be interpreted by static analysis tools. (This is an intentionally somewhat vague requirement.) .. FIXME: Define rigorously what is/isn't supported. When used as an annotation, the expression ``None`` is considered equivalent to ``NoneType`` (i.e., ``type(None)`` for type hinting purposes. Type aliases are also valid type hints:: integer = int def retry(url: str, retry_count: integer): ... New names that are added to support features described in following sections are available in the ``typing`` package. Callbacks --------- Frameworks expecting callback functions of specific signatures might be type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. Examples:: from typing import Any, AnyArgs, Callable def feeder(get_next_item: Callable[[], Item]): ... def async_query(on_success: Callable[[int], None], on_error: Callable[[int, Exception], None]): ... def partial(func: Callable[AnyArgs, Any], *args): ... Since using callbacks with keyword arguments is not perceived as a common use case, there is currently no support for specifying keyword arguments with ``Callable``. Generics -------- Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. Example:: from typing import Mapping, Set def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]): ... Generics can be parametrized by using a new factory available in ``typing`` called ``TypeVar``. Example:: from typing import Sequence, TypeVar T = TypeVar('T') # Declare type variable def first(l: Sequence[T]) -> T: # Generic function return l[0] In this case the contract is that the returning value is consistent with the elements held by the collection. ``TypeVar`` supports constraining parametric types to classes with any of the specified bases. Example:: from typing import Iterable X = TypeVar('X') Y = TypeVar('Y', Iterable[X]) def filter(rule: Callable[[X], bool], input: Y) -> Y: ... .. FIXME: Add an example with multiple bases defined. In the example above we specify that ``Y`` can be any subclass of Iterable with elements of type ``X``, as long as the return type of ``filter()`` will be the same as the type of the ``input`` argument. .. FIXME: Explain more about how this works. Forward references ------------------ When a type hint contains names that have not been defined yet, that definition may be expressed as a string, to be resolved later. For example, instead of writing:: def notify_by_email(employees: Set[Employee]): ... one might write:: def notify_by_email(employees: 'Set[Employee]'): ... .. FIXME: Rigorously define this. Defend it, or find an alternative. Union types ----------- Since accepting a small, limited set of expected types for a single argument is common, there is a new special factory called ``Union``. Example:: from typing import Union def handle_employees(e: Union[Employee, Sequence[Employee]]): if isinstance(e, Employee): e = [e] ... A type factored by ``Union[T1, T2, ...]`` responds ``True`` to ``issubclass`` checks for ``T1`` and any of its subclasses, ``T2`` and any of its subclasses, and so on. One common case of union types are *optional* types. By default, ``None`` is an invalid value for any type, unless a default value of ``None`` has been provided in the function definition. Examples:: def handle_employee(e: Union[Employee, None]): ... As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``; for example, the above is equivalent to:: from typing import Optional def handle_employee(e: Optional[Employee]): ... An optional type is also automatically assumed when the default value is ``None``, for example:: def handle_employee(e: Employee = None): ... This is equivalent to:: def handle_employee(e: Optional[Employee] = None): ... .. FIXME: Is this really a good idea? A special kind of union type is ``Any``, a class that responds ``True`` to ``issubclass`` of any class. This lets the user explicitly state that there are no constraints on the type of a specific argument or return value. Platform-specific type checking ------------------------------- In some cases the typing information will depend on the platform that the program is being executed on. To enable specifying those differences, simple conditionals can be used:: from typing import PY2, WINDOWS if PY2: text = unicode else: text = str def f() -> text: ... if WINDOWS: loop = ProactorEventLoop else: loop = UnixSelectorEventLoop Arbitrary literals defined in the form of ``NAME = True`` will also be accepted by the type checker to differentiate type resolution:: DEBUG = False ... if DEBUG: class Tracer: else: class Tracer: For the purposes of type hinting, the type checker assumes ``__debug__`` is set to ``True``, in other words the ``-O`` command-line option is not used while type checking. Compatibility with other uses of function annotations ----------------------------------------------------- A number of existing or potential use cases for function annotations exist, which are incompatible with type hinting. These may confuse a static type checker. However, since type hinting annotations have no run time behavior (other than evaluation of the annotation expression and storing annotations in the ``__annotations__`` attribute of the function object), this does not make the program incorrect -- it just makes it issue warnings when a static analyzer is used. To mark portions of the program that should not be covered by type hinting, use the following: * a ``@no_type_checks`` decorator on classes and functions * a ``# type: ignore`` comment on arbitrary lines .. FIXME: should we have a module-wide comment as well? Type Hints on Local and Global Variables ======================================== No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP. To help with type inference in complex cases, a comment of the following format may be used:: x = [] # type: List[Employee] In the case where type information for a local variable is needed before if was declared, an ``Undefined`` placeholder might be used:: from typing import Undefined x = Undefined # type: List[Employee] y = Undefined(int) If type hinting proves useful in general, a syntax for typing variables may be provided in a future Python version. Explicit raised exceptions ========================== No support for listing explicitly raised exceptions is being defined by this PEP. Currently the only known use case for this feature is documentational, in which case the recommendation is to put this information in a docstring. The ``typing`` package ====================== To open the usage of static type checking to Python 3.5 as well as older versions, a uniform namespace is required. For this purpose, a new package in the standard library is introduced called ``typing``. It holds a set of classes representing builtin types with generics, namely: * Dict, used as ``Dict[key_type, value_type]`` * List, used as ``List[element_type]`` * Set, used as ``Set[element_type]``. See remark for ``AbstractSet`` below. * FrozenSet, used as ``FrozenSet[element_type]`` * Tuple, used as ``Tuple[index0_type, index1_type, ...]``. Arbitrary-length tuples might be expressed using ellipsis, in which case the following arguments are considered the same type as the last defined type on the tuple. It also introduces factories and helper members needed to express generics and union types: * Any, used as ``def get(key: str) -> Any: ...`` * Union, used as ``Union[Type1, Type2, Type3]`` * TypeVar, used as ``X = TypeVar('X', Type1, Type2, Type3)`` or simply ``Y = TypeVar('Y')`` * Undefined, used as ``local_variable = Undefined # type: List[int]`` or ``local_variable = Undefined(List[int])`` (the latter being slower during runtime) * Callable, used as ``Callable[[Arg1Type, Arg2Type], ReturnType]`` * AnyArgs, used as ``Callable[AnyArgs, ReturnType]`` * AnyStr, equivalent to ``TypeVar('AnyStr', str, bytes)`` All abstract base classes available in ``collections.abc`` are importable from the ``typing`` package, with added generics support: * ByteString * Callable * Container * Hashable * ItemsView * Iterable * Iterator * KeysView * Mapping * MappingView * MutableMapping * MutableSequence * MutableSet * Sequence * Set as ``AbstractSet``. This name change was required because ``Set`` in the ``typing`` module means ``set()`` with generics. * Sized * ValuesView * Mapping The library includes literals for platform-specific type hinting: * PY2 * PY3, equivalent to ``not PY2`` * WINDOWS * UNIXOID, equivalent to ``not WINDOWS`` The following types are available in the ``typing.io`` module: * IO * BinaryIO * TextIO The following types are provided by the ``typing.re`` module: * Match and Pattern, types of ``re.match()`` and ``re.compile()`` results As a convenience measure, types from ``typing.io`` and ``typing.re`` are also available in ``typing`` (quoting Guido, "There's a reason those modules have two-letter names."). The place of the ``typing`` module in the standard library ---------------------------------------------------------- .. FIXME: complete this section Usage Patterns ============== The main use case of type hinting is static analysis using an external tool without executing the analyzed program. Existing tools used for that purpose like ``pyflakes`` [pyflakes]_ or ``pylint`` [pylint]_ might be extended to support type checking. New tools, like mypy's ``mypy -S`` mode, can be adopted specifically for this purpose. Type checking based on type hints is understood as a best-effort mechanism. In other words, whenever types are not annotated and cannot be inferred, the type checker considers such code valid. Type errors are only reported in case of explicit or inferred conflict. Moreover, as a mechanism that is not tied to execution of the code, it does not affect runtime behaviour. In other words, even in the case of a typing error, the program will continue running. The implementation of a type checker, whether linting source files or enforcing type information during runtime, is out of scope for this PEP. .. FIXME: Describe stub modules. .. FIXME: Describe run-time behavior of generic types. Existing Approaches =================== PEP 482 lists existing approaches in Python and other languages. Is type hinting Pythonic? ========================= Type annotations provide important documentation for how a unit of code should be used. Programmers should therefore provide type hints on public APIs, namely argument and return types on functions and methods considered public. However, because types of local and global variables can be often inferred, they are rarely necessary. The kind of information that type hints hold has always been possible to achieve by means of docstrings. In fact, a number of formalized mini-languages for describing accepted arguments have evolved. Moving this information to the function declaration makes it more visible and easier to access both at runtime and by static analysis. Adding to that the notion that “explicit is better than implicit”, type hints are indeed *Pythonic*. Acknowledgements ================ This document could not be completed without valuable input, encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson Vitousek, Andrey Vlasovskikh, and Radomir Dopieralski. Influences include existing languages, libraries and frameworks mentioned in PEP 482. Many thanks to their creators, in alphabetical order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer, Raoul-Gabriel Urma, and Julien Verlaguet. References ========== .. [mypy] http://mypy-lang.org .. [pyflakes] https://github.com/pyflakes/pyflakes/ .. [pylint] http://www.pylint.org Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: