python-peps/pep-0585.rst

PEP: 585
Title: Type Hinting Usability Conventions
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa <lukasz@python.org>
Discussions-To: Python-Dev <python-dev@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 03-Mar-2019
Python-Version: 3.8

Status of this PEP
==================

The draft of this PEP is not yet complete.  I am shamelessly squatting
on the PEP number which provides a cute relation to the original PEP 484.
The draft will be completed in the upcoming days.

Abstract
========

Static typing as defined by PEPs 484, 526, 544, 560, and 563 was built
incrementally on top of the existing Python runtime and constrained by
existing syntax and runtime behavior. For this reason, its usability is
lacking and some parts of typing necessarily feel like an afterthought.

This PEP addresses some of the major complaints of typing users, namely:

* the necessity for programmers to perform import book-keeping of names
  only used in static typing contexts;
* the surprising placement of runtime collections in the typing module
  (ABCs and ``NamedTuple``);
* the surprising dichotomy between ``List`` and ``list``, and so on;
* parts of static typing still performed at runtime (aliasing, cast,
  ``NewType``, ``TypeVar``).

Rationale and Goals
===================

The overarching goal of this PEP is to make static typing fully free of
runtime side effects. In other words, no operations related to the
process of annotating arguments, return values, and variables with types
should generate runtime behavior which is otherwise useless at runtime.

Backwards compatibility
=======================

This PEP is fully backwards compatible.  Code written in previous ways
might trigger some deprecations but will ultimately work as intended.

The newly described functionality requires Python 3.7 (for uses of
the "annotations" future-import) or Python 3.8 (for refactorings of the
``typing`` module).

Tooling, including type checkers and linters, will have to be adapted to
enable the new functionality.

Implementation
==============

Syntactic support for generics on builtin types within annotations
------------------------------------------------------------------

Starting with Python 3.7, when ``from __future__ import annotations`` is
used, function and variable annotations can specify generics directly on
builtin types. Example::

    from __future__ import annotations

    def find(haystack: dict[str, list[int]]) -> int:
        ...

This new way is preferred, the names ``List``, ``Dict``, ``FrozenSet``,
``Set`` are deprecated. They won't be removed from the ``typing`` module
for backwards compatibility but type checkers may warn about them in
future versions when used in conjunction with the "annotations" future
import.

Note: no runtime component is added to builtin collections to facilitate
generics in any sense. This syntax is only supported in an annotation.

Importing of typing
-------------------

Starting with Python 3.7, when ``from __future__ import annotations`` is
used, function and variable annotations can use special names from the
``typing`` module without the relevant explicit imports being present
in the module.

Example::

    from __future__ import annotations

    def loads(
        input: Union[str, bytes], *, encoding: Optional[str] = None
    ) -> dict[str, Any]:
        ...

Runtime collections in typing
-----------------------------

All abstract base classes redefined in the typing module are being moved
back to ``collections.abc`` including all additional functionality they
gained in the typing module (in particular, generics support). The
``Generic`` type is also moved to ``collections.abc``.

``typing.NamedTuple`` is also moved to ``collections``.

Aliases for all moved names will remain in the `typing` module for
backwards compatibility. Using them directly becomes deprecated.

Moving the remaining runtime syntax for typing-related functionality to annotations
-----------------------------------------------------------------------------------

Aliasing, cast, ``NewType``, and ``TypeVar`` require definitions which
have a runtime effect. This means they require importing names from
typing, cannot support forward references, and have negative (even if
minimal) effect on runtime performance.

New syntax for those looks like this::

    FBID: NewType[int]
    some_fbid: Cast[FBID] = some_int_from_db
    Inbox: Alias[dict[FBID, list[Message]]]
    T: TypeVar
    XXX: How to bind in TypeVar?

All of the above use the variable annotation syntax, removing the
runtime component from the functionality. In the case of NewType and
TypeVar, they additionally remove the necessity to repeat yourself with
the name of the type variable.

Rejected alternatives
=====================

Do nothing
----------

The usability issues described in the abstract are increasingly visible
when a codebase adopts type hinting holistically.  The need to jump
between the type the programmer is just describing and imports needed to
describe the type breaks the flow of thought.  The need to import
lookalike built-in collections for generics within annotations is a
kludge which makes it harder to teach Python and looks inelegant.  The
remaining runtime component, even with use of the "annotations"
future-import, impacts startup performance of annotated applications.