PEP 563: Postponed Evaluation of Annotations

Draft 0.  PEP number assigned by GvR in
77557db51c (commitcomment-24210467)

pep-0563.rst

@@ -0,0 +1,264 @@
+PEP: 563
+Title: Postponed Evaluation of Annotations
+Version: $Revision$
+Last-Modified: $Date$
+Author: Łukasz Langa <lukasz@langa.pl>
+Discussions-To: Python-Dev <python-dev@python.org>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 8-Sep-2017
+Python-Version: 3.7
+Post-History:
+Resolution:
+
+
+Abstract
+========
+
+PEP 3107 introduced syntax for function annotations, but the semantics
+were deliberately left undefined.  PEP 484 introduced a standard meaning
+to annotations: type hints.  PEP 526 defined variable annotations,
+explicitly tying them with the type hinting use case.
+
+This PEP proposes changing function annotations and variable annotations
+so that they are no longer evaluated at function definition time.
+Instead, they are preserved in ``__annotations__`` in string form.
+
+This change is going to be introduced gradually, starting with a new
+``__future__`` import in Python 3.7.
+
+
+Rationale and Goals
+===================
+
+PEP 3107 added support for arbitrary annotations on parts of a function
+definition.  Just like default values, annotations are evaluated at
+function definition time.  This creates a number of issues for the type
+hinting use case:
+
+* forward references: when a type hint contains names that have not been
+  defined yet, that definition needs to be expressed as a string
+  literal;
+
+* type hints are executed at module import time, which is not
+  computationally free.
+
+Postponing the evaluation of annotations solves both problems.
+
+Non-goals
+---------
+
+Just like in PEP 484 and PEP 526, it should be emphasized that **Python
+will remain a dynamically typed language, and the authors have no desire
+to ever make type hints mandatory, even by convention.**
+
+Annotations are still available for arbitrary use besides type checking.
+Using ``@typing.no_type_hints`` in this case is recommended to
+disambiguate the use case.
+
+
+Implementation
+==============
+
+In a future version of Python, function and variable annotations will no
+longer be evaluated at definition time.  Instead, a string form will be
+preserved in the respective ``__annotations__`` dictionary.  Static type
+checkers will see no difference in behavior, whereas tools using
+annotations at runtime will have to perform postponed evaluation.
+
+If an annotation was already a string, this string is preserved
+verbatim.  In other cases, the string form is obtained from the AST
+during the compilation step, which means that the string form preserved
+might not preserve the exact formatting of the source.
+
+Annotations need to be syntactically valid Python expressions, also when
+passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
+Annotations can only use names present in the module scope as postponed
+evaluation using local names is not reliable.
+
+Note that as per PEP 526, local variable annotations are not evaluated
+at all since they are not accessible outside of the function's closure.
+
+Enabling the future behavior in Python 3.7
+------------------------------------------
+
+The functionality described above can be enabled starting from Python
+3.7 using the following special import::
+
+    from __future__ import annotations
+
+
+Resolving Type Hints at Runtime
+===============================
+
+To resolve an annotation at runtime from its string form to the result
+of the enclosed expression, user code needs to evaluate the string.
+
+For code that uses type hints, the ``typing.get_type_hints()`` function
+correctly evaluates expressions back from its string form.  Note that
+all valid code currently using ``__annotations__`` should already be
+doing that since a type annotation can be expressed as a string literal.
+
+For code which uses annotations for other purposes, a regular
+``eval(ann, globals, locals)`` call is enough to resolve the
+annotation.  The trick here is to get the correct value for globals.
+Fortunately, in the case of functions, they hold a reference to globals
+in an attribute called ``__globals__``.  To get the correct module-level
+context to resolve class variables, use::
+
+    cls_globals = sys.modules[SomeClass.__module__].__dict__
+
+Runtime annotation resolution and class decorators
+--------------------------------------------------
+
+Metaclasses and class decorators that need to resolve annotations for
+the current class will fail for annotations that use the name of the
+current class.  Example::
+
+    def class_decorator(cls):
+        annotations = get_type_hints(cls)  # raises NameError on 'C'
+        print(f'Annotations for {cls}: {annotations}')
+        return cls
+
+    @class_decorator
+    class C:
+        singleton: 'C' = None
+
+This was already true before this PEP.  The class decorator acts on
+the class before it's assigned a name in the current definition scope.
+
+The situation is made somewhat stricter when class-level variables are
+considered.  Previously, when the string form wasn't used in annotations,
+a class decorator would be able to cover situations like::
+
+    @class_decorator
+    class Restaurant:
+        class MenuOption(Enum):
+            SPAM = 1
+            EGGS = 2
+
+        default_menu: List[MenuOption] = []
+
+This is no longer possible.
+
+Runtime annotation resolution and ``TYPE_CHECKING``
+---------------------------------------------------
+
+Sometimes there's code that must be seen by a type checker but should
+not be executed.  For such situations the ``typing`` module defines a
+constant, ``TYPE_CHECKING``, that is considered ``True`` during type
+checking but ``False`` at runtime.  Example::
+
+  import typing
+
+  if typing.TYPE_CHECKING:
+      import expensive_mod
+
+  def a_func(arg: expensive_mod.SomeClass) -> None:
+      a_var: expensive_mod.SomeClass = arg
+      ...
+
+This approach is also useful when handling import cycles.
+
+Trying to resolve annotations of ``a_func`` at runtime using
+``typing.get_type_hints()`` will fail since the name ``expensive_mod``
+is not defined (``TYPE_CHECKING`` variable being ``False`` at runtime).
+This was already true before this PEP.
+
+
+Backwards Compatibility
+=======================
+
+This is a backwards incompatible change.  Applications depending on
+arbitrary objects to be directly present in annotations will break
+if they are not using ``typing.get_type_hints()`` or ``eval()``.
+
+Annotations that depend on locals at the time of the function/class
+definition are now invalid.  Example::
+
+    def generate_class():
+        some_local = datetime.datetime.now()
+        class C:
+            field: some_local = 1  # NOTE: INVALID ANNOTATION
+            def method(self, arg: some_local.day) -> None:  # NOTE: INVALID ANNOTATION
+                ...
+
+Annotations using nested classes and their respective state are still
+valid, provided they use the fully qualified name.  Example::
+
+    class C:
+        field = 'c_field'
+        def method(self, arg: C.field) -> None:  # this is OK
+            ...
+
+        class D:
+            field2 = 'd_field'
+            def method(self, arg: C.field -> C.D.field2:  # this is OK
+                ...
+
+In the presence of an annotation that cannot be resolved using the
+current module's globals, a NameError is raised at compile time.
+
+
+Deprecation policy
+------------------
+
+In Python 3.7, a ``__future__`` import is required to use the described
+functionality and a ``PendingDeprecationWarning`` is raised by the
+compiler in the presence of type annotations in modules without the
+``__future__`` import.  In Python 3.8 the warning becomes a
+``DeprecationWarning``.  In the next version this will become the
+default behavior.
+
+
+Rejected Ideas
+==============
+
+Keep the ability to use local state when defining annotations
+-------------------------------------------------------------
+
+With postponed evaluation, this is impossible for function locals.  For
+classes, it would be possible to keep the ability to define annotations
+using the local scope.  However, when using ``eval()`` to perform the
+postponed evaluation, we need to provide the correct globals and locals
+to the ``eval()`` call.  In the face of nested classes, the routine to
+get the effective "globals" at definition time would have to look
+something like this::
+
+    def get_class_globals(cls):
+        result = {}
+        result.update(sys.modules[cls.__module__].__dict__)
+        for child in cls.__qualname__.split('.'):
+            result.update(result[child].__dict__)
+        return result
+
+This is brittle and doesn't even cover slots.  Requiring the use of
+module-level names simplifies runtime evaluation and provides the
+"one obvious way" to read annotations.  It's the equivalent of absolute
+imports.
+
+
+Acknowledgements
+================
+
+This document could not be completed without valuable input,
+encouragement and advice from Guido van Rossum, Jukka Lehtosalo, and
+Ivan Levkivskyi.
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End: