[pep 563] Second public draft to be discussed on python-dev.
This commit is contained in:
Lukasz Langa
parent
0c8ec1a7da
commit
651083fb6f
176
pep-0563.rst
176
pep-0563.rst
|
|
@ -70,10 +70,13 @@ checking, it is worth mentioning that the design of this PEP, as well
|
|||
as its precursors (PEP 484 and PEP 526), is predominantly motivated by
|
||||
the type hinting use case.
|
||||
|
||||
With Python 3.7, PEP 484 graduates from provisional status. Other
|
||||
In Python 3.8 PEP 484 will graduate from provisional status. Other
|
||||
enhancements to the Python programming language like PEP 544, PEP 557,
|
||||
or PEP 560, are already being built on this basis as they depend on
|
||||
type annotations and the ``typing`` module as defined by PEP 484.
|
||||
In fact, the reason PEP 484 is staying provisional in Python 3.7 is to
|
||||
enable rapid evolution for another release cycle that some of the
|
||||
aforementioned enhancements require.
|
||||
|
||||
With this in mind, uses for annotations incompatible with the
|
||||
aforementioned PEPs should be considered deprecated.
|
||||
|
|
@ -88,10 +91,10 @@ in the respective ``__annotations__`` dictionary. Static type checkers
|
|||
will see no difference in behavior, whereas tools using annotations at
|
||||
runtime will have to perform postponed evaluation.
|
||||
|
||||
If an annotation was already a string, this string is preserved
|
||||
verbatim. In other cases, the string form is obtained from the AST
|
||||
during the compilation step, which means that the string form preserved
|
||||
might not preserve the exact formatting of the source.
|
||||
The string form is obtained from the AST during the compilation step,
|
||||
which means that the string form might not preserve the exact formatting
|
||||
of the source. Note: if an annotation was a string literal already, it
|
||||
will still be wrapped in a string.
|
||||
|
||||
Annotations need to be syntactically valid Python expressions, also when
|
||||
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
|
||||
|
|
@ -110,6 +113,9 @@ The functionality described above can be enabled starting from Python
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
A reference implementation of this functionality is available
|
||||
`on GitHub <https://github.com/ambv/cpython/tree/string_annotations>`_.
|
||||
|
||||
|
||||
Resolving Type Hints at Runtime
|
||||
===============================
|
||||
|
|
@ -129,10 +135,10 @@ annotation.
|
|||
|
||||
In both cases it's important to consider how globals and locals affect
|
||||
the postponed evaluation. An annotation is no longer evaluated at the
|
||||
time of definition and, more importantly, *in the same scope* it was
|
||||
defined. Consequently, using local state in annotations is no longer
|
||||
possible in general. As for globals, the module where the annotation
|
||||
was defined is the correct context for postponed evaluation.
|
||||
time of definition and, more importantly, *in the same scope* where it
|
||||
was defined. Consequently, using local state in annotations is no
|
||||
longer possible in general. As for globals, the module where the
|
||||
annotation was defined is the correct context for postponed evaluation.
|
||||
|
||||
The ``get_type_hints()`` function automatically resolves the correct
|
||||
value of ``globalns`` for functions and classes. It also automatically
|
||||
|
|
@ -236,17 +242,33 @@ valid. They can use local names or the fully qualified name. Example::
|
|||
|
||||
class C:
|
||||
field = 'c_field'
|
||||
def method(self, arg: C.field) -> None: # this is OK
|
||||
def method(self) -> C.field: # this is OK
|
||||
...
|
||||
|
||||
def method(self) -> field: # this is OK
|
||||
...
|
||||
|
||||
def method(self) -> C.D: # this is OK
|
||||
...
|
||||
|
||||
def method(self) -> D: # this is OK
|
||||
...
|
||||
|
||||
class D:
|
||||
field2 = 'd_field'
|
||||
def method(self, arg: C.field) -> C.D.field2: # this is OK
|
||||
def method(self) -> C.D.field2: # this is OK
|
||||
...
|
||||
|
||||
def method(self, arg: field) -> D.field2: # this is OK
|
||||
def method(self) -> D.field2: # this is OK
|
||||
...
|
||||
|
||||
def method(self) -> field2: # this is OK
|
||||
...
|
||||
|
||||
def method(self) -> field: # this FAILS, class D doesn't
|
||||
... # see C's attributes, This was
|
||||
# already true before this PEP.
|
||||
|
||||
In the presence of an annotation that isn't a syntactically valid
|
||||
expression, SyntaxError is raised at compile time. However, since names
|
||||
aren't resolved at that time, no attempt is made to validate whether
|
||||
|
|
@ -284,22 +306,23 @@ forward references with string literals. The list includes:
|
|||
|
||||
* type definitions::
|
||||
|
||||
T = TypeVar('T', bound='UserId')
|
||||
UserId = NewType('UserId', 'SomeType')
|
||||
Employee = NamedTuple('Employee', [('name', str), ('id', 'UserId')])
|
||||
T = TypeVar('T', bound='<type>')
|
||||
UserId = NewType('UserId', '<type>')
|
||||
Employee = NamedTuple('Employee', [('name', '<type>', ('id', '<type>')])
|
||||
|
||||
* aliases::
|
||||
|
||||
Alias = Optional['SomeType']
|
||||
AnotherAlias = Union['SomeType', 'OtherType']
|
||||
Alias = Optional['<type>']
|
||||
AnotherAlias = Union['<type>', '<type>']
|
||||
YetAnotherAlias = '<type>'
|
||||
|
||||
* casting::
|
||||
|
||||
cast('SomeType', value)
|
||||
cast('<type>', value)
|
||||
|
||||
* base classes::
|
||||
|
||||
class C(Tuple['SomeType', 'OtherType']): ...
|
||||
class C(Tuple['<type>', '<type>']): ...
|
||||
|
||||
Depending on the specific case, some of the cases listed above might be
|
||||
worked around by placing the usage in a ``if TYPE_CHECKING:`` block.
|
||||
|
|
@ -316,8 +339,8 @@ language and is out of scope for this PEP.
|
|||
Rejected Ideas
|
||||
==============
|
||||
|
||||
Keep the ability to use function local state when defining annotations
|
||||
----------------------------------------------------------------------
|
||||
Keeping the ability to use function local state when defining annotations
|
||||
-------------------------------------------------------------------------
|
||||
|
||||
With postponed evaluation, this would require keeping a reference to
|
||||
the frame in which an annotation got created. This could be achieved
|
||||
|
|
@ -327,6 +350,11 @@ This would be prohibitively expensive for highly annotated code as the
|
|||
frames would keep all their objects alive. That includes predominantly
|
||||
objects that won't ever be accessed again.
|
||||
|
||||
To be able to address class-level scope, the lambda approach would
|
||||
require a new kind of cell in the interpreter. This would proliferate
|
||||
the number of types that can appear in ``__annotations__``, as well as
|
||||
wouldn't be as introspectable as strings.
|
||||
|
||||
Note that in the case of nested classes, the functionality to get the
|
||||
effective "globals" and "locals" at definition time is provided by
|
||||
``typing.get_type_hints()``.
|
||||
|
|
@ -336,15 +364,15 @@ have to use local variables, it can populate the given generated
|
|||
object's ``__annotations__`` dictionary directly, without relying on
|
||||
the compiler.
|
||||
|
||||
Disallow local state usage for classes, too
|
||||
-------------------------------------------
|
||||
Disallowing local state usage for classes, too
|
||||
----------------------------------------------
|
||||
|
||||
This PEP originally proposed limiting names within annotations to only
|
||||
allow names from the model-level scope, including for classes. The
|
||||
author argued this makes name resolution unambiguous, including in cases
|
||||
of conflicts between local names and module-level names.
|
||||
|
||||
This idea was ultimately rejected in case of nested classes. Instead,
|
||||
This idea was ultimately rejected in case of classes. Instead,
|
||||
``typing.get_type_hints()`` got modified to populate the local namespace
|
||||
correctly if class-level annotations are needed.
|
||||
|
||||
|
|
@ -355,8 +383,8 @@ local scope access is required for class decorators to be able to
|
|||
evaluate type annotations. This is because class decorators are applied
|
||||
before the class receives its name in the outer scope.
|
||||
|
||||
Introduce a new dictionary for the string literal form instead
|
||||
--------------------------------------------------------------
|
||||
Introducing a new dictionary for the string literal form instead
|
||||
----------------------------------------------------------------
|
||||
|
||||
Yury Selivanov shared the following idea:
|
||||
|
||||
|
|
@ -384,8 +412,8 @@ recognize that forward references are allowed.
|
|||
Finally, just-in-time evaluation in ``__annotations__`` is an
|
||||
unnecessary step if ``get_type_hints()`` is used later.
|
||||
|
||||
Drop annotations with -O
|
||||
------------------------
|
||||
Dropping annotations with -O
|
||||
----------------------------
|
||||
|
||||
There are two reasons this is not satisfying for the purpose of this
|
||||
PEP.
|
||||
|
|
@ -404,6 +432,42 @@ a possibility in the future, as it's conceptually compatible with
|
|||
existing -O behavior (dropping docstrings and assert statements). This
|
||||
PEP does not invalidate the idea.
|
||||
|
||||
Pass string literals in annotations verbatim to ``__annotations__``
|
||||
-------------------------------------------------------------------
|
||||
|
||||
This PEP originally suggested directly storing the contents of a string
|
||||
literal under its respective key in ``__annotations__``. This was
|
||||
meant to simplify support for runtime type checkers.
|
||||
|
||||
Mark Shannon pointed out this idea was flawed since it wasn't handling
|
||||
situations where strings are only part of a type annotation.
|
||||
|
||||
The inconsistency of it was always apparent but given that it doesn't
|
||||
fully prevent cases of double-wrapping strings anyway, it is not worth
|
||||
it.
|
||||
|
||||
Make the name of the future import more verbose
|
||||
-----------------------------------------------
|
||||
|
||||
Instead of requiring the following import::
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
the PEP could call the feature more explicitly, for example
|
||||
``string_annotations``, ``stringify_annotations``,
|
||||
``annotation_strings``, ``annotations_as_strings``, ``lazy_anotations``,
|
||||
``static_annotations``, etc.
|
||||
|
||||
The problem with those names is that they are very verbose. Each of
|
||||
them besides ``lazy_annotations`` would constitute the longest future
|
||||
feature name in Python. They are long to type and harder to remember
|
||||
than the single-word form.
|
||||
|
||||
There is precedence of a future import name that sounds overly generic
|
||||
but in practice was obvious to users as to what it does::
|
||||
|
||||
from __future__ import division
|
||||
|
||||
|
||||
Prior discussion
|
||||
================
|
||||
|
|
@ -538,6 +602,62 @@ helper, which could solve self-referencing classes even in the form of a
|
|||
class decorator. He suggested the PEP should provide this helper
|
||||
function in the standard library.
|
||||
|
||||
Second draft discussion on python-dev
|
||||
-------------------------------------
|
||||
|
||||
Discussion happened mainly in the `announcement thread <https://mail.python.org/pipermail/python-dev/2017-November/150062.html>`_,
|
||||
followed by a brief discussion under `Mark Shannon's post
|
||||
<https://mail.python.org/pipermail/python-dev/2017-November/150637.html>`_.
|
||||
|
||||
Steven D'Aprano was concerned whether it's acceptable for typos to be
|
||||
allowed in annotations after the change proposed by the PEP. Brett
|
||||
Cannon responded that type checkers and other static analyzers (like
|
||||
linters or programming text editors) will catch this type of error.
|
||||
Jukka Lehtosalo added that this situation is analogous to how names in
|
||||
function bodies are not resolved until the function is called.
|
||||
|
||||
A major topic of discussion was Nick Coghlan's suggestion to store
|
||||
annotations in "thunk form", in other words as a specialized lambda
|
||||
which would be able to access class-level scope (and allow for scope
|
||||
customization at call time). He presented a possible design for it
|
||||
(`indirect attribute cells
|
||||
<https://mail.python.org/pipermail/python-dev/2017-November/150141.html>`_).
|
||||
This was later seen as equivalent to "special forms" in Lisp. Guido van
|
||||
Rossum expressed worry that this sort of feature cannot be safely
|
||||
implemented in twelve weeks (i.e. in time before the Python 3.7 beta
|
||||
freeze).
|
||||
|
||||
After a while it became clear that the point of division between
|
||||
supporters of the string form vs. supporters of the thunk form is
|
||||
actually about whether annotations should be perceived as a general
|
||||
syntactic element vs. something tied to the type checking use case.
|
||||
|
||||
Finally, Guido van Rossum declared he's rejecting the thunk idea
|
||||
based on the fact that it would require a new building block in the
|
||||
interpreter. This block would be exposed in annotations, multiplying
|
||||
possible types of values stored in ``__annotations__`` (arbitrary
|
||||
objects, strings, and now thunks). Moreover, thunks aren't as
|
||||
introspectable as strings. Most importantly, Guido van Rossum
|
||||
explicitly stated interest in gradually restricting the use of
|
||||
annotations to static typing (with an optional runtime component).
|
||||
|
||||
Nick Coghlan got convinced to PEP 563, too, promptly beginning
|
||||
the mandatory bike shedding session on the name of the ``__future__``
|
||||
import. Many debaters agreed that ``annotations`` seems like
|
||||
an overly broad name for the feature name. Guido van Rossum briefly
|
||||
decided to call it ``string_annotations`` but then changed his mind,
|
||||
arguing that ``division`` is a precedent of a broad name with a clear
|
||||
meaning.
|
||||
|
||||
The final improvement to the PEP suggested in the discussion by Mark
|
||||
Shannon was the rejection of the temptation to pass string literals
|
||||
through to ``__annotations__`` verbatim.
|
||||
|
||||
A side-thread of discussion started around the runtime penalty of
|
||||
static typing, with topic like the import time of the ``typing``
|
||||
module (which is comparable to ``re`` without dependencies, and
|
||||
three times as heavy as ``re`` when counting dependencies).
|
||||
|
||||
|
||||
Acknowledgements
|
||||
================
|
||||
|
|
|
|||
Loading…
Reference in New Issue