iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 0526 deemed read for python-dev (#81)

This commit is contained in:
Guido van Rossum 2016-08-30 14:03:57 -07:00 committed by GitHub
parent dc4b8d72c6
commit e05f4a3aba
1 changed files with 431 additions and 101 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

528
pep-0526.txt
View File

@ -2,53 +2,66 @@ PEP: 526
Title: Syntax for Variable and Attribute Annotations
Version: $Revision$
Last-Modified: $Date$
Author: Ryan Gonzalez <rymg19@gmail.com>, Philip House <phouse512@gmail.com>, Guido van Rossum <guido@python.org>
Author: Ryan Gonzalez <rymg19@gmail.com>, Philip House <phouse512@gmail.com>, Ivan Levkivskyi <levkivskyi@gmail.com>, Lisa Roach <lisaroach14@gmail.com>, Guido van Rossum <guido@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Aug-2016
Python-Version: 3.6
Notice for Reviewers
====================
This PEP is not ready for review. We're merely committing changes
frequently so we don't end up with a huge merge conflict. For minor
textual nits please use https://github.com/python/peps/pull/72. For
discussion about contents, please refer to
https://github.com/python/typing/issues/258 (but please be patient, we
know we're way behind addressing all comments).
This PEP was drafted in a separate repo:
https://github.com/phouse512/peps/tree/pep-0526.
There was preliminary discussion on python-ideas and at
https://github.com/python/typing/issues/258.
Before you bring up an objection in a public forum please at least
read the summary of rejected ideas listed at the end of this PEP.
Abstract
========
PEP 484 introduced type hints and; In particular, it introduced the notion of
type comments::
PEP 484 introduced type hints, a.k.a. type annotations. While its
main focus was function annotations, it also introduced the notion of
type comments to annotate variables::
# a is specified to be a list of ints.
a = [] # type: List[int]
# b is a string
b = None # type: str
class Cls:
my_class_attr = True # type: bool
# 'primes' is a list of integers
primes = [] # type: List[int]
This PEP aims at adding syntax to Python for annotating the types of variables and
attributes, instead of expressing them through comments::
# 'captain' is a string (Note: initial value is a problem)
captain = ... # type: str
class Starship:
# 'stats' is a class attribute
stats = {} # type: Dict[str, int]
This PEP aims at adding syntax to Python for annotating the types of variables
and attributes, instead of expressing them through comments::
primes: List[int] = []
captain: str # Note: no initial value!
class Starship:
stats: ClassVar[Dict[str, int]] = {}
a: List[int] = []
b: str
class Cls:
my_class_attr: ClassAttr[bool] = True
Rationale
=========
Although type comments work well, the fact that they're expressed through
comments has some downsides:
Although type comments work well enough, the fact that they're
expressed through comments has some downsides:
- Text editors often highlight comments differently from type annotations.
- There isn't a way to annotate the type of an undefined variable; you need to
- There's no way to annotate the type of an undefined variable; you need to
initialize it to ``None`` (e.g. ``a = None # type: int``).
- Variables annotated in a conditional branch are difficult to read::
if some_value:
@ -59,21 +72,54 @@ comments has some downsides:
- Since type comments aren't actually part of the language, if a Python script
wants to parse them, it would require a custom parser instead of just using
``ast``.
- It's impossible to retrieve the annotations at runtime outside of attempting to
find the module's source code and parse it at runtime, which is inelegant, to
say the least.
The majority of these issues can be alleviated by making the syntax a core part of
the language.
- Type comments are used a lot in typeshed. Migrating typeshed to use
the variable annotation syntax instead of type comments would improve
readability of stubs.
- In situations where normal comments and type comments used together, it is
difficult to distinguish them::
path = None # type: Optional[str] # Path to module source
- It's impossible to retrieve the annotations at runtime outside of
attempting to find the module's source code and parse it at runtime,
which is inelegant, to say the least.
The majority of these issues can be alleviated by making the syntax
a core part of the language.
Non-goals
*********
While the proposal is accompanied by an extension of ``typing.get_type_hints``
standard library function for runtime retrieval of annotations, the variable
annotations are not designed for runtime type checking. Third party packages
would have to be developed to implement such functionality.
It should also be emphasized that **Python will remain a dynamically typed
language, and the authors have no desire to ever make type hints mandatory,
even by convention.** The goal of annotation syntax is to provide an
easy way to specify the structured type metadata for third party tools.
Specification
=============
*** big key concepts, not quite sure what the best way to organize this would be,
or if they deserve their own sections ***
Type annotation can be added to an assignment statement or to a simple
name indicating the desired type of the annotation target to a third
party type checker::
Normal Variable Typing
**********************
my_var: int
my_var = 5 # Passes type check.
other_var: int = 'a' # Flagged as error by type checker,
# but OK at runtime.
Below we specify the semantics of type annotations for type checkers
in different contexts and their runtime effects.
Variable Annotations
********************
The types of locals and globals can be annotated as follows::
@ -89,20 +135,17 @@ assigned in conditional branches::
else:
sane_world = False
Note that, although this syntax does allow tuple packing, it does *not* allow one
to annotate the types of variables when tuple unpacking is used::
Note that, although the syntax does allow tuple packing, it does *not* allow
one to annotate the types of variables when tuple unpacking is used::
# Tuple packing with variable annotation syntax
t: Any = (1, 2, 3)
# Tuple unpacking with type comments
x, y, z = t # type: int, int, int
t: Tuple[int, ...] = (1, 2, 3)
# Tuple unpacking with variable annotation syntax
x: int
y: int
z: int
x, y, z = t
header: str
kind: int
body: Optional[List[str]]
header, kind, body = message
Omitting a default value leaves the variable uninitialized::
@ -115,7 +158,7 @@ it a local::
def f():
a: int
print(a) # raises UnboundLocalError
# Commenting out the `a: int` makes it a NameError!
# Commenting out the ``a: int`` makes it a NameError.
as if the code were::
@ -123,97 +166,384 @@ as if the code were::
if False: a = 0
print(a) # raises UnboundLocalError
Class Variable Typing
*********************
Adding variable types allow for us annotate the types of instance variables in class
bodies. In particular, the value-less notation (`a: int`) allows us to annotate
instance variables that should be initialized in `__init__` or `__new__`. The
proposed syntax looks as follows::
class Starship:
captain: str # instance variable without default
damage: int = 0 # instance variable with default
stats: class Dict[str, int] = {} # class variable with initialization
Duplicate annotations
*********************
Any duplicate type annotations will be ignored::
Duplicate type annotations will be ignored. However, static type
checkers will issue a warning for annotations of the same variable
by a different type::
a: int
a: int # Doesn't do anything.
a: str # Static type checker will warn about this.
The Python compiler will not validate the type expression, and leave it to
the type checker to complain. The above code will be allowed by the
compiler at runtime.
``__annotations__`` is writable, so this is permitted::
__annotations__['s'] = str
But attempting to update ``__annotations__`` to something other than a dict
may result in a TypeError::
class C:
__annotations__ = 42
x: int = 5 # raises TypeError
(Note that the assignment to ``__annotations__``, which is the
culprit, is accepted by the Python interpreter without questioning it
-- but the subsequent type annotation expects it to be a
``MutableMapping`` and will fail.)
Attribute annotations
*********************
Type annotations can also be used to annotate attributes
in class bodies. In particular, the value-less notation ``a: int`` allows us
to annotate instance variables that should be initialized in ``__init__``
or ``__new__``. The proposed syntax is as follows::
class BasicStarship:
captain: str = 'Picard' # instance variable with default
damage: int # instance variable without default
stats: ClassVar[Dict[str, int]] = {} # class variable
Here ``ClassVar`` is a special class in typing module that indicates to
static type checker that this attribute should not be set on class instances.
This could be illustrated with a more detailed example. In this class::
class Starship:
captain = 'Picard'
stats = {}
def __init__(self, damage, captain=None):
self.damage = damage
if captain:
self.captain = captain # Else keep the default
def hit(self):
Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1
``stats`` is intended to be a class variable (keeping track of many different
per-game statistics), while ``captain`` is an instance variable with a default
value set in the class. This difference could not be seen by type
checker -- both get initialized in the class, but ``captain`` serves only
as a convenient default value for the instance variable, while ``stats``
is truly a class variable -- it is intended to be shared by all instances.
Since both variables happen to be initialized at the class level, it is
useful to distinguish them by marking class variables as annotated with
types wrapped in ``ClassVar[...]``. In such way type checker will prevent
accidental assignments to attributes with a same name on class instances.
For example, annotating the discussed class::
class Starship:
captain: str = 'Picard'
damage: int
stats: ClassVar[Dict[str, int]] = {}
def __init__(self, damage: int, captain: str = None):
self.damage = damage
if captain:
self.captain = captain # Else keep the default
def hit(self):
Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1
enterprise_d = Starship(3000)
enterprise_d.stats = {} # Flagged as error by a type checker
Starship.stats = {} # This is OK
As a matter of convenience, instance attributes can be annotated in
``__init__`` or other methods, rather than in class::
from typing import Generic, TypeVar
T = TypeVar(’T’)
class Box(Generic[T]):
def __init__(self, content):
self.content: T = content
Annotating expressions
**********************
If the initial value is specified, then the target of the annotation can be
any valid single assignment target::
class Cls:
pass
c = Cls()
c.x: int = 0 # Annotates c.x with int.
c.y: int # Invalid syntax: no initial value was specified!
d = {}
d['a']: int = 0 # Annotates d['a'] with int.
d['b']: int # Invalid again.
Note that even ``(my_var)`` is considered an expression, not a simple name.
Consequently::
(x): int # Invalid syntax
(x): int = 0 # OK
It is up to the type checker to decide exactly when to accept this syntax.
Where annotations aren't allowed
********************************
It's illegal to attempt to annotate ``global`` and ``nonlocal``::
It is illegal to attempt to annotate variables subject to ``global``
or ``nonlocal`` in the same function scope::
def f():
global x: int # SyntaxError
def g():
x: int # Also a SyntaxError
global x
The reason is that ``global`` and ``nonlocal`` don't own variables;
therefore, the type annotations belong in the scope owning the variable.
In addition, you cannot annotate variable used in a ``for`` or ``with``
statement; they must be annotated ahead of time, in a similar manner to tuple
Only single assignment targets and single right hand side values are allowed.
In addition, one cannot annotate variables used in a ``for`` or ``with``
statement; they can be annotated ahead of time, in a similar manner to tuple
unpacking::
a: int
for a in my_iter:
...
f: MyFile
with myfunc() as f:
# ...
...
Capturing Types at Runtime
**************************
In order to capture variable types that are usable at runtime, we store the
types in `__annotations__` as dictionaries at various levels. At each level (for
example, global), the types dictionary would be stored in the `__annotations__`
dictionary for that given level. Here is an example for both global and class
level types::
Changes to standard library and documentation
=============================================
# print global type annotations
- A new covariant type ``ClassVar[T_co]`` is added to the ``typing``
module. It accepts only a single argument that should be a valid type,
and is used to annotate class variables that should no be set on class
instances. This restriction is ensured by static checkers,
but not at runtime. See Attribute Annotations for examples
and explanations for the usage of ``ClassVar``, and see the Rejected
Proposals section for more information on the reasoning behind ``ClassVar``.
- Function ``get_type_hints`` in the ``typing`` module will be extended,
so that one can retrieve type annotations at runtime from modules
and classes in addition to functions.
Annotations are returned as a dictionary mapping from variable, arguments,
or attributes to their type hints with forward references evaluated.
For classes it returns a mapping (perhaps ``collections.ChainMap``)
constructed from annotations in method resolution order.
- Recommended guidelines for using annotations will be added to the
documentation, containing a pedagogical recapitulation of specifications
described in this PEP and in PEP 484. In addition, a helper script for
translating type comments into type annotations will be published
separately from the standard library.
Runtime effects of type annotations
===================================
Annotating a local variable will cause
the interpreter to treat it as a local, even if it was never assigned to.
Annotations for local variables will not be evaluated::
def f():
x: NonexistentName # No error.
However, if it is at a module or class level, then the type *will* be
evaluated::
x: NonexistentName # Error!
class X:
attr: NonexistentName # Error!
In addition, at the module or class level, if the item being annotated is a
simple name, then it and the annotation will be stored in the
``__annotations__`` attribute of that module or class as a dictionary mapping
from names to evaluated annotations. Here is an example::
from typing import Dict
class Player:
...
players: Dict[str, Player]
print(__annotations__)
# prints: {'players': typing.Dict[str, __main__.Player]}
# print class type annotations
The recommended way of getting annotations at runtime is by using
``typing.get_type_hints`` function; as with all dunder attributes,
any undocummented use of ``__annotations__`` is subject to breakage
without warning::
from typing import Dict, ClassVar, get_type_hints
class Starship:
hitpoints: class int = 50
stats: class Dict[str, int] = {}
hitpoints: int = 50
stats: ClassVar[Dict[str, int]] = {}
shield: int = 100
captain: str # no initial value
print(Starship.__annotations__)
captain: str
def __init__(self, captain: str) -> None:
...
A note about locals -- the value of having annotations available locally does not
offset the cost of having to create and populate the annotations dictionary on
every function call.
These annotations would be printed out from the previous program as follows::
{'players': Dict[str, Player]}
{'hitpoints': ClassVar[int],
assert get_type_hints(Starship) == {'hitpoints': int,
'stats': ClassVar[Dict[str, int]],
'shield': int,
'captain': str
}
'captain': str}
assert get_type_hints(Starship.__init__) == {'captain': str,
'return': None}
Note that if annotations are not found statically, then the
``__annotations__`` dictionary is not created at all. Also the
value of having annotations available locally does not offset
the cost of having to create and populate the annotations dictionary
on every function call. Therefore annotations at function level are
not evaluated and not stored.
Other uses of annotations
*************************
While Python with this PEP will not object to::
alice: 'well done' = 'A+'
bob: 'what a shame' = 'F-'
since it will not care about the type annotation beyond "it evaluates
without raising", a type checker that encounters it will flag it,
unless disabled with ``# type: ignore`` or ``@no_type_check``.
However, since Python won't care what the "type" is,
if the above snippet is at the global level or in a class, ``__annotations__``
will include ``{'alice': 'well done', 'bob': 'what a shame'}``.
These stored annotations might be used for other purposes,
but with this PEP we explicitly recommend type hinting as the
preferred use of annotations.
Rejected proposals and things left out for now
==============================================
- **Should we introduce variable annotations at all?**
Variable annotations have *already* been around for almost two years
in the form of type comments, sanctioned by PEP 484. They are
extensively used by third party type checkers (mypy, pytype,
PyCharm, etc.) and by projects using the type checkers. However, the
comment syntax has many downsides listed in Rationale. This PEP is
not about the need for type annotations, it is about what should be
the syntax for such annotations.
- **Introduce a new keyword:**
The choice of a good keyword is hard,
e.g. it can't be ``var`` because that is way too common a variable name,
and it can't be ``local`` if we want to use it for class variables or
globals. Second, no matter what we choose, we'd still need
a ``__future__`` import.
- **Allow type annotations for tuple unpacking:**
This cause an ambiguity: It's not clear What meaning should be
assigned to this statement::
x, y: T
Are ``x`` and ``y`` both of type ``T``, or do we expect ``T`` to be
a tuple type of two items that are distributed over ``x`` and ``y``,
or perhaps ``x`` has type ``Any`` and ``y`` has type ``T``? (The
latter is what this would mean if this occurred in a function
signature.) Rather than leave the (human) reader guessing, we
forbid this, at least for now.
- **Parenthesized form ``(var: type)`` for annotations:**
It was brought up on python-ideas as a remedy for the above-mentioned
ambiguity, but it was rejected since such syntax would be hairy,
the benefits are slight, and the readability would be poor.
- **Allow annotations in chained assignments:**
This has problems of ambiguity and readability similar to tuple
unpacking, for example in::
x: int = y = 1
z = w: int = 1
it is ambiguous, what should be the type of ``y``, and what should
be the type of ``z``. Also the second line is difficult to parse.
- **Allow annotations in ``with`` and ``for`` statement:**
This was rejected because in ``for`` it would make it hard to spot the actual
iterable, and in ``with`` it would confuse the CPython's LL(1) parser.
- **Evaluate local annotations at function definition time:**
This has been rejected by Guido because the placement of the annotation
strongly suggests that it's in the same scope as the surrounding code.
- **Store variable annotations also in function scope:**
The value of having the annotations available locally is just not enough
to significantly offset the cost of creating and populating the dictionary
on *each* function call.
- **Initialize variables annotated without assignment:**
It was proposed on python-ideas to initialize ``x`` in ``x: int`` to
``None`` or to an additional special constant like Javascript's
``undefined``. However, adding yet another singleton value to the language
would needed to be checked for everywhere in the code. Therefore,
Guido just said plain "No" to this.
- **Add also** ``InstanceAttr`` **to the typing module:**
This is redundant because instance variables are way more common than
class variables. The more common usage deserves to be the default.
- **Allow instance attribute annotations only in methods:**
The problem is that many ``__init__`` methods do a lot of things besides
initializing instance variables, and it would be harder (for a human)
to find all the instance variable declarations.
And sometimes ``__init__`` is factored into more helper methods
so it's even harder to chase them down. Putting the instance variable
declarations together in the class makes it easier to find them,
and helps a first-time reader of the code.
- **Use syntax** ``x: class t = v`` **for class variables:**
This would require a more complicated parser and the ``class``
keyword would confuse simple-minded syntax highlighters. Anyway we
need to have ``ClassVar`` to store class variables to
``__annotations__``, so that it was decided to go with a simpler
syntax.
- **Forget about** ``ClassVar`` **altogether:**
This was proposed since mypy seems to be getting along fine without a way
to distinguish between class and instance variables. But a type checker
can do useful things with the extra information, for example flag
accidental assignments to a class variable via the instance
(which would create an instance variable shadowing the class variable).
It could also flag instance variables with mutable defaults,
a well-known hazard.
- **Do not evaluate annotations, treat them as strings:**
This would be inconsistent with the behavior of function annotations that
are always evaluated. Although this might be reconsidered in future,
it was decided in PEP 484 that this would have to be a separate PEP.
- **Declare attribute types in class docstring:**
Many projects already use various docstring conventions, often without
much consistency and generally without conforming to the PEP 484 annotation
syntax yet. Also this would require a special sophisticated parser.
This, in turn, would defeat the purpose of the PEP --
collaborating with the third party type checking tools.
- **Implement ``__annotations__`` as a descriptor:**
This was proposed to prohibit setting ``__annotations__`` to something
non-dictionary or non-None. Guido has rejected this idea as unnecessary;
instead a TypeError will be raised if an attempt is made to update
``__annotations__`` when it is anything other than a dict.
Mypy supports allowing `# type` on assignments to instance variables and other things.
In case you prefer annotating instance variables in `__init__` or `__new__`, you can
also annotate variable types for instance variables in methods. Despite this,
`__annotations__` will not be updated for that class.
Backwards Compatibility
=======================
This PEP is fully backwards compatible.
Implementation
==============
An implementation for Python 3.6 is found on GitHub repo at
https://github.com/ilevkivskyi/cpython/tree/pep-526
Copyright
=========