From 837ab51351334cd3112633923ee215f3b42ada5e Mon Sep 17 00:00:00 2001 From: Chris Angelico Date: Tue, 20 Jan 2015 09:31:33 +1100 Subject: [PATCH] Oops - remove accidentally-committed files --- pep-0479.txt.orig | 603 ---------------------------------------------- pep-0483.txt.orig | 328 ------------------------- 2 files changed, 931 deletions(-) delete mode 100644 pep-0479.txt.orig delete mode 100644 pep-0483.txt.orig diff --git a/pep-0479.txt.orig b/pep-0479.txt.orig deleted file mode 100644 index e123a71a1..000000000 --- a/pep-0479.txt.orig +++ /dev/null @@ -1,603 +0,0 @@ -PEP: 479 -Title: Change StopIteration handling inside generators -Version: $Revision$ -Last-Modified: $Date$ -Author: Chris Angelico , Guido van Rossum -Status: Accepted -Type: Standards Track -Content-Type: text/x-rst -Created: 15-Nov-2014 -Python-Version: 3.5 -Post-History: 15-Nov-2014, 19-Nov-2014, 5-Dec-2014 - - -Abstract -======== - -This PEP proposes a change to generators: when ``StopIteration`` is -raised inside a generator, it is replaced it with ``RuntimeError``. -(More precisely, this happens when the exception is about to bubble -out of the generator's stack frame.) Because the change is backwards -incompatible, the feature is initially introduced using a -``__future__`` statement. - - -Acceptance -========== - -This PEP was accepted by the BDFL on November 22. Because of the -exceptionally short period from first draft to acceptance, the main -objections brought up after acceptance were carefully considered and -have been reflected in the "Alternate proposals" section below. -However, none of the discussion changed the BDFL's mind and the PEP's -acceptance is now final. (Suggestions for clarifying edits are still -welcome -- unlike IETF RFCs, the text of a PEP is not cast in stone -after its acceptance, although the core design/plan/specification -should not change after acceptance.) - - -Rationale -========= - -The interaction of generators and ``StopIteration`` is currently -somewhat surprising, and can conceal obscure bugs. An unexpected -exception should not result in subtly altered behaviour, but should -cause a noisy and easily-debugged traceback. Currently, -``StopIteration`` can be absorbed by the generator construct. - -The main goal of the proposal is to ease debugging in the situation -where an unguarded ``next()`` call (perhaps several stack frames deep) -raises ``StopIteration`` and causes the iteration controlled by the -generator to terminate silently. (When another exception is raised, a -traceback is printed pinpointing the cause of the problem.) - -This is particularly pernicious in combination with the ``yield from`` -construct of PEP 380 [1]_, as it breaks the abstraction that a -subgenerator may be factored out of a generator. That PEP notes this -limitation, but notes that "use cases for these [are] rare to non- -existent". Unfortunately while intentional use is rare, it is easy to -stumble on these cases by accident:: - - import contextlib - - @contextlib.contextmanager - def transaction(): - print('begin') - try: - yield from do_it() - except: - print('rollback') - raise - else: - print('commit') - - def do_it(): - print('Refactored initial setup') - yield # Body of with-statement is executed here - print('Refactored finalization of successful transaction') - - - import pathlib - - with transaction(): - print('commit file {}'.format( - # I can never remember what the README extension is - next(pathlib.Path('/some/dir').glob('README*')))) - -Here factoring out ``do_it`` into a subgenerator has introduced a subtle -bug: if a bug in the wrapped block allows ``StopIteration`` to escape -(here because the README doesn't exist), under the current behavior -``do_it`` will properly abort, but the exception will be swallowed by -the ``yield from`` and the original context manager will commit the -unfinished transaction! Similarly problematic behavior occurs when an -``asyncio`` coroutine raises ``StopIteration``, causing it to terminate -silently. In both cases, the refactoring abstraction of ``yield from`` -breaks in the presence of bugs in client code. - -Additionally, the proposal reduces the difference between list -comprehensions and generator expressions, preventing surprises such as -the one that started this discussion [2]_. Henceforth, the following -statements will produce the same result if either produces a result at -all:: - - a = list(F(x) for x in xs if P(x)) - a = [F(x) for x in xs if P(x)] - -With the current state of affairs, it is possible to write a function -``F(x)`` or a predicate ``P(x)`` that causes the first form to produce -a (truncated) result, while the second form raises an exception -(namely, ``StopIteration``). With the proposed change, both forms -will raise an exception at this point (albeit ``RuntimeError`` in the -first case and ``StopIteration`` in the second). - -Finally, the proposal also clears up the confusion about how to -terminate a generator: the proper way is ``return``, not -``raise StopIteration``. - -As an added bonus, the above changes bring generator functions much -more in line with regular functions. If you wish to take a piece of -code presented as a generator and turn it into something else, you -can usually do this fairly simply, by replacing every ``yield`` with -a call to ``print()`` or ``list.append()``; however, if there are any -bare ``next()`` calls in the code, you have to be aware of them. If -the code was originally written without relying on ``StopIteration`` -terminating the function, the transformation would be that much -easier. - - -Background information -====================== - -When a generator frame is (re)started as a result of a ``__next__()`` -(or ``send()`` or ``throw()``) call, one of three outcomes can occur: - -* A yield point is reached, and the yielded value is returned. -* The frame is returned from; ``StopIteration`` is raised. -* An exception is raised, which bubbles out. - -In the latter two cases the frame is abandoned (and the generator -object's ``gi_frame`` attribute is set to None). - - -Proposal -======== - -If a ``StopIteration`` is about to bubble out of a generator frame, it -is replaced with ``RuntimeError``, which causes the ``next()`` call -(which invoked the generator) to fail, passing that exception out. -From then on it's just like any old exception. [4]_ - -This affects the third outcome listed above, without altering any -other effects. Furthermore, it only affects this outcome when the -exception raised is ``StopIteration`` (or a subclass thereof). - -Note that the proposed replacement happens at the point where the -exception is about to bubble out of the frame, i.e. after any -``except`` or ``finally`` blocks that could affect it have been -exited. The ``StopIteration`` raised by returning from the frame is -not affected (the point being that ``StopIteration`` means that the -generator terminated "normally", i.e. it did not raise an exception). - -A subtle issue is what will happen if the caller, having caught the -``RuntimeError``, calls the generator object's ``__next__()`` method -again. The answer is that from this point on it will raise -``StopIteration`` -- the behavior is the same as when any other -exception was raised by the generator. - -Another logical consequence of the proposal: if someone uses -``g.throw(StopIteration)`` to throw a ``StopIteration`` exception into -a generator, if the generator doesn't catch it (which it could do -using a ``try/except`` around the ``yield``), it will be transformed -into ``RuntimeError``. - -During the transition phase, the new feature must be enabled -per-module using:: - - from __future__ import generator_stop - -Any generator function constructed under the influence of this -directive will have the ``REPLACE_STOPITERATION`` flag set on its code -object, and generators with the flag set will behave according to this -proposal. Once the feature becomes standard, the flag may be dropped; -code should not inspect generators for it. - - -Consequences for existing code -============================== - -This change will affect existing code that depends on -``StopIteration`` bubbling up. The pure Python reference -implementation of ``groupby`` [3]_ currently has comments "Exit on -``StopIteration``" where it is expected that the exception will -propagate and then be handled. This will be unusual, but not unknown, -and such constructs will fail. Other examples abound, e.g. [6]_, [7]_. - -(Nick Coghlan comments: """If you wanted to factor out a helper -function that terminated the generator you'd have to do "return -yield from helper()" rather than just "helper()".""") - -There are also examples of generator expressions floating around that -rely on a ``StopIteration`` raised by the expression, the target or the -predicate (rather than by the ``__next__()`` call implied in the ``for`` -loop proper). - -Writing backwards and forwards compatible code ----------------------------------------------- - -With the exception of hacks that raise ``StopIteration`` to exit a -generator expression, it is easy to write code that works equally well -under older Python versions as under the new semantics. - -This is done by enclosing those places in the generator body where a -``StopIteration`` is expected (e.g. bare ``next()`` calls or in some -cases helper functions that are expected to raise ``StopIteration``) -in a ``try/except`` construct that returns when ``StopIteration`` is -raised. The ``try/except`` construct should appear directly in the -generator function; doing this in a helper function that is not itself -a generator does not work. If ``raise StopIteration`` occurs directly -in a generator, simply replace it with ``return``. - - -Examples of breakage --------------------- - -Generators which explicitly raise ``StopIteration`` can generally be -changed to simply return instead. This will be compatible with all -existing Python versions, and will not be affected by ``__future__``. -Here are some illustrations from the standard library. - -Lib/ipaddress.py:: - - if other == self: - raise StopIteration - -Becomes:: - - if other == self: - return - -In some cases, this can be combined with ``yield from`` to simplify -the code, such as Lib/difflib.py:: - - if context is None: - while True: - yield next(line_pair_iterator) - -Becomes:: - - if context is None: - yield from line_pair_iterator - return - -(The ``return`` is necessary for a strictly-equivalent translation, -though in this particular file, there is no further code, and the -``return`` can be omitted.) For compatibility with pre-3.3 versions -of Python, this could be written with an explicit ``for`` loop:: - - if context is None: - for line in line_pair_iterator: - yield line - return - -More complicated iteration patterns will need explicit ``try/except`` -constructs. For example, a hypothetical parser like this:: - - def parser(f): - while True: - data = next(f) - while True: - line = next(f) - if line == "- end -": break - data += line - yield data - -would need to be rewritten as:: - - def parser(f): - while True: - try: - data = next(f) - while True: - line = next(f) - if line == "- end -": break - data += line - yield data - except StopIteration: - return - -or possibly:: - - def parser(f): - for data in f: - while True: - line = next(f) - if line == "- end -": break - data += line - yield data - -The latter form obscures the iteration by purporting to iterate over -the file with a ``for`` loop, but then also fetches more data from -the same iterator during the loop body. It does, however, clearly -differentiate between a "normal" termination (``StopIteration`` -instead of the initial line) and an "abnormal" termination (failing -to find the end marker in the inner loop, which will now raise -``RuntimeError``). - -This effect of ``StopIteration`` has been used to cut a generator -expression short, creating a form of ``takewhile``:: - - def stop(): - raise StopIteration - print(list(x for x in range(10) if x < 5 or stop())) - # prints [0, 1, 2, 3, 4] - -Under the current proposal, this form of non-local flow control is -not supported, and would have to be rewritten in statement form:: - - def gen(): - for x in range(10): - if x >= 5: return - yield x - print(list(gen())) - # prints [0, 1, 2, 3, 4] - -While this is a small loss of functionality, it is functionality that -often comes at the cost of readability, and just as ``lambda`` has -restrictions compared to ``def``, so does a generator expression have -restrictions compared to a generator function. In many cases, the -transformation to full generator function will be trivially easy, and -may improve structural clarity. - - -Explanation of generators, iterators, and StopIteration -======================================================= - -Under this proposal, generators and iterators would be distinct, but -related, concepts. Like the mixing of text and bytes in Python 2, -the mixing of generators and iterators has resulted in certain -perceived conveniences, but proper separation will make bugs more -visible. - -An iterator is an object with a ``__next__`` method. Like many other -special methods, it may either return a value, or raise a specific -exception - in this case, ``StopIteration`` - to signal that it has -no value to return. In this, it is similar to ``__getattr__`` (can -raise ``AttributeError``), ``__getitem__`` (can raise ``KeyError``), -and so on. A helper function for an iterator can be written to -follow the same protocol; for example:: - - def helper(x, y): - if x > y: return 1 / (x - y) - raise StopIteration - - def __next__(self): - if self.a: return helper(self.b, self.c) - return helper(self.d, self.e) - -Both forms of signalling are carried through: a returned value is -returned, an exception bubbles up. The helper is written to match -the protocol of the calling function. - -A generator function is one which contains a ``yield`` expression. -Each time it is (re)started, it may either yield a value, or return -(including "falling off the end"). A helper function for a generator -can also be written, but it must also follow generator protocol:: - - def helper(x, y): - if x > y: yield 1 / (x - y) - - def gen(self): - if self.a: return (yield from helper(self.b, self.c)) - return (yield from helper(self.d, self.e)) - -In both cases, any unexpected exception will bubble up. Due to the -nature of generators and iterators, an unexpected ``StopIteration`` -inside a generator will be converted into ``RuntimeError``, but -beyond that, all exceptions will propagate normally. - - -Transition plan -=============== - -- Python 3.5: Enable new semantics under ``__future__`` import; silent - deprecation warning if ``StopIteration`` bubbles out of a generator - not under ``__future__`` import. - -- Python 3.6: Non-silent deprecation warning. - -- Python 3.7: Enable new semantics everywhere. - - -Alternate proposals -=================== - -Raising something other than RuntimeError ------------------------------------------ - -Rather than the generic ``RuntimeError``, it might make sense to raise -a new exception type ``UnexpectedStopIteration``. This has the -downside of implicitly encouraging that it be caught; the correct -action is to catch the original ``StopIteration``, not the chained -exception. - - -Supplying a specific exception to raise on return -------------------------------------------------- - -Nick Coghlan suggested a means of providing a specific -``StopIteration`` instance to the generator; if any other instance of -``StopIteration`` is raised, it is an error, but if that particular -one is raised, the generator has properly completed. This subproposal -has been withdrawn in favour of better options, but is retained for -reference. - - -Making return-triggered StopIterations obvious ----------------------------------------------- - -For certain situations, a simpler and fully backward-compatible -solution may be sufficient: when a generator returns, instead of -raising ``StopIteration``, it raises a specific subclass of -``StopIteration`` (``GeneratorReturn``) which can then be detected. -If it is not that subclass, it is an escaping exception rather than a -return statement. - -The inspiration for this alternative proposal was Nick's observation -[8]_ that if an ``asyncio`` coroutine [9]_ accidentally raises -``StopIteration``, it currently terminates silently, which may present -a hard-to-debug mystery to the developer. The main proposal turns -such accidents into clearly distinguishable ``RuntimeError`` exceptions, -but if that is rejected, this alternate proposal would enable -``asyncio`` to distinguish between a ``return`` statement and an -accidentally-raised ``StopIteration`` exception. - -Of the three outcomes listed above, two change: - -* If a yield point is reached, the value, obviously, would still be - returned. -* If the frame is returned from, ``GeneratorReturn`` (rather than - ``StopIteration``) is raised. -* If an instance of ``GeneratorReturn`` would be raised, instead an - instance of ``StopIteration`` would be raised. Any other exception - bubbles up normally. - -In the third case, the ``StopIteration`` would have the ``value`` of -the original ``GeneratorReturn``, and would reference the original -exception in its ``__cause__``. If uncaught, this would clearly show -the chaining of exceptions. - -This alternative does *not* affect the discrepancy between generator -expressions and list comprehensions, but allows generator-aware code -(such as the ``contextlib`` and ``asyncio`` modules) to reliably -differentiate between the second and third outcomes listed above. - -However, once code exists that depends on this distinction between -``GeneratorReturn`` and ``StopIteration``, a generator that invokes -another generator and relies on the latter's ``StopIteration`` to -bubble out would still be potentially wrong, depending on the use made -of the distinction between the two exception types. - - -Converting the exception inside next() --------------------------------------- - -Mark Shannon suggested [12]_ that the problem could be solved in -``next()`` rather than at the boundary of generator functions. By -having ``next()`` catch ``StopIteration`` and raise instead -``ValueError``, all unexpected ``StopIteration`` bubbling would be -prevented; however, the backward-incompatibility concerns are far -more serious than for the current proposal, as every ``next()`` call -now needs to be rewritten to guard against ``ValueError`` instead of -``StopIteration`` - not to mention that there is no way to write one -block of code which reliably works on multiple versions of Python. -(Using a dedicated exception type, perhaps subclassing ``ValueError``, -would help this; however, all code would still need to be rewritten.) - - -Sub-proposal: decorator to explicitly request current behaviour ---------------------------------------------------------------- - -Nick Coghlan suggested [13]_ that the situations where the current -behaviour is desired could be supported by means of a decorator:: - - from itertools import allow_implicit_stop - - @allow_implicit_stop - def my_generator(): - ... - yield next(it) - ... - -Which would be semantically equivalent to:: - - def my_generator(): - try: - ... - yield next(it) - ... - except StopIteration - return - -but be faster, as it could be implemented by simply permitting the -``StopIteration`` to bubble up directly. - -Single-source Python 2/3 code would also benefit in a 3.7+ world, -since libraries like six and python-future could just define their own -version of "allow_implicit_stop" that referred to the new builtin in -3.5+, and was implemented as an identity function in other versions. - -However, due to the implementation complexities required, the ongoing -compatibility issues created, the subtlety of the decorator's effect, -and the fact that it would encourage the "quick-fix" solution of just -slapping the decorator onto all generators instead of properly fixing -the code in question, this sub-proposal has been rejected. [14]_ - - -Criticism -========= - -Unofficial and apocryphal statistics suggest that this is seldom, if -ever, a problem. [5]_ Code does exist which relies on the current -behaviour (e.g. [3]_, [6]_, [7]_), and there is the concern that this -would be unnecessary code churn to achieve little or no gain. - -Steven D'Aprano started an informal survey on comp.lang.python [10]_; -at the time of writing only two responses have been received: one was -in favor of changing list comprehensions to match generator -expressions (!), the other was in favor of this PEP's main proposal. - -The existing model has been compared to the perfectly-acceptable -issues inherent to every other case where an exception has special -meaning. For instance, an unexpected ``KeyError`` inside a -``__getitem__`` method will be interpreted as failure, rather than -permitted to bubble up. However, there is a difference. Special -methods use ``return`` to indicate normality, and ``raise`` to signal -abnormality; generators ``yield`` to indicate data, and ``return`` to -signal the abnormal state. This makes explicitly raising -``StopIteration`` entirely redundant, and potentially surprising. If -other special methods had dedicated keywords to distinguish between -their return paths, they too could turn unexpected exceptions into -``RuntimeError``; the fact that they cannot should not preclude -generators from doing so. - - -References -========== - -.. [1] PEP 380 - Syntax for Delegating to a Subgenerator - (https://www.python.org/dev/peps/pep-0380) - -.. [2] Initial mailing list comment - (https://mail.python.org/pipermail/python-ideas/2014-November/029906.html) - -.. [3] Pure Python implementation of groupby - (https://docs.python.org/3/library/itertools.html#itertools.groupby) - -.. [4] Proposal by GvR - (https://mail.python.org/pipermail/python-ideas/2014-November/029953.html) - -.. [5] Response by Steven D'Aprano - (https://mail.python.org/pipermail/python-ideas/2014-November/029994.html) - -.. [6] Split a sequence or generator using a predicate - (http://code.activestate.com/recipes/578416-split-a-sequence-or-generator-using-a-predicate/) - -.. [7] wrap unbounded generator to restrict its output - (http://code.activestate.com/recipes/66427-wrap-unbounded-generator-to-restrict-its-output/) - -.. [8] Post from Nick Coghlan mentioning asyncio - (https://mail.python.org/pipermail/python-ideas/2014-November/029961.html) - -.. [9] Coroutines in asyncio - (https://docs.python.org/3/library/asyncio-task.html#coroutines) - -.. [10] Thread on comp.lang.python started by Steven D'Aprano - (https://mail.python.org/pipermail/python-list/2014-November/680757.html) - -.. [11] Tracker issue with Proof-of-Concept patch - (http://bugs.python.org/issue22906) - -.. [12] Post from Mark Shannon with alternate proposal - (https://mail.python.org/pipermail/python-dev/2014-November/137129.html) - -.. [13] Idea from Nick Coghlan - (https://mail.python.org/pipermail/python-dev/2014-November/137201.html) - -.. [14] Rejection by GvR - (https://mail.python.org/pipermail/python-dev/2014-November/137243.html) - -Copyright -========= - -This document has been placed in the public domain. - - - -.. - Local Variables: - mode: indented-text - indent-tabs-mode: nil - sentence-end-double-space: t - fill-column: 70 - coding: utf-8 - End: diff --git a/pep-0483.txt.orig b/pep-0483.txt.orig deleted file mode 100644 index 0a4335222..000000000 --- a/pep-0483.txt.orig +++ /dev/null @@ -1,328 +0,0 @@ -PEP: 483 -Title: The Theory of Type Hinting -Version: $Revision$ -Last-Modified: $Date$ -Author: Guido van Rossum -Discussions-To: Python-Ideas -Status: Draft -Type: Informational -Content-Type: text/x-rst -Created: 08-Jan-2015 -Post-History: -Resolution: - -The Theory of Type Hinting -========================== - -Guido van Rossum, Dec 19, 2014 (with a few more recent updates) - -This work is licensed under a `Creative Commons -Attribution-NonCommercial-ShareAlike 4.0 International -License `_. - - -Introduction ------------- - -This document lays out the theory of the new type hinting proposal for -Python 3.5. It's not quite a full proposal or specification because -there are many details that need to be worked out, but it lays out the -theory without which it is hard to discuss more detailed specifications. -We start by explaining gradual typing; then we state some conventions -and general rules; then we define the new special types (such as Union) -that can be used in annotations; and finally we define the approach to -generic types. (The latter section needs more fleshing out; sorry!) - - -Summary of gradual typing -------------------------- - -We define a new relationship, is-consistent-with, which is similar to -is-subclass-of, except it is not transitive when the new type **Any** is -involved. (Neither relationship is symmetric.) Assigning x to y is OK if -the type of x is consistent with the type of y. (Compare this to “... if -the type of x is a subclass of the type of y,” which states one of the -fundamentals of OO programming.) The is-consistent-with relationship is -defined by three rules: - -- A type t1 is consistent with a type t2 if t1 is a subclass of t2. - (But not the other way around.) -- **Any** is consistent with every type. (But **Any** is not a subclass - of every type.) -- Every type is a subclass of **Any**. (Which also makes every type - consistent with **Any**, via rule 1.) - -That's all! See Jeremy Siek's blog post `What is Gradual -Typing `_ -for a longer explanation and motivation. Note that rule 3 places **Any** -at the root of the class graph. This makes it very similar to -**object**. The difference is that **object** is not consistent with -most types (e.g. you can't use an object() instance where an int is -expected). IOW both **Any** and **object** mean “any type is allowed” -when used to annotate an argument, but only **Any** can be passed no -matter what type is expected (in essence, **Any** shuts up complaints -from the static checker). - -Here's an example showing how these rules work out in practice: - -Say we have an Employee class, and a subclass Manager: - -- class Employee: ... -- class Manager(Employee): ... - -Let's say variable e is declared with type Employee: - -- e = Employee()  # type: Employee - -Now it's okay to assign a Manager instance to e (rule 1): - -- e = Manager() - -It's not okay to assign an Employee instance to a variable declared with -type Manager: - -- m = Manager()  # type: Manager -- m = Employee()  # Fails static check - -However, suppose we have a variable whose type is **Any**: - -- a = some\_func()  # type: Any - -Now it's okay to assign a to e (rule 2): - -- e = a  # OK - -Of course it's also okay to assign e to a (rule 3), but we didn't need -the concept of consistency for that: - -- a = e  # OK - -Notational conventions -~~~~~~~~~~~~~~~~~~~~~~ - -- t1, t2 etc.  and u1, u2 etc. are types or classes. Sometimes we write - ti or tj to refer to “any of t1, t2, etc.” -- X, Y etc. are type variables (defined with Var(), see below). -- C, D etc. are classes defined with a class statement. -- x, y etc. are objects or instances. -- We use the terms  type and  class interchangeably, and we assume - type(x) is x.\_\_class\_\_. - -General rules -~~~~~~~~~~~~~ - -- Instance-ness is  derived from class-ness, e.g. x is an instance of - t1 if  type(x) is a subclass of t1. -- No types defined below (i.e. Any, Union etc.) can be instantiated. - (But non-abstract subclasses of Generic can be.) -- No types defined below can be subclassed, except for Generic and - classes derived from it. -- Where a type is expected, None can be substituted for type(None); - e.g. Union[t1, None] == Union[t1, type(None)]. - -Types -~~~~~ - -- **Any**. Every class is a subclass of Any; however, to the static - type checker it is also consistent with every class (see above). -- **Union[t1, t2, ...]**. Classes that are subclass of at least one of - t1 etc. are subclasses of this. So are unions whose components are - all subclasses of t1 etc. (Example: Union[int, str] is a subclass of - Union[int, float, str].) The order of the arguments doesn't matter. - (Example: Union[int, str] == Union[str, int].) If ti is itself a - Union the result is flattened. (Example: Union[int, Union[float, - str]] == Union[int, float, str].) If ti and tj have a subclass - relationship, the less specific type survives. (Example: - Union[Employee, Manager] == Union[Employee].) Union[t1] returns just - t1. Union[] is illegal, so is Union[()]. Corollary: Union[..., Any, - ...] returns Any; Union[..., object, ...] returns object; to cut a - tie, Union[Any, object] == Union[object, Any] == Any. -- **Optional[t1]**. Alias for Union[t1, None], i.e. Union[t1, - type(None)]. -- **Tuple[t1, t2, ..., tn]**. A tuple whose items are instances of t1 - etc.. Example: Tuple[int, float] means a tuple of two items, the - first is an int, the second a float; e.g., (42, 3.14). Tuple[u1, u2, - ..., um] is a subclass of Tuple[t1, t2, ..., tn] if they have the - same length (n==m) and each ui is a subclass of ti. To spell the type - of the empty tuple, use Tuple[()]. There is no way to define a - variadic tuple type. (TODO: Maybe Tuple[t1, ...] with literal - ellipsis?) -- **Callable[[t1, t2, ..., tn], tr]**. A function with positional - argument types t1 etc., and return type tr. The argument list may be - empty (n==0). There is no way to indicate optional or keyword - arguments, nor varargs (we don't need to spell those often enough to - complicate the syntax — however, Reticulated Python has a useful idea - here). This is covariant in the return type, but contravariant in the - arguments. “Covariant” here means that for two callable types that - differ only in the return type, the subclass relationship for the - callable types follows that of the return types. (Example: - Callable[[], Manager] is a subclass of Callable[[], Employee].) - “Contravariant“ here means that for two callable types that differ - only in the type of one argument, the subclass relationship for the - callable types goes in the opposite direction as for the argument - types. (Example: Callable[[Employee], None] is a subclass of - Callable[[Mananger], None]. Yes, you read that right.) - -We might add: - -- **Intersection[t1, t2, ...]**. Classes that are subclass of *each* of - t1, etc are subclasses of this. (Compare to Union, which has *at - least one* instead of *each* in its definition.) The order of the - arguments doesn't matter. Nested intersections are flattened, e.g. - Intersection[int, Intersection[float, str]] == Intersection[int, - float, str]. An intersection of fewer types is a subclass of an - intersection of more types, e.g. Intersection[int, str] is a subclass - of Intersection[int, float, str]. An intersection of one argument is - just that argument, e.g. Intersection[int] is int. When argument have - a subclass relationship, the more specific class survives, e.g. - Intersection[str, Employee, Manager] is Intersection[str, Manager]. - Intersection[] is illegal, so is Intersection[()]. Corollary: Any - disappears from the argument list, e.g. Intersection[int, str, Any] - == Intersection[int, str].Intersection[Any, object] is object. The - interaction between Intersection and Union is complex but should be - no surprise if you understand the interaction between intersections - and unions in set theory (note that sets of types can be infinite in - size, since there is no limit on the number of new subclasses). - -Pragmatics -~~~~~~~~~~ - -Some things are irrelevant to the theory but make practical use more -convenient. (This is not a full list; I probably missed a few and some -are still controversial or not fully specified.) - -Type aliases, e.g. - -- point = Tuple[float, float] -- def distance(p: point) -> float: ...  - -Forward references via strings, e.g. - -class C: - -- def compare(self, other: “C”) -> int: ... - -If a default of None is specified, the type is implicitly optional, e.g. - -- def get(key: KT, default: VT = None) -> VT: ... - -Don't use dynamic type expressions; use builtins and imported types -only. No 'if'. - -- def display(message: str if WINDOWS else bytes):  # NOT OK - -Type declaration in comments, e.g. - -- x = []  # type: Sequence[int] - -Type declarations using Undefined, e.g. - -- x = Undefined(str) - -Other things, e.g. casts, overloading and stub modules; best left to an -actual PEP. - -Generic types -~~~~~~~~~~~~~ - -(TODO: Explain more. See also the `mypy docs on -generics `_.) - -**X = Var('X')**. Declares a unique type variable. The name must match -the variable name. - -**Y = Var('Y', t1, t2, ...).** Ditto, constrained to t1 etc. Behaves - like Union[t1, t2, ...] for most purposes, but when used as a type -variable, subclasses of t1 etc. are replaced by the most-derived base -class among t1 etc. - -Example of constrained type variables: - -AnyStr = Var('AnyStr', str, bytes) - -def longest(a: AnyStr, b: AnyStr) -> AnyStr: - -- return a if len(a) >= len(b) else b - -x = longest('a', 'abc')  # The inferred type for x is str - -y = longest('a', b'abc')  # Fails static type check - -In this example, both arguments to longest() must have the same type -(str or bytes), and moreover, even if the arguments are instances of a -common str subclass, the return type is still str, not that subclass -(see next example). - -For comparison, if the type variable was unconstrained, the common -subclass would be chosen as the return type, e.g.: - -S = Var('S') - -def longest(a: S, b: S) -> S: - -- return a if len(a) >= b else b - -class MyStr(str): ... - -x = longest(MyStr('a'), MyStr('abc')) - -The inferred type of x is MyStr (whereas in the AnyStr example it would -be str). - -Also for comparison, if a Union is used, the return type also has to be -a Union: - -U = Union[str, bytes] - -def longest(a: U, b: U) -> U: - -- return a if len(a) >- b else b - -x = longest('a', 'abc') - -The inferred type of x is still Union[str, bytes], even though both -arguments are str. - -**class C(Generic[X, Y, ...]):** ... Define a generic class C over type -variables X etc. C itself becomes parameterizable, e.g. C[int, str, ...] -is a specific class with substitutions X→int etc. - -TODO: Explain use of generic types in function signatures. E.g. -Sequence[X], Sequence[int], Sequence[Tuple[X, Y, Z]], and mixtures. -Think about co\*variance. No gimmicks like deriving from -Sequence[Union[int, str]] or Sequence[Union[int, X]]. - -**Protocol**. Similar to Generic but uses structural equivalence. (TODO: -Explain, and think about co\*variance.) - -Predefined generic types and Protocols in typing.py -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -(See also the `mypy typing.py -module `_.) - -- Everything from collections.abc (but Set renamed to AbstractSet). -- Dict, List, Set, a few more. (FrozenSet?) -- Pattern, Match. (Why?) -- IO, TextIO, BinaryIO. (Why?) - -Another reference -~~~~~~~~~~~~~~~~~ - -Lest mypy gets all the attention, I should mention \ `Reticulated -Python `_ by Michael Vitousek -as an example of a slightly different approach to gradual typing for -Python. It is described in an actual `academic -paper `_ -written by Vitousek with Jeremy Siek and Jim Baker (the latter of Jython -fame). - - -.. - Local Variables: - mode: indented-text - indent-tabs-mode: nil - sentence-end-double-space: t - fill-column: 70 - coding: utf-8 - End: