pep-0492: v3; tp_await, noiter, new await syntax, etc

pep-0492.txt

@@ -8,7 +8,7 @@ Type: Standards Track
 Content-Type: text/x-rst
 Created: 09-Apr-2015
 Python-Version: 3.5
-Post-History: 17-Apr-2015, 21-Apr-2015
+Post-History: 17-Apr-2015, 21-Apr-2015, 27-Apr-2015
 
 
 Abstract
@@ -155,12 +155,40 @@ can be one of:
   It is a ``TypeError`` if ``__await__`` returns anything but an
   iterator.
 
+* Objects defined with CPython C API with a ``tp_await`` function,
+  returning an iterator (similar to ``__await__`` method).
+
 It is a ``SyntaxError`` to use ``await`` outside of a coroutine.
 
 It is a ``TypeError`` to pass anything other than an *awaitable* object
 to an ``await`` expression.
 
 
+Syntax of "await" expression
+''''''''''''''''''''''''''''
+
+``await`` keyword is defined differently from ``yield`` and ``yield
+from``.  The main difference is that *await expressions* do not require
+parentheses around them most of the times.
+
+Examples::
+
+================================== ==================================
+Expression                         Will be parsed as
+================================== ==================================
+``if await fut: pass``             ``if (await fut): pass``
+``if await fut + 1: pass``         ``if (await fut) + 1: pass``
+``pair = await fut, 'spam'``       ``pair = (await fut), 'spam'``
+``with await fut, open(): pass``   ``with (await fut), open(): pass``
+``await foo()['spam'].baz()()``    ``await ( foo()['spam'].baz()() )``
+``return await coro()``            ``return ( await coro() )``
+``res = await coro() ** 2``        ``res = (await coro()) ** 2``
+``func(a1=await coro(), a2=0)``    ``func(a1=(await coro()), a2=0)``
+================================== ==================================
+
+See `Grammar Updates`_ section for details.
+
+
 Asynchronous Context Managers and "async with"
 ----------------------------------------------
 
@@ -487,6 +515,49 @@ replaces the previous wrapper.  ``sys.set_coroutine_wrapper(None)``
 unsets the wrapper.
 
 
+inspect.iscoroutine() and inspect.iscoroutineobject()
+-----------------------------------------------------
+
+Two new functions are added to the ``inspect`` module:
+
+* ``inspect.iscoroutine(obj)`` returns ``True`` if ``obj`` is a
+  coroutine object.
+
+* ``inspect.iscoroutinefunction(obj)`` returns ``True`` is ``obj`` is a
+  coroutine function.
+
+
+Differences between coroutines and generators
+---------------------------------------------
+
+A great effort has been made to make sure that coroutines and
+generators are separate concepts:
+
+1. Coroutine objects do not implement ``__iter__`` and ``__next__``
+   methods.  Therefore they cannot be iterated over or passed to
+   ``iter()``, ``list()``, ``tuple()`` and other built-ins.  They
+   also cannot be used in a ``for..in`` loop.
+
+2. ``yield from`` does not accept coroutine objects (unless it is used
+   in a generator-based coroutine decorated with ``types.coroutine``.)
+
+3. ``yield from`` does not accept coroutine objects from plain Python
+   generators (*not* generator-based coroutines.)
+
+4. ``inspect.isgenerator()`` and ``inspect.isgeneratorfunction()``
+   return ``False`` for coroutine objects and coroutine functions.
+
+
+Coroutine objects
+-----------------
+
+Coroutines are based on generators internally, thus they share the
+implementation.  Similarly to generator objects, coroutine objects have
+``throw``, ``send`` and ``close`` methods.  ``StopIteration`` and
+``GeneratorExit`` play the same role for coroutine objects (although
+PEP 479 is enabled by default for coroutines).
+
+
 Glossary
 ========
 
@@ -500,11 +571,12 @@ Glossary
     details.
 
 :Future-like object:
-    An object with an ``__await__`` method returning an iterator.  Can
-    be consumed by an ``await`` expression in a coroutine. A coroutine
-    waiting for a Future-like object is suspended until the Future-like
-    object's ``__await__`` completes, and returns the result.  See
-    `Await Expression`_ for details.
+    An object with an ``__await__`` method, or a C object with
+    ``tp_await`` function, returning an iterator.  Can be consumed by
+    an ``await`` expression in a coroutine. A coroutine waiting for a
+    Future-like object is suspended until the Future-like object's
+    ``__await__`` completes, and returns the result.  See `Await
+    Expression`_ for details.
 
 :Awaitable:
     A *Future-like* object or a *coroutine object*.  See `Await
@@ -602,29 +674,16 @@ Grammar Updates
 
 Grammar changes are also fairly minimal::
 
-    await_expr: AWAIT test
-    await_stmt: await_expr
-
     decorated: decorators (classdef | funcdef | async_funcdef)
     async_funcdef: ASYNC funcdef
 
+    compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt | with_stmt
+                    | funcdef | classdef | decorated | async_stmt)
+
     async_stmt: ASYNC (funcdef | with_stmt | for_stmt)
 
-    compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt |
-                    with_stmt | funcdef | classdef | decorated |
-                    async_stmt)
-
-    flow_stmt: (break_stmt | continue_stmt | return_stmt |
-                raise_stmt | yield_stmt | await_stmt)
-
-    atom: ('(' [yield_expr|await_expr|testlist_comp] ')' |
-          '[' [testlist_comp] ']' |
-          '{' [dictorsetmaker] '}' |
-          NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False’)
-
-    expr_stmt: testlist_star_expr
-                    (augassign (yield_expr|await_expr|testlist) |
-                    ('=' (yield_expr|await_expr|testlist_star_expr))*)
+    power: atom_expr ['**' factor]
+    atom_expr: [AWAIT] atom trailer*
 
 
 Transition Period Shortcomings
@@ -889,6 +948,15 @@ stating that the statement is asynchronous.  It is also more consistent
 with the existing grammar.
 
 
+Why "async for/with" instead of "await for/with"
+------------------------------------------------
+
+``async`` is an adjective, and hence it is a better choice for a
+*statement qualifier* keyword.  ``await for/with`` would imply that
+something is awaiting for a completion of a ``for`` or ``with``
+statement.
+
+
 Why "async def" and not "def async"
 -----------------------------------
 
@@ -946,11 +1014,13 @@ This approach has the following downsides:
 * it would not be possible to create an object that works in both
   ``with`` and ``async with`` statements;
 
-* it would look confusing and would require some implicit magic behind
-  the scenes in the interpreter;
+* it would break backwards compatibility, as nothing prohibits from
+  returning a Future-like objects from ``__enter__`` and/or
+  ``__exit__`` in Python <= 3.4;
 
 * one of the main points of this proposal is to make coroutines as
-  simple and foolproof as possible.
+  simple and foolproof as possible, hence the clear separation of the
+  protocols.
 
 
 Why not reuse existing "for" and "with" statements
@@ -1074,7 +1144,8 @@ List of high-level changes and new protocols
 1. New syntax for defining coroutines: ``async def`` and new ``await``
    keyword.
 
-2. New ``__await__`` method for Future-like objects.
+2. New ``__await__`` method for Future-like objects, and new
+   ``tp_await`` slot in ``PyTypeObject``.
 
 3. New syntax for asynchronous context managers: ``async with``.  And
    associated protocol with ``__aenter__`` and ``__aexit__`` methods.
@@ -1087,7 +1158,8 @@ List of high-level changes and new protocols
    ``Await``.
 
 6. New functions: ``sys.set_coroutine_wrapper(callback)``,
-   ``sys.get_coroutine_wrapper()``, and ``types.coroutine(gen)``.
+   ``sys.get_coroutine_wrapper()``, ``types.coroutine(gen)``,
+   ``inspect.iscoroutinefunction()``, and ``inspect.iscoroutine()``.
 
 7. New ``CO_COROUTINE`` bit flag for code objects.