python-peps/pep-3156.txt

PEP: 3156
Title: Asynchronous IO Support Rebooted
Version: $Revision$
Last-Modified: $Date$
Author: Guido van Rossum <guido@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Dec-2012
Post-History: TBD

Abstract
========

This is a proposal for asynchronous I/O in Python 3, starting with
Python 3.3.  Consider this the concrete proposal that is missing from
PEP 3153.  The proposal includes a pluggable event loop API, transport
and protocol abstractions similar to those in Twisted, and a
higher-level scheduler based on yield-from (PEP 380).  A reference
implementation is in the works under the code name tulip.


Introduction
============

The event loop is the place where most interoperability occurs.  It
should be easy for (Python 3.3 ports of) frameworks like Twisted,
Tornado, or ZeroMQ to either adapt the default event loop
implementation to their needs using a lightweight wrapper or proxy, or
to replace the default event loop implementation with an adaptation of
their own event loop implementation.  (Some frameworks, like Twisted,
have multiple event loop implementations.  This should not be a
problem since these all have the same interface.)

It should even be possible for two different third-party frameworks to
interoperate, either by sharing the default event loop implementation
(each using its own adapter), or by sharing the event loop
implementation of either framework.  In the latter case two levels of
adaptation would occur (from framework A's event loop to the standard
event loop interface, and from there to framework B's event loop).
Which event loop implementation is used should be under control of the
main program (though a default policy for event loop selection is
provided).

Thus, two separate APIs are defined:

- getting and setting the current event loop object
- the interface of a conforming event loop and its minimum guarantees

An event loop implementation may provide additional methods and
guarantees.

The event loop interface does not depend on yield-from.  Rather, it
uses a combination of callbacks, additional interfaces (transports and
protocols), and Futures.  The latter are similar to those defined in
PEP 3148, but have a different implementation and are not tied to
threads.  In particular, they have no wait() method; the user is
expected to use callbacks.

For users (like myself) who don't like using callbacks, a scheduler is
provided for writing asynchronous I/O code as coroutines using the PEP
380 yield-from expressions.  The scheduler is not pluggable;
pluggability occurs at the event loop level, and the scheduler should
work with any conforming event loop implementation.

For interoperability between code written using coroutines and other
async frameworks, the scheduler has a Task class that behaves like a
Future.  A framework that interoperates at the event loop level can
wait for a Future to complete by adding a callback to the Future.
Likewise, the scheduler offers an operation to suspend a coroutine
until a callback is called.

Limited interoperability with threads is provided by the event loop
interface; there is an API to submit a function to an executor (see
PEP 3148) which returns a Future that is compatible with the event
loop.


Non-goals
=========

Interoperability with systems like Stackless Python or
greenlets/gevent is not a goal of this PEP.


Specification
=============

Dependencies
------------

Python 3.3 is required.  No new language or standard library features
beyond Python 3.3 are required.  No third-party modules or packages
are required.

Module Namespace
----------------

The specification here will live in a new toplevel package.  Different
components will live in separate submodules of that package.  The
package will import common APIs from their respective submodules and
make them available as package attributes (similar to the way the
email package works).

The name of the toplevel package is currently unspecified.  The
reference implementation uses the name 'tulip', but the name will
change to something more boring if and when the implementation is
moved into the standard library (hopefully for Python 3.4).

Until the boring name is chosen, this PEP will use 'tulip' as the
toplevel package name.  Classes and functions given without a module
name are assumed to be accessed via the toplevel package.

Getting and Setting the Event Loop
----------------------------------

To get the current event loop, use ``get_event_loop()``.  This returns
an instance of the ``EventLoop`` class defined below or an equivalent
object.  It is possible that ``get_event_loop()`` returns a different
object depending on the current thread, or depending on some other
notion of context.

To set the current event loop, use ``set_event_loop(eventloop)``,
where ``eventloop`` is an instance of the ``EventLoop`` class or
equivalent.  This uses the same notion of context as
``get_event_loop()``.

To change the way ``get_event_loop()`` and ``set_event_loop()`` work
(including their notion of context), call
``set_event_loop_policy(policy)``, where ``policy`` is an event loop
policy object.  The policy object can be any object that has methods
``get_event_loop()`` and ``set_event_loop(eventloop)`` behaving like
the functions described above.  The default event loop policy is an
instance of the class ``EventLoopPolicy``.  The current event loop
policy object can be retrieved by calling ``get_event_loop_policy()``.

Event Loop Interface
--------------------

TBD.


Acknowledgments
===============

Apart from PEP 3153, influences include PEP 380 and Greg Ewing's
tutorial for yield-from, Twisted, Tornado, ZeroMQ, pyftpdlib, tulip
(the author's attempts at synthesis of all these), wattle (Steve
Dower's counter-proposal), numerous discussions on python-ideas from
September through December 2012, a Skype session with Steve Dower and
Dino Viehland, email exchanges with Ben Darnell, an audience with
Niels Provos (original author of libevent), and two in-person meetings
with several Twisted developers, including Glyph, Brian Warner, David
Reid, and Duncan McGreggor.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: