Add PEP 3145: Asynchronous I/O For subprocess.Popen

pep-3145.txt

@@ -0,0 +1,135 @@
+PEP: 3145
+Title: Asynchronous I/O For subprocess.Popen
+Version: $Revision$
+Last-Modified: $Date$
+Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson
+Status: Draft
+Type: Standards Track
+Content-Type: text/plain
+Created: 04-Aug-2009
+Python-Version: 3.2
+Post-History:
+
+Abstract:
+
+    In its present form, the ``subprocess.Popen`` implementation is prone to
+    dead-locking and blocking of the parent Python script while waiting on data
+    from the child process. This PEP proposes to make
+    ``subprocess.Popen`` more asynchronous to help alleviate these
+    problems.
+
+Motivation:
+
+    A search for "python asynchronous subprocess" will turn up numerous
+    accounts of people wanting to execute a child process and communicate with
+    it from time to time reading only the data that is available instead of
+    blocking to wait for the program to produce data [1] [2] [3].  The current
+    behavior of the subprocess module is that when a user sends or receives
+    data via the stdin, stderr and stdout file objects, dead locks are common
+    and documented [4] [5].  While ``communicate`` can be used to alleviate some of
+    the buffering issues, it will still cause the parent process to block while
+    attempting to read data when none is available to be read from the child
+    process.  
+
+Rationale:
+
+    There is a documented need for asynchronous, non-blocking functionality in
+    ``subprocess.Popen`` [6] [7] [2] [3].  Inclusion of the code would improve the
+    utility of the Python standard library that can be used on Unix based and
+    Windows builds of Python.  Practically every I/O object in Python has a
+    file-like wrapper of some sort.  Sockets already act as such and for
+    strings there is ``StringIO``.  Popen can be made to act like a file by simply
+    using the methods attached the the ``subprocess.Popen.stderr``,
+    ``stdout`` and
+    ``stdin`` file-like objects.  But when using the read and write methods of
+    those options, you do not have the benefit of asynchronous I/O.  In the
+    proposed solution the wrapper wraps the asynchronous methods to mimic a
+    file object.
+
+Reference Implementation:
+
+    I have been maintaining a Google Code repository that contains all of my
+    changes including tests and documentation [9] as well as blog detailing
+    the problems I have come across in the development process [10].  
+
+    I have been working on implementing non-blocking asynchronous I/O in the
+    ``subprocess.Popen`` module as well as a wrapper class for
+    ``subprocess.Popen``
+    that makes it so that an executed process can take the place of a file by
+    duplicating all of the methods and attributes that file objects have.  
+
+    There are two base functions that have been added to the
+    ``subprocess.Popen``
+    class: ``Popen.send`` and ``Popen._recv``, each with two separate implementations,
+    one for Windows and one for Unix based systems.  The Windows
+    implementation uses ``ctypes`` to access the functions needed to control pipes
+    in the kernel 32 DLL in an asynchronous manner.  On Unix based systems,
+    the Python interface for file control serves the same purpose.  The
+    different implementations of ``Popen.send`` and ``Popen._recv`` have identical
+    arguments to make code that uses these functions work across multiple
+    platforms.  
+
+    When calling the ``Popen._recv`` function, it requires the pipe name be
+    passed as an argument so there exists the ``Popen.recv`` function that passes
+    selects ``stdout`` as the pipe for ``Popen._recv`` by default.
+    ``Popen.recv_err``
+    selects ``stderr`` as the pipe by default. ``Popen.recv`` and
+    ``Popen.recv_err``
+    are much easier to read and understand than
+    ``Popen._recv('stdout' ...)`` and
+    ``Popen._recv('stderr' ...)`` respectively.  
+
+    Since the ``Popen._recv`` function does not wait on data to be produced
+    before returning a value, it may return empty bytes.
+    ``Popen.asyncread``
+    handles this issue by returning all data read over a given time
+    interval.  
+
+    The ``ProcessIOWrapper`` class uses the ``asyncread`` and
+    ``asyncwrite`` functions to
+    allow a process to act like a file so that there are no blocking issues
+    that can arise from using the ``stdout`` and ``stdin`` file objects produced from
+    a ``subprocess.Popen`` call.
+    
+
+References:
+
+    [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess
+        http://mail.python.org/pipermail/python-bugs-list/2006-December/
+          036524.html
+
+    [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess
+        http://ivory.idyll.org/blog/feb-07/problems-with-subprocess
+
+    [3] How can I run an external command asynchronously from Python? - Stack 
+        Overflow
+        http://stackoverflow.com/questions/636561/how-can-i-run-an-external-
+          command-asynchronously-from-python
+
+    [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
+        http://docs.python.org/library/subprocess.html#subprocess.Popen.wait
+
+    [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
+        http://docs.python.org/library/subprocess.html#subprocess.Popen.kill
+
+    [6] Issue 1191964: asynchronous Subprocess - Python tracker
+        http://bugs.python.org/issue1191964
+
+    [7] Module to allow Asynchronous subprocess use on Windows and Posix
+        platforms - ActiveState Code
+        http://code.activestate.com/recipes/440554/
+
+    [8] subprocess.rst - subprocdev - Project Hosting on Google Code
+        http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437
+
+    [9] subprocdev - Project Hosting on Google Code
+        http://code.google.com/p/subprocdev
+
+    [10] Python Subprocess Dev
+         http://subdev.blogspot.com/
+
+Copyright:
+
+    This P.E.P. is licensed under the Open Publication License;
+    http://www.opencontent.org/openpub/.
+