added PEP 324, popen5 - New POSIX process module, by Peter Astrand

pep-0000.txt

@@ -119,6 +119,7 @@ Index by Category
  S   319  Python Synchronize/Asynchronize Block        Pelletier
  S   321  Date/Time Parsing and Formatting             Kuchling
  S   323  Copyable Iterators                           Martelli
+ S   324  popen5 - New POSIX process module            Astrand
  S   754  IEEE 754 Floating Point Special Values       Warnes
 
  Finished PEPs (done, implemented in CVS)
@@ -338,6 +339,8 @@ Numerical Index
  I   320  Python 2.4 Release Schedule                  Warsaw
  S   321  Date/Time Parsing and Formatting             Kuchling
  SF  322  Reverse Iteration                            Hettinger
+ S   323  Copyable Iterators                           Martelli
+ S   324  popen5 - New POSIX process module            Astrand
  SR  666  Reject Foolish Indentation                   Creighton
  S   754  IEEE 754 Floating Point Special Values       Warnes
 
@@ -361,6 +364,7 @@ Owners
     Althoff, Jim             james_althoff@i2.com
     Altis, Kevin             altis@semi-retired.com
     Ascher, David            davida@activestate.com
+    Astrand, Peter           astrand@lysator.liu.se
     Barrett, Paul            barrett@stsci.edu
     Baxter, Anthony          anthony@interlink.com.au
     Bellman, Thomas          bellman+pep-divmod@lysator.liu.se

pep-0324.txt

@@ -0,0 +1,298 @@
+PEP: 324
+Title: popen5 - New POSIX process module
+Version: $Revision$
+Last-Modified: $Date$
+Author: Peter Astrand <astrand@lysator.liu.se>
+Status: Draft
+Type: Standards Track (library)
+Created: 19-Nov-2003
+Content-Type: text/plain
+Python-Version: 2.4
+
+
+Abstract
+
+    This PEP describes a new module for starting and communicating
+    with processes on POSIX systems.
+
+
+Motivation
+
+    Starting new processes is a common task in any programming
+    language, and very common in a high-level language like Python.
+    Good support for this task is needed, because:
+
+    - Inappropriate functions for starting processes could mean a
+      security risk: If the program is started through the shell, and
+      the arguments contain shell meta characters, the result can be
+      disastrous. [1]
+
+    - It makes Python an even better replacement language for
+      over-complicated shell scripts.
+
+    Currently, Python has a large number of different functions for
+    process creation. This makes it hard for developers to choose.
+
+    The popen5 modules provides the following enhancements over
+    previous functions:
+
+    - One "unified" module provides all functionality from previous
+      functions.
+
+    - Cross-process exceptions: Exceptions happening in the child
+      before the new process has started to execute are re-raised in
+      the parent.  This means that it's easy to handle exec()
+      failures, for example.  With popen2, for example, it's
+      impossible to detect if the execution failed.
+
+    - A hook for executing custom code between fork and exec.  This
+      can be used for, for example, changing uid.
+
+    - No implicit call of /bin/sh.  This means that there is no need
+      for escaping dangerous shell meta characters.
+
+    - All combinations of file descriptor redirection is possible.
+      For example, the "python-dialog" [2] needs to spawn a process
+      and redirect stderr, but not stdout.  This is not possible with
+      current functions, without using temporary files.
+
+    - With popen5, it's possible to control if all open file
+      descriptors should be closed before the new program is
+      executed.
+
+    - Support for connecting several subprocesses (shell "pipe").
+
+    - Universal newline support.
+
+    - A communicate() method, which makes it easy to send stdin data
+      and read stdout and stderr data, without risking deadlocks.
+      Most people are aware of the flow control issues involved with
+      child process communication, but not all have the patience or
+      skills to write a fully correct and deadlock-free select loop.
+      This means that many Python applications contain race
+      conditions.  A communicate() method in the standard library
+      solves this problem.
+
+
+Rationale
+
+    The following points summarizes the design:
+
+    - popen5 was based on popen2, which is tried-and-tested.
+
+    - The factory functions in popen2 have been removed, because I
+      consider the class constructor equally easy to work with.
+
+    - popen2 contains several factory functions and classes for
+      different combinations of redirection.  popen5, however,
+      contains one single class.  Since popen5 supports 12 different
+      combinations of redirection, providing a class or function for
+      each of them would be cumbersome and not very intuitive.  Even
+      with popen2, this is a readability problem.  For example, many
+      people cannot tell the difference between popen2.popen2 and
+      popen2.popen4 without using the documentation.
+
+    - One small utility function is provided: popen5.run().  It aims
+      to be an enhancement over os.system(), while still very easy to
+      use:
+
+        - It does not use the Standard C function system(), which has
+          limitations.
+
+        - It does not call the shell implicitly.
+
+        - No need for quoting; using a variable argument list.
+
+        - The return value is easier to work with.
+
+    - The "preexec" functionality makes it possible to run arbitrary
+      code between fork and exec.  One might ask why there are special
+      arguments for setting the environment and current directory, but
+      not for, for example, setting the uid.  The answer is:
+
+        - Changing environment and working directory is considered
+          fairly common.
+
+        - Old functions like spawn() has support for an
+          "env"-argument.
+
+        - env and cwd are considered quite cross-platform: They make
+          sense even on Windows.
+
+     - No MS Windows support is available, currently.  To be able to
+       provide more functionality than what is already available from
+       the popen2 module, help from C modules is required.
+
+
+Specification
+
+    This module defines one class called Popen:
+
+        class Popen(args, bufsize=0, argv0=None,
+                    stdin=None, stdout=None, stderr=None,
+                    preexec_fn=None, preexec_args=(), close_fds=0,
+                    cwd=None, env=None, universal_newlines=0)
+
+    Arguments are:
+
+    - args should be a sequence of program arguments.  The program to
+      execute is normally the first item in the args sequence, but can
+      be explicitly set by using the argv0 argument.  The Popen class
+      uses os.execvp() to execute the child program.
+
+    - bufsize, if given, has the same meaning as the corresponding
+      argument to the built-in open() function: 0 means unbuffered, 1
+      means line buffered, any other positive value means use a buffer
+      of (approximately) that size.  A negative bufsize means to use
+      the system default, which usually means fully buffered.  The
+      default value for bufsize is 0 (unbuffered).
+
+    - stdin, stdout and stderr specify the executed programs' standard
+      input, standard output and standard error file handles,
+      respectively.  Valid values are PIPE, an existing file
+      descriptor (a positive integer), an existing file object, and
+      None.  PIPE indicates that a new pipe to the child should be
+      created.  With None, no redirection will occur; the child's file
+      handles will be inherited from the parent.  Additionally, stderr
+      can be STDOUT, which indicates that the stderr data from the
+      applications should be captured into the same file handle as for
+      stdout.
+
+    - If preexec_fn is set to a callable object, this object will be
+      called in the child process just before the child is executed,
+      with arguments preexec_args.
+
+    - If close_fds is true, all file descriptors except 0, 1 and 2
+      will be closed before the child process is executed.
+
+    - If cwd is not None, the current directory will be changed to cwd
+      before the child is executed.
+
+    - If env is not None, it defines the environment variables for the
+      new process.
+
+    - If universal_newlines is true, the file objects fromchild and
+      childerr are opened as a text files, but lines may be terminated
+      by any of '\n', the Unix end-of-line convention, '\r', the
+      Macintosh convention or '\r\n', the Windows convention.  All of
+      these external representations are seen as '\n' by the Python
+      program.  Note: This feature is only available if Python is
+      built with universal newline support (the default).  Also, the
+      newlines attribute of the file objects fromchild, tochild and
+      childerr are not updated by the communicate() method.
+
+    The module also defines one shortcut function:
+
+        run(*args):
+            Run command with arguments.  Wait for command to complete,
+            then return the returncode attribute.  Example:
+
+                retcode = popen5.run("stty", "sane")
+
+
+    Exceptions
+    ----------
+    Exceptions raised in the child process, before the new program has
+    started to execute, will be re-raised in the parent.  Additionally,
+    the exception object will have one extra attribute called
+    'child_traceback', which is a string containing traceback
+    information from the child's point of view.
+
+    The most common exception raised is OSError.  This occurs, for
+    example, when trying to execute a non-existent file.  Applications
+    should prepare for OSErrors.
+
+    A PopenException will also be raised if Popen is called with
+    invalid arguments.
+
+
+    Security
+    --------
+    popen5 will never call /bin/sh implicitly.  This means that all
+    characters, including shell metacharacters, can safely be passed
+    to child processes.
+
+
+    Popen objects
+    -------------
+    Instances of the Popen class have the following methods:
+
+    poll()
+        Returns -1 if child process hasn't completed yet, or its exit
+        status otherwise.  See below for a description of how the exit
+        status is encoded.
+
+    wait()
+        Waits for and returns the exit status of the child process.
+        The exit status encodes both the return code of the process
+        and information about whether it exited using the exit()
+        system call or died due to a signal.  Functions to help
+        interpret the status code are defined in the os module (the
+        W*() family of functions).
+
+    communicate(input=None)
+        Interact with process: Send data to stdin.  Read data from
+        stdout and stderr, until end-of-file is reached.  Wait for
+        process to terminate.  The optional stdin argument should be a
+        string to be sent to the child process, or None, if no data
+        should be sent to the child.
+
+        communicate() returns a tuple (stdout, stderr).
+
+        Note: The data read is buffered in memory, so do not use this
+        method if the data size is large or unlimited.
+
+    The following attributes are also available:
+
+    fromchild
+        A file object that provides output from the child process.
+
+    tochild
+        A file object that provides input to the child process.
+
+    childerr
+        A file object that provides error output from the child
+        process.
+
+    pid
+        The process ID of the child process.
+
+    returncode
+        The child return code.  A None value indicates that the
+        process hasn't terminated yet.  A negative value means that
+        the process was terminated by a signal with number
+        -returncode.
+
+
+Open Issues
+
+    Perhaps the module should be called something like "process",
+    instead of "popen5".
+
+
+Reference Implementation
+
+    A reference implementation is available from
+    http://www.lysator.liu.se/~astrand/popen5/.
+
+
+References
+
+    [1] Secure Programming for Linux and Unix HOWTO, section 8.3.
+        http://www.dwheeler.com/secure-programs/
+
+    [2] Python Dialog
+        http://pythondialog.sourceforge.net/
+
+
+Copyright
+
+    This document has been placed in the public domain.
+
+
+Local Variables:
+mode: indented-text
+indent-tabs-mode: nil
+sentence-end-double-space: t
+fill-column: 70
+End: