iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

update from Peter Astrand; title change

This commit is contained in:
David Goodger 2004-10-08 13:03:22 +00:00
parent dee69d03e1
commit abd928379d
2 changed files with 300 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
pep-0000.txt
View File

@ -116,7 +116,7 @@ Index by Category
S 319 Python Synchronize/Asynchronize Block Pelletier
S 321 Date/Time Parsing and Formatting Kuchling
S 323 Copyable Iterators Martelli
S 324 process - New POSIX process module Astrand
S 324 subprocess - New process module Astrand
S 325 Resource-Release Support for Generators Pedroni
S 330 Python Bytecode Verification Pelletier
S 331 Locale-Independent Float/String conversions Reis
@ -354,7 +354,7 @@ Numerical Index
S 321 Date/Time Parsing and Formatting Kuchling
SF 322 Reverse Iteration Hettinger
S 323 Copyable Iterators Martelli
S 324 process - New POSIX process module Astrand
S 324 subprocess - New POSIX process module Astrand
S 325 Resource-Release Support for Generators Pedroni
SR 326 A Case for Top and Bottom Values Carlson, Reedy
SF 327 Decimal Data Type Batista

379
pep-0324.txt
View File

@ -1,5 +1,5 @@
PEP: 324
Title: process - New POSIX process module
Title: subprocess - New process module
Version: $Revision$
Last-Modified: $Date$
Author: Peter Astrand <astrand@lysator.liu.se>
@ -13,7 +13,7 @@ Python-Version: 2.4
Abstract
This PEP describes a new module for starting and communicating
with processes on POSIX systems.
with processes.
Motivation
@ -31,9 +31,9 @@ Motivation
over-complicated shell scripts.
Currently, Python has a large number of different functions for
process creation. This makes it hard for developers to choose.
process creation. This makes it hard for developers to choose.
The process module provides the following enhancements over
The subprocess module provides the following enhancements over
previous functions:
- One "unified" module provides all functionality from previous
@ -56,7 +56,7 @@ Motivation
and redirect stderr, but not stdout. This is not possible with
current functions, without using temporary files.
- With the process module, it's possible to control if all open
- With the subprocess module, it's possible to control if all open
file descriptors should be closed before the new program is
executed.
@ -78,22 +78,22 @@ Rationale
The following points summarizes the design:
- process was based on popen2, which is tried-and-tested.
- subprocess was based on popen2, which is tried-and-tested.
- The factory functions in popen2 have been removed, because I
consider the class constructor equally easy to work with.
- popen2 contains several factory functions and classes for
different combinations of redirection. process, however,
contains one single class. Since the process module supports 12
different combinations of redirection, providing a class or
different combinations of redirection. subprocess, however,
contains one single class. Since the subprocess module supports
12 different combinations of redirection, providing a class or
function for each of them would be cumbersome and not very
intuitive. Even with popen2, this is a readability problem.
For example, many people cannot tell the difference between
popen2.popen2 and popen2.popen4 without using the documentation.
- Two small utility functions are provided: process.call() and
process.callv(). These aims to be an enhancement over
- Two small utility functions are provided: subprocess.call() and
subprocess.callv(). These aims to be an enhancement over
os.system(), while still very easy to use:
- It does not use the Standard C function system(), which has
@ -106,8 +106,8 @@ Rationale
- The return value is easier to work with.
The call() utility function accepts an 'args' argument, just
like the Popen class constructor. It waits for the command to
complete, then returns the returncode attribute. The
like the Popen class constructor. It waits for the command to
complete, then returns the returncode attribute. The
implementation is very simple:
def call(*args, **kwargs):
@ -117,30 +117,30 @@ Rationale
process and wait for it to finish is a common task.
The callv() function is identical to call(), except that each
non-keyword argument is treated as a program argument. This
gives a slightly nicer syntax. The drawback is that callv() does
not allow specifying the program and it's arguments as a
non-keyword argument is treated as a program argument. This
gives a slightly nicer syntax. The drawback is that callv()
does not allow specifying the program and it's arguments as a
whitespace-separated string: The entire (first) string would be
intepreted as the executable. The implementation of callv() is
interpreted as the executable. The implementation of callv() is
also very simple:
def callv(*args, **kwargs):
return Popen(args, **kwargs).wait()
While Popen supports a wide range of options, many users have
simple needs. Many people are using os.system() today, mainly
because it provides a simple interface. Consider this example:
simple needs. Many people are using os.system() today, mainly
because it provides a simple interface. Consider this example:
os.system("stty sane -F " + device)
With process.call(), this would look like:
With subprocess.call(), this would look like:
process.call(["stty", "sane", "-F", device])
subprocess.call(["stty", "sane", "-F", device])
Some people feel that the list brackets are clumsy. With
Some people feel that the list brackets are clumsy. With
callv(), they are not needed:
process.callv("stty", "sane", "-F", device)
subprocess.callv("stty", "sane", "-F", device)
- The "preexec" functionality makes it possible to run arbitrary
code between fork and exec. One might ask why there are special
@ -156,6 +156,13 @@ Rationale
- env and cwd are considered quite cross-platform: They make
sense even on Windows.
- On POSIX platforms, no extension module is required: the module
uses os.fork(), os.execvp() etc.
- On Windows platforms, the module requires either Mark Hammond's
Windows extensions[5], or a small extension module called
_subprocess.
Specification
@ -163,24 +170,24 @@ Specification
class Popen(args, bufsize=0, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=False,
preexec_fn=None, close_fds=False, shell=False,
cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0):
Arguments are:
- args should be a sequence of program arguments. The program to
execute is normally the first item in the args sequence, but can
be explicitly set by using the executable argument.
- On UNIX: the Popen class uses os.execvp() to execute the child
program, which operates on sequences. If args is a string, it
will be converted to a sequence using the cmdline2list method.
Please note that syntax for quoting arguments is different from
a typical UNIX shell. See the documentation of the cmdline2list
method for more information.
- On Windows: the Popen class uses CreateProcess() to execute the
child program, which operates on strings. If args is a
sequence, it will be converted to a string using the
@ -188,7 +195,7 @@ Specification
applications interpret the command line the same way: The
list2cmdline is designed for applications using the same rules
as the MS C runtime.
- bufsize, if given, has the same meaning as the corresponding
argument to the built-in open() function: 0 means unbuffered, 1
means line buffered, any other positive value means use a buffer
@ -213,6 +220,13 @@ Specification
- If close_fds is true, all file descriptors except 0, 1 and 2
will be closed before the child process is executed.
- If shell is true, the specified command will be executed through
the shell. Note: When executing through the shell on UNIX
systems and the args argument is a sequence, only the first
element in the sequence will be passed as a command string to
the shell. The remaining arguments can be used to specify
additional shell options.
- If cwd is not None, the current directory will be changed to cwd
before the child is executed.
@ -233,66 +247,74 @@ Specification
the underlying CreateProcess() function. They can specify
things such as appearance of the main window and priority for
the new process. (Windows only)
This module also defines two shortcut functions:
- call(*args, **kwargs):
Run command with arguments. Wait for command to complete, then
return the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
Run command with arguments. Wait for command to complete,
then return the returncode attribute.
The arguments are the same as for the Popen constructor.
Example:
retcode = call(["ls", "-l"])
- callv(*args, **kwargs):
Run command with arguments. Wait for command to complete, then
return the returncode attribute.
This function is identical to call(), except that each non-keyword
argument is treated as a program argument. Example:
Run command with arguments. Wait for command to complete,
then return the returncode attribute.
This function is identical to call(), except that each
non-keyword argument is treated as a program argument.
Example:
retcode = callv("ls", "-l")
This is equivalent to:
retcode = call(["ls", "-l"])
Exceptions
----------
Exceptions raised in the child process, before the new program has
started to execute, will be re-raised in the parent.
Additionally, the exception object will have one extra attribute
called 'child_traceback', which is a string containing traceback
information from the childs point of view.
information from the child's point of view.
The most common exception raised is OSError. This occurs, for
example, when trying to execute a non-existent file. Applications
should prepare for OSErrors.
A ValueError will be raised if Popen is called with invalid arguments.
A ValueError will be raised if Popen is called with invalid
arguments.
Security
--------
Unlike some other popen functions, this implementation will never call
/bin/sh implicitly. This means that all characters, including shell
metacharacters, can safely be passed to child processes.
Unlike some other popen functions, this implementation will never
call /bin/sh implicitly. This means that all characters,
including shell meta-characters, can safely be passed to child
processes.
Popen objects
-------------
Instances of the Popen class have the following methods:
poll()
Check if child process has terminated. Returns returncode
attribute.
wait()
Wait for child process to terminate. Returns returncode attribute.
Wait for child process to terminate. Returns returncode
attribute.
communicate(input=None)
Interact with process: Send data to stdin. Read data from
stdout and stderr, until end-of-file is reached. Wait for
@ -306,21 +328,22 @@ Specification
method if the data size is large or unlimited.
The following attributes are also available:
stdin
If the stdin argument is PIPE, this attribute is a file object
that provides input to the child process. Otherwise, it is None.
that provides input to the child process. Otherwise, it is
None.
stdout
If the stdout argument is PIPE, this attribute is a file object
that provides output from the child process. Otherwise, it is
None.
If the stdout argument is PIPE, this attribute is a file
object that provides output from the child process.
Otherwise, it is None.
stderr
If the stderr argument is PIPE, this attribute is file object that
provides error output from the child process. Otherwise, it is
None.
If the stderr argument is PIPE, this attribute is file object
that provides error output from the child process. Otherwise,
it is None.
pid
The process ID of the child process.
@ -328,14 +351,200 @@ Specification
The child return code. A None value indicates that the
process hasn't terminated yet. A negative value -N indicates
that the child was terminated by signal N (UNIX only).
Replacing older functions with the subprocess module
In this section, "a ==> b" means that b can be used as a
replacement for a.
Note: All functions in this section fail (more or less) silently
if the executed program cannot be found; this module raises an
OSError exception.
In the following examples, we assume that the subprocess module is
imported with "from subprocess import *".
Replacing /bin/sh shell backquote
---------------------------------
output=`mycmd myarg`
==>
output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
Replacing shell pipe line
-------------------------
output=`dmesg | grep hda`
==>
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout)
output = p2.communicate()[0]
Replacing os.system()
---------------------
sts = os.system("mycmd" + " myarg")
==>
p = Popen(["mycmd" + " myarg"], shell=True)
sts = os.waitpid(p.pid, 0)
Note:
* Calling the program through the shell is usually not required.
* It's easier to look at the returncode attribute than the
exit status.
A more real-world example would look like this:
try:
retcode = callv("mycmd", "myarg")
if retcode < 0:
print >>sys.stderr, "Child was terminated by signal", -retcode
else:
print >>sys.stderr, "Child returned", retcode
except OSError, e:
print >>sys.stderr, "Execution failed:", e
Replacing os.spawn*
-------------------
P_NOWAIT example:
pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
==>
pid = Popen(["/bin/mycmd", "myarg"]).pid
P_WAIT example:
retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
==>
retcode = callv("/bin/mycmd", "myarg")
Vector example:
os.spawnvp(os.P_NOWAIT, path, args)
==>
Popen([path] + args[1:])
Environment example:
os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Replacing os.popen*
-------------------
pipe = os.popen(cmd, mode='r', bufsize)
==>
pipe = Popen([cmd], shell=True, bufsize=bufsize, stdout=PIPE).stdout
pipe = os.popen(cmd, mode='w', bufsize)
==>
pipe = Popen([cmd], shell=True, bufsize=bufsize, stdin=PIPE).stdin
(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
==>
p = Popen([cmd], shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdin, child_stdout) = (p.stdin, p.stdout)
(child_stdin,
child_stdout,
child_stderr) = os.popen3(cmd, mode, bufsize)
==>
p = Popen([cmd], shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,
child_stdout,
child_stderr) = (p.stdin, p.stdout, p.stderr)
(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
==>
p = Popen([cmd], shell=True, bufsize=bufsize,
stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
Replacing popen2.*
------------------
Note: If the cmd argument to popen2 functions is a string, the
command is executed through /bin/sh. If it is a list, the command
is directly executed.
(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
==>
p = Popen(["somestring"], shell=True, bufsize=bufsize
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)
(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
==>
p = Popen(["mycmd", "myarg"], bufsize=bufsize,
stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)
The popen2.Popen3 and popen3.Popen4 basically works as
subprocess.Popen, except that:
* subprocess.Popen raises an exception if the execution fails
* the capturestderr argument is replaced with the stderr argument.
* stdin=PIPE and stdout=PIPE must be specified.
* popen2 closes all file descriptors by default, but you have to
specify close_fds=True with subprocess.Popen.
Open Issues
Currently, the reference implementation requires the "win32all"
extensions when running on the Windows platform. This dependency
could probably be eliminated by providing a small "glue" module
written in C, just like the _winreg module.
Some features have been requested but is not yet implemented.
This includes:
* Support for managing a whole flock of subprocesses
* Support for managing "daemon" processes
* Built-in method for killing subprocesses
While these are useful features, it's expected that these can be
added later without problems.
* expect-like functionality, including pty support.
pty support is highly platform-dependent, which is a
problem. Also, there are already other modules that provides this
kind of functionality[6].
Backwards Compatibility
Since this is a new module, no major backward compatible issues
are expected. The module name "subprocess" might collide with
other, previous modules[3] with the same name, but the name
"subprocess" seems to be the best suggested name so far. The
first name of this module was "popen5", but this name was
considered too unintuitive. For a while, the module was called
"process", but this name is already used by Trent Mick's
module[4].
The functions and modules that this new module is trying to
replace (os.system, os.spawn*, os.popen*, popen2.*, commands.*)
are expected to be available in future Python versions for a long
time, to preserve backwards compatibility.
Reference Implementation
@ -352,6 +561,14 @@ References
[2] Python Dialog
http://pythondialog.sourceforge.net/
[3] http://www.iol.ie/~padraiga/libs/subProcess.py
[4] http://starship.python.net/crew/tmick/
[5] http://starship.python.net/crew/mhammond/win32/
[6] http://www.lysator.liu.se/~ceder/pcl-expect/
Copyright