iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 433: 

* argument=>parameter
 * "always set" => "set by default"
 * unset => clear
 * add more example of inherance issues
 * add more examples of security issues
This commit is contained in:
Victor Stinner 2013-01-13 14:53:59 +01:00
parent 6bf3a71f57
commit d68bfbbf9f
1 changed files with 66 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

100
pep-0433.txt
View File

@ -1,5 +1,5 @@
PEP: 433 PEP: 433
Title: Add cloexec argument to functions creating file descriptors Title: Add cloexec parameter to functions creating file descriptors
Version: $Revision$ Version: $Revision$
Last-Modified: $Date$ Last-Modified: $Date$
Author: Victor Stinner <victor.stinner@gmail.com> Author: Victor Stinner <victor.stinner@gmail.com>
@ -13,18 +13,21 @@ Python-Version: 3.4
Abstract Abstract
======== ========
This PEP proposes to add a new optional argument ``cloexec`` on This PEP proposes to add a new optional parameter ``cloexec`` on
functions creating file descriptors in the Python standard library. If functions creating file descriptors in the Python standard library. If
the argument is ``True``, the close-on-exec flag will be set on the the parameter is ``True``, the close-on-exec flag will be set on the
new file descriptor. new file descriptor.
Rationale Rationale
========= =========
XXX recap briefly what the close-on-exec flag does
On UNIX, subprocess closes file descriptors greater than 2 by default On UNIX, subprocess closes file descriptors greater than 2 by default
since Python 3.2 [#subprocess_close]_. All file descriptors created by since Python 3.2 [#subprocess_close]_. All file descriptors created by
the parent process are automatically closed. the parent process are automatically closed in the child process.
``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of ``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of
the listening socket, the parent class ``socketserver.BaseServer`` the listening socket, the parent class ``socketserver.BaseServer``
does not set this flag. does not set this flag.
@ -58,6 +61,17 @@ the parent closes all files, creating a new file descriptor may fail
with "too many files" because files are still open in the child with "too many files" because files are still open in the child
process. process.
See also the following issues:
* `Issue #2320: Race condition in subprocess using stdin
<http://bugs.python.org/issue2320>`_ (2008)
* `Issue #3006: subprocess.Popen causes socket to remain open after
close <http://bugs.python.org/issue3006>`_ (2008)
* `Issue #7213: subprocess leaks open file descriptors between Popen
instances causing hangs <http://bugs.python.org/issue7213>`_ (2009)
* `Issue #12786: subprocess wait() hangs when stdin is closed
<http://bugs.python.org/issue12786>`_ (2011)
Security Security
-------- --------
@ -67,6 +81,22 @@ untrusted child process can read sensitive data like passwords and
take control of the parent process though leaked file descriptors. It take control of the parent process though leaked file descriptors. It
is for example a known vulnerability to escape from a chroot. is for example a known vulnerability to escape from a chroot.
See also the CERT recommandation:
`FIO42-C. Ensure files are properly closed when they are no longer needed
<https://www.securecoding.cert.org/confluence/display/seccode/FIO42-C.+Ensure+files+are+properly+closed+when+they+are+no+longer+needed>`_.
Example of vulnerabilities:
* `OpenSSH Security Advisory: portable-keysign-rand-helper.adv
<http://www.openssh.com/txt/portable-keysign-rand-helper.adv>`_
(April 2011)
* `CWE-403: Exposure of File Descriptor to Unintended Control Sphere
<http://cwe.mitre.org/data/definitions/403.html>`_ (2008)
* `Hijacking Apache https by mod_php
<http://www.securityfocus.com/archive/1/348368>`_ (Dec 2003)
Atomicity Atomicity
--------- ---------
@ -100,6 +130,8 @@ the kernel ignores ``O_CLOEXEC`` or ``SOCK_CLOEXEC`` flag, a call to
close-on-exec flag set if ``fork()`` is used before ``exec()``, but close-on-exec flag set if ``fork()`` is used before ``exec()``, but
it works correctly if ``exec()`` is called without ``fork()``. it works correctly if ``exec()`` is called without ``fork()``.
XXX recheck this OpenBSD bug using a C program. XXX
Scope Scope
----- -----
@ -128,8 +160,8 @@ Impacted modules:
* ``xmlrpc.server`` * ``xmlrpc.server``
* Maybe: ``signal``, ``threading`` * Maybe: ``signal``, ``threading``
XXX Should ``subprocess.Popen`` set the close-on-exec flag on file XXX XXX Should ``subprocess.Popen`` clear the close-on-exec flag on file
XXX descriptors of the constructor the ``pass_fds`` argument? XXX XXX descriptors of the constructor the ``pass_fds`` parameter?
.. note:: .. note::
See `Close file descriptors after fork`_ for a possible solution See `Close file descriptors after fork`_ for a possible solution
@ -139,17 +171,17 @@ XXX descriptors of the constructor the ``pass_fds`` argument? XXX
Proposal Proposal
======== ========
This PEP proposes to add a new optional argument ``cloexec`` on This PEP proposes to add a new optional parameter ``cloexec`` on
functions creating file descriptors in the Python standard library. If functions creating file descriptors in the Python standard library. If
the argument is ``True``, the close-on-exec flag will be set on the the parameter is ``True``, the close-on-exec flag will be set on the
new file descriptor. new file descriptor.
Add a new function: Add a new function:
* ``os.set_cloexec(fd: int, cloexec: bool)``: set or unset the * ``os.set_cloexec(fd: int, cloexec: bool)``: set or clear the
close-on-exec flag of a file descriptor close-on-exec flag of a file descriptor
Add a new optional ``cloexec`` argument to: Add a new optional ``cloexec`` parameter to:
* ``open()``: ``os.fdopen()`` is indirectly modified * ``open()``: ``os.fdopen()`` is indirectly modified
* ``os.dup()``, ``os.dup2()`` * ``os.dup()``, ``os.dup2()``
@ -166,7 +198,7 @@ Add a new optional ``cloexec`` argument to:
* ``socket.socket.recvmsg()``: use ``MSG_CMSG_CLOEXEC``, * ``socket.socket.recvmsg()``: use ``MSG_CMSG_CLOEXEC``,
or ``os.set_cloexec()`` or ``os.set_cloexec()``
The default value of the ``cloexec`` argument is ``False`` to keep the The default value of the ``cloexec`` parameter is ``False`` to keep the
backward compatibility. backward compatibility.
The close-on-exec flag will not be set on file descriptors 0 (stdin), The close-on-exec flag will not be set on file descriptors 0 (stdin),
@ -177,12 +209,12 @@ explicitly using ``os.set_cloexec()``.
Drawbacks: Drawbacks:
* Many functions of the Python standard library creating file * Many functions of the Python standard library creating file
descriptors are cannot be changed by this proposal, because adding descriptors cannot be changed by this proposal, because adding
a ``cloexec`` optional argument would be surprising and too many a ``cloexec`` optional parameter would be surprising and too many
functions would need it. For example, ``os.urandom()`` uses a functions would need it. For example, ``os.urandom()`` uses a
temporary file on UNIX, but it calls a function of Windows API on temporary file on UNIX, but it calls a function of Windows API on
Windows. Adding a ``cloexec`` argument to ``os.urandom()`` would Windows. Adding a ``cloexec`` parameter to ``os.urandom()`` would
not make sense. See `Always set close-on-exec flag`_ for an not make sense. See `Set the close-on-exec flag by default`_ for an
incomplete list of functions creating file descriptors. incomplete list of functions creating file descriptors.
* Checking if a module creates file descriptors is difficult. For * Checking if a module creates file descriptors is difficult. For
example, ``os.urandom()`` creates a file descriptor on UNIX to read example, ``os.urandom()`` creates a file descriptor on UNIX to read
@ -195,17 +227,17 @@ Drawbacks:
Alternatives Alternatives
============ ============
Always set close-on-exec flag Set the close-on-exec flag by default
----------------------------- -------------------------------------
Always set close-on-exec flag on new file descriptors created by Set the close-on-exec flag by default on new file descriptors created
Python. This alternative just changes the default value of the new by Python. This alternative just changes the default value of the new
``cloexec`` argument. ``cloexec`` parameter.
If a file must be inherited by child processes, ``cloexec=False`` If a file must be inherited by child processes, ``cloexec=False``
argument can be used. parameter can be used.
``subprocess.Popen`` constructor has an ``pass_fds`` argument to ``subprocess.Popen`` constructor has an ``pass_fds`` parameter to
specify which file descriptors must be inherited. The close-on-exec specify which file descriptors must be inherited. The close-on-exec
flag of these file descriptors must be changed with flag of these file descriptors must be changed with
``os.set_cloexec()``. ``os.set_cloexec()``.
@ -254,12 +286,12 @@ Drawbacks of setting close-on-exec flag by default:
Backward compatibility: only a few programs rely on inherance of file Backward compatibility: only a few programs rely on inherance of file
descriptors, and they only pass a few file descriptors, usually just descriptors, and they only pass a few file descriptors, usually just
one. These programs will fail immediatly with ``EBADF`` error, and it one. These programs will fail immediatly with ``EBADF`` error, and it
will be simple to fix them: add ``cloexec=False`` argument or use will be simple to fix them: add ``cloexec=False`` parameter or use
``os.set_cloexec(fd, False)``. ``os.set_cloexec(fd, False)``.
The ``subprocess`` module will be changed anyway to unset The ``subprocess`` module will be changed anyway to clear
close-on-exec flag on file descriptors listed in the ``pass_fds`` close-on-exec flag on file descriptors listed in the ``pass_fds``
argument of Popen constructor. So it possible that these programs will parameter of Popen constructor. So it possible that these programs will
not need any fix if they use the ``subprocess`` module. not need any fix if they use the ``subprocess`` module.
@ -280,12 +312,12 @@ Add new functions:
close-on-exec flag, the state of the flag can be overriden in each close-on-exec flag, the state of the flag can be overriden in each
function creating a file descriptor function creating a file descriptor
The major change is that the default value of the ``cloexec`` argument The major change is that the default value of the ``cloexec`` parameter
is ``sys.getdefaultcloexec()``, instead of ``False``. is ``sys.getdefaultcloexec()``, instead of ``False``.
When ``sys.setdefaultcloexec(True)`` is called to set close-on-exec by When ``sys.setdefaultcloexec(True)`` is called to set close-on-exec by
default, we have the same drawbacks than `Always set close-on-exec default, we have the same drawbacks than `Set the close-on-exec flag
flag`_ alternative. by default`_ alternative.
There are additionnal drawbacks of having two behaviours depending on There are additionnal drawbacks of having two behaviours depending on
``sys.getdefaultcloexec()`` value: ``sys.getdefaultcloexec()`` value:
@ -299,7 +331,7 @@ Close file descriptors after fork
This PEP does not fix issues with applications using ``fork()`` This PEP does not fix issues with applications using ``fork()``
without ``exec()``. Python needs a generic process to register without ``exec()``. Python needs a generic process to register
callbacks which would be called after a fork, see `Add an 'afterfork' callbacks which would be called after a fork, see `Add an 'atfork'
module`_. Such registry could be used to close file descriptors just module`_. Such registry could be used to close file descriptors just
after a ``fork()``. after a ``fork()``.
@ -325,7 +357,7 @@ open(): add "e" flag to mode
A new "e" mode would set close-on-exec flag (best-effort). A new "e" mode would set close-on-exec flag (best-effort).
This alternative only solves the problem for ``open()``. This alternative only solves the problem for ``open()``.
socket.socket() and os.pipe() do not have a ``mode`` argument for socket.socket() and os.pipe() do not have a ``mode`` parameter for
example. example.
Since its version 2.7, the GNU libc supports ``"e"`` flag for Since its version 2.7, the GNU libc supports ``"e"`` flag for
@ -341,7 +373,7 @@ Most developers don't know that file descriptors are inherited by
default. Most programs do not rely on inherance of file descriptors. default. Most programs do not rely on inherance of file descriptors.
For example, ``subprocess.Popen`` was changed in Python 3.2 to close For example, ``subprocess.Popen`` was changed in Python 3.2 to close
all file descriptors greater than 2 in the child process by default. all file descriptors greater than 2 in the child process by default.
No user complained about this behavior change. No user complained about this behavior change yet.
Network servers using fork may want to pass the client socket to the Network servers using fork may want to pass the client socket to the
child process. For example, on UNIX a CGI server pass the socket child process. For example, on UNIX a CGI server pass the socket
@ -513,7 +545,7 @@ For example, it is supported by ``open()`` and ``_pipe()``.
The value of the flag can be modified using: The value of the flag can be modified using:
``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``. ``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``.
``CreateProcess()`` has an ``bInheritHandles`` argument: if it is ``CreateProcess()`` has an ``bInheritHandles`` parameter: if it is
FALSE, the handles are not inherited. It is used by FALSE, the handles are not inherited. It is used by
``subprocess.Popen`` with ``close_fds`` option. ``subprocess.Popen`` with ``close_fds`` option.
@ -534,7 +566,7 @@ ioctl
Functions: Functions:
* ``ioctl(fd, FIOCLEX, 0)`` sets close-on-exec flag * ``ioctl(fd, FIOCLEX, 0)`` sets close-on-exec flag
* ``ioctl(fd, FIONCLEX, 0)`` unsets close-on-exec flag * ``ioctl(fd, FIONCLEX, 0)`` clears close-on-exec flag
Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD. Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.
@ -597,7 +629,7 @@ Python issues:
<http://bugs.python.org/issue16860>`_ <http://bugs.python.org/issue16860>`_
* `Support accept4() for atomic setting of flags at socket creation * `Support accept4() for atomic setting of flags at socket creation
<http://bugs.python.org/issue10115>`_ <http://bugs.python.org/issue10115>`_
* `Add an 'afterfork' module * `Add an 'atfork' module
<http://bugs.python.org/issue16500>`_ <http://bugs.python.org/issue16500>`_
Ruby: Ruby: