PEP 433:
* argument=>parameter * "always set" => "set by default" * unset => clear * add more example of inherance issues * add more examples of security issues
This commit is contained in:
Victor Stinner
parent
6bf3a71f57
commit
d68bfbbf9f
100
pep-0433.txt
100
pep-0433.txt
|
|
@ -1,5 +1,5 @@
|
|||
PEP: 433
|
||||
Title: Add cloexec argument to functions creating file descriptors
|
||||
Title: Add cloexec parameter to functions creating file descriptors
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Victor Stinner <victor.stinner@gmail.com>
|
||||
|
|
@ -13,18 +13,21 @@ Python-Version: 3.4
|
|||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes to add a new optional argument ``cloexec`` on
|
||||
This PEP proposes to add a new optional parameter ``cloexec`` on
|
||||
functions creating file descriptors in the Python standard library. If
|
||||
the argument is ``True``, the close-on-exec flag will be set on the
|
||||
the parameter is ``True``, the close-on-exec flag will be set on the
|
||||
new file descriptor.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
XXX recap briefly what the close-on-exec flag does
|
||||
|
||||
On UNIX, subprocess closes file descriptors greater than 2 by default
|
||||
since Python 3.2 [#subprocess_close]_. All file descriptors created by
|
||||
the parent process are automatically closed.
|
||||
the parent process are automatically closed in the child process.
|
||||
|
||||
``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of
|
||||
the listening socket, the parent class ``socketserver.BaseServer``
|
||||
does not set this flag.
|
||||
|
|
@ -58,6 +61,17 @@ the parent closes all files, creating a new file descriptor may fail
|
|||
with "too many files" because files are still open in the child
|
||||
process.
|
||||
|
||||
See also the following issues:
|
||||
|
||||
* `Issue #2320: Race condition in subprocess using stdin
|
||||
<http://bugs.python.org/issue2320>`_ (2008)
|
||||
* `Issue #3006: subprocess.Popen causes socket to remain open after
|
||||
close <http://bugs.python.org/issue3006>`_ (2008)
|
||||
* `Issue #7213: subprocess leaks open file descriptors between Popen
|
||||
instances causing hangs <http://bugs.python.org/issue7213>`_ (2009)
|
||||
* `Issue #12786: subprocess wait() hangs when stdin is closed
|
||||
<http://bugs.python.org/issue12786>`_ (2011)
|
||||
|
||||
|
||||
Security
|
||||
--------
|
||||
|
|
@ -67,6 +81,22 @@ untrusted child process can read sensitive data like passwords and
|
|||
take control of the parent process though leaked file descriptors. It
|
||||
is for example a known vulnerability to escape from a chroot.
|
||||
|
||||
See also the CERT recommandation:
|
||||
`FIO42-C. Ensure files are properly closed when they are no longer needed
|
||||
<https://www.securecoding.cert.org/confluence/display/seccode/FIO42-C.+Ensure+files+are+properly+closed+when+they+are+no+longer+needed>`_.
|
||||
|
||||
|
||||
Example of vulnerabilities:
|
||||
|
||||
|
||||
* `OpenSSH Security Advisory: portable-keysign-rand-helper.adv
|
||||
<http://www.openssh.com/txt/portable-keysign-rand-helper.adv>`_
|
||||
(April 2011)
|
||||
* `CWE-403: Exposure of File Descriptor to Unintended Control Sphere
|
||||
<http://cwe.mitre.org/data/definitions/403.html>`_ (2008)
|
||||
* `Hijacking Apache https by mod_php
|
||||
<http://www.securityfocus.com/archive/1/348368>`_ (Dec 2003)
|
||||
|
||||
|
||||
Atomicity
|
||||
---------
|
||||
|
|
@ -100,6 +130,8 @@ the kernel ignores ``O_CLOEXEC`` or ``SOCK_CLOEXEC`` flag, a call to
|
|||
close-on-exec flag set if ``fork()`` is used before ``exec()``, but
|
||||
it works correctly if ``exec()`` is called without ``fork()``.
|
||||
|
||||
XXX recheck this OpenBSD bug using a C program. XXX
|
||||
|
||||
|
||||
Scope
|
||||
-----
|
||||
|
|
@ -128,8 +160,8 @@ Impacted modules:
|
|||
* ``xmlrpc.server``
|
||||
* Maybe: ``signal``, ``threading``
|
||||
|
||||
XXX Should ``subprocess.Popen`` set the close-on-exec flag on file XXX
|
||||
XXX descriptors of the constructor the ``pass_fds`` argument? XXX
|
||||
XXX Should ``subprocess.Popen`` clear the close-on-exec flag on file
|
||||
XXX descriptors of the constructor the ``pass_fds`` parameter?
|
||||
|
||||
.. note::
|
||||
See `Close file descriptors after fork`_ for a possible solution
|
||||
|
|
@ -139,17 +171,17 @@ XXX descriptors of the constructor the ``pass_fds`` argument? XXX
|
|||
Proposal
|
||||
========
|
||||
|
||||
This PEP proposes to add a new optional argument ``cloexec`` on
|
||||
This PEP proposes to add a new optional parameter ``cloexec`` on
|
||||
functions creating file descriptors in the Python standard library. If
|
||||
the argument is ``True``, the close-on-exec flag will be set on the
|
||||
the parameter is ``True``, the close-on-exec flag will be set on the
|
||||
new file descriptor.
|
||||
|
||||
Add a new function:
|
||||
|
||||
* ``os.set_cloexec(fd: int, cloexec: bool)``: set or unset the
|
||||
* ``os.set_cloexec(fd: int, cloexec: bool)``: set or clear the
|
||||
close-on-exec flag of a file descriptor
|
||||
|
||||
Add a new optional ``cloexec`` argument to:
|
||||
Add a new optional ``cloexec`` parameter to:
|
||||
|
||||
* ``open()``: ``os.fdopen()`` is indirectly modified
|
||||
* ``os.dup()``, ``os.dup2()``
|
||||
|
|
@ -166,7 +198,7 @@ Add a new optional ``cloexec`` argument to:
|
|||
* ``socket.socket.recvmsg()``: use ``MSG_CMSG_CLOEXEC``,
|
||||
or ``os.set_cloexec()``
|
||||
|
||||
The default value of the ``cloexec`` argument is ``False`` to keep the
|
||||
The default value of the ``cloexec`` parameter is ``False`` to keep the
|
||||
backward compatibility.
|
||||
|
||||
The close-on-exec flag will not be set on file descriptors 0 (stdin),
|
||||
|
|
@ -177,12 +209,12 @@ explicitly using ``os.set_cloexec()``.
|
|||
Drawbacks:
|
||||
|
||||
* Many functions of the Python standard library creating file
|
||||
descriptors are cannot be changed by this proposal, because adding
|
||||
a ``cloexec`` optional argument would be surprising and too many
|
||||
descriptors cannot be changed by this proposal, because adding
|
||||
a ``cloexec`` optional parameter would be surprising and too many
|
||||
functions would need it. For example, ``os.urandom()`` uses a
|
||||
temporary file on UNIX, but it calls a function of Windows API on
|
||||
Windows. Adding a ``cloexec`` argument to ``os.urandom()`` would
|
||||
not make sense. See `Always set close-on-exec flag`_ for an
|
||||
Windows. Adding a ``cloexec`` parameter to ``os.urandom()`` would
|
||||
not make sense. See `Set the close-on-exec flag by default`_ for an
|
||||
incomplete list of functions creating file descriptors.
|
||||
* Checking if a module creates file descriptors is difficult. For
|
||||
example, ``os.urandom()`` creates a file descriptor on UNIX to read
|
||||
|
|
@ -195,17 +227,17 @@ Drawbacks:
|
|||
Alternatives
|
||||
============
|
||||
|
||||
Always set close-on-exec flag
|
||||
-----------------------------
|
||||
Set the close-on-exec flag by default
|
||||
-------------------------------------
|
||||
|
||||
Always set close-on-exec flag on new file descriptors created by
|
||||
Python. This alternative just changes the default value of the new
|
||||
``cloexec`` argument.
|
||||
Set the close-on-exec flag by default on new file descriptors created
|
||||
by Python. This alternative just changes the default value of the new
|
||||
``cloexec`` parameter.
|
||||
|
||||
If a file must be inherited by child processes, ``cloexec=False``
|
||||
argument can be used.
|
||||
parameter can be used.
|
||||
|
||||
``subprocess.Popen`` constructor has an ``pass_fds`` argument to
|
||||
``subprocess.Popen`` constructor has an ``pass_fds`` parameter to
|
||||
specify which file descriptors must be inherited. The close-on-exec
|
||||
flag of these file descriptors must be changed with
|
||||
``os.set_cloexec()``.
|
||||
|
|
@ -254,12 +286,12 @@ Drawbacks of setting close-on-exec flag by default:
|
|||
Backward compatibility: only a few programs rely on inherance of file
|
||||
descriptors, and they only pass a few file descriptors, usually just
|
||||
one. These programs will fail immediatly with ``EBADF`` error, and it
|
||||
will be simple to fix them: add ``cloexec=False`` argument or use
|
||||
will be simple to fix them: add ``cloexec=False`` parameter or use
|
||||
``os.set_cloexec(fd, False)``.
|
||||
|
||||
The ``subprocess`` module will be changed anyway to unset
|
||||
The ``subprocess`` module will be changed anyway to clear
|
||||
close-on-exec flag on file descriptors listed in the ``pass_fds``
|
||||
argument of Popen constructor. So it possible that these programs will
|
||||
parameter of Popen constructor. So it possible that these programs will
|
||||
not need any fix if they use the ``subprocess`` module.
|
||||
|
||||
|
||||
|
|
@ -280,12 +312,12 @@ Add new functions:
|
|||
close-on-exec flag, the state of the flag can be overriden in each
|
||||
function creating a file descriptor
|
||||
|
||||
The major change is that the default value of the ``cloexec`` argument
|
||||
The major change is that the default value of the ``cloexec`` parameter
|
||||
is ``sys.getdefaultcloexec()``, instead of ``False``.
|
||||
|
||||
When ``sys.setdefaultcloexec(True)`` is called to set close-on-exec by
|
||||
default, we have the same drawbacks than `Always set close-on-exec
|
||||
flag`_ alternative.
|
||||
default, we have the same drawbacks than `Set the close-on-exec flag
|
||||
by default`_ alternative.
|
||||
|
||||
There are additionnal drawbacks of having two behaviours depending on
|
||||
``sys.getdefaultcloexec()`` value:
|
||||
|
|
@ -299,7 +331,7 @@ Close file descriptors after fork
|
|||
|
||||
This PEP does not fix issues with applications using ``fork()``
|
||||
without ``exec()``. Python needs a generic process to register
|
||||
callbacks which would be called after a fork, see `Add an 'afterfork'
|
||||
callbacks which would be called after a fork, see `Add an 'atfork'
|
||||
module`_. Such registry could be used to close file descriptors just
|
||||
after a ``fork()``.
|
||||
|
||||
|
|
@ -325,7 +357,7 @@ open(): add "e" flag to mode
|
|||
A new "e" mode would set close-on-exec flag (best-effort).
|
||||
|
||||
This alternative only solves the problem for ``open()``.
|
||||
socket.socket() and os.pipe() do not have a ``mode`` argument for
|
||||
socket.socket() and os.pipe() do not have a ``mode`` parameter for
|
||||
example.
|
||||
|
||||
Since its version 2.7, the GNU libc supports ``"e"`` flag for
|
||||
|
|
@ -341,7 +373,7 @@ Most developers don't know that file descriptors are inherited by
|
|||
default. Most programs do not rely on inherance of file descriptors.
|
||||
For example, ``subprocess.Popen`` was changed in Python 3.2 to close
|
||||
all file descriptors greater than 2 in the child process by default.
|
||||
No user complained about this behavior change.
|
||||
No user complained about this behavior change yet.
|
||||
|
||||
Network servers using fork may want to pass the client socket to the
|
||||
child process. For example, on UNIX a CGI server pass the socket
|
||||
|
|
@ -513,7 +545,7 @@ For example, it is supported by ``open()`` and ``_pipe()``.
|
|||
The value of the flag can be modified using:
|
||||
``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``.
|
||||
|
||||
``CreateProcess()`` has an ``bInheritHandles`` argument: if it is
|
||||
``CreateProcess()`` has an ``bInheritHandles`` parameter: if it is
|
||||
FALSE, the handles are not inherited. It is used by
|
||||
``subprocess.Popen`` with ``close_fds`` option.
|
||||
|
||||
|
|
@ -534,7 +566,7 @@ ioctl
|
|||
Functions:
|
||||
|
||||
* ``ioctl(fd, FIOCLEX, 0)`` sets close-on-exec flag
|
||||
* ``ioctl(fd, FIONCLEX, 0)`` unsets close-on-exec flag
|
||||
* ``ioctl(fd, FIONCLEX, 0)`` clears close-on-exec flag
|
||||
|
||||
Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.
|
||||
|
||||
|
|
@ -597,7 +629,7 @@ Python issues:
|
|||
<http://bugs.python.org/issue16860>`_
|
||||
* `Support accept4() for atomic setting of flags at socket creation
|
||||
<http://bugs.python.org/issue10115>`_
|
||||
* `Add an 'afterfork' module
|
||||
* `Add an 'atfork' module
|
||||
<http://bugs.python.org/issue16500>`_
|
||||
|
||||
Ruby:
|
||||
|
|
|
|||
Loading…
Reference in New Issue