python-peps/pep-0433.txt

722 lines
25 KiB
Plaintext
Raw Normal View History

PEP: 433
2013-01-18 06:45:30 -05:00
Title: Easier suppression of file descriptor inheritance
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 10-January-2013
Python-Version: 3.4
Abstract
========
This PEP proposes to add a new optional parameter ``cloexec`` on
2013-01-12 19:30:02 -05:00
functions creating file descriptors in the Python standard library. If
the parameter is ``True``, the close-on-exec flag will be set on the
2013-01-12 19:30:02 -05:00
new file descriptor.
Rationale
=========
2013-01-24 18:33:38 -05:00
A file descriptor has a close-on-exec flag which indicates if the file
descriptor will be inherited or not.
2013-01-24 18:33:38 -05:00
On UNIX, the file descriptor will be closed on the execution of child processes
if the close-on-exec flag is set, the file descriptor is inherited by child
processes if the flag is cleared.
On Windows, the file descriptor is not inherited if the close-on-exec flag is
set, the file descriptor is inherited by child processes if the flag is cleared
and if ``CreateProcess()`` is called with the *bInheritHandles* parameter set
to ``TRUE`` (when ``subprocess.Popen`` is created with ``close_fds=False`` for
example).
Status in Python 3.3
--------------------
On UNIX, the subprocess module closes file descriptors greater than 2 by
default since Python 3.2 [#subprocess_close]_. All file descriptors created by
the parent process are automatically closed in the child process.
2013-01-12 19:30:02 -05:00
``xmlrpc.server.SimpleXMLRPCServer`` sets the close-on-exec flag of
the listening socket, the parent class ``socketserver.BaseServer``
does not set this flag.
2013-01-12 19:30:02 -05:00
There are other cases creating a subprocess or executing a new program
where file descriptors are not closed: functions of the os.spawn*()
family and third party modules calling ``exec()`` or ``fork()`` +
``exec()``. In this case, file descriptors are shared between the
parent and the child processes which is usually unexpected and causes
various issues.
2013-01-12 18:04:29 -05:00
This PEP proposes to continue the work started with the change in the
2013-01-12 19:30:02 -05:00
subprocess, to fix the issue in any code, and not just code using
subprocess.
2013-01-12 18:04:29 -05:00
Inherited file descriptors issues
---------------------------------
2013-01-12 19:30:02 -05:00
Closing the file descriptor in the parent process does not close the
related resource (file, socket, ...) because it is still open in the
child process.
2013-01-12 19:30:02 -05:00
The listening socket of TCPServer is not closed on ``exec()``: the
child process is able to get connection from new clients; if the
parent closes the listening socket and create a new listening socket
on the same address, it would get an "address already is used" error.
2013-01-12 19:30:02 -05:00
Not closing file descriptors can lead to resource exhaustion: even if
the parent closes all files, creating a new file descriptor may fail
with "too many files" because files are still open in the child
process.
See also the following issues:
* `Issue #2320: Race condition in subprocess using stdin
<http://bugs.python.org/issue2320>`_ (2008)
* `Issue #3006: subprocess.Popen causes socket to remain open after
close <http://bugs.python.org/issue3006>`_ (2008)
* `Issue #7213: subprocess leaks open file descriptors between Popen
instances causing hangs <http://bugs.python.org/issue7213>`_ (2009)
* `Issue #12786: subprocess wait() hangs when stdin is closed
<http://bugs.python.org/issue12786>`_ (2011)
Security
--------
2013-01-12 19:30:02 -05:00
Leaking file descriptors is a major security vulnerability. An
untrusted child process can read sensitive data like passwords and
take control of the parent process though leaked file descriptors. It
is for example a known vulnerability to escape from a chroot.
See also the CERT recommandation:
`FIO42-C. Ensure files are properly closed when they are no longer needed
<https://www.securecoding.cert.org/confluence/display/seccode/FIO42-C.+Ensure+files+are+properly+closed+when+they+are+no+longer+needed>`_.
Example of vulnerabilities:
* `OpenSSH Security Advisory: portable-keysign-rand-helper.adv
<http://www.openssh.com/txt/portable-keysign-rand-helper.adv>`_
(April 2011)
* `CWE-403: Exposure of File Descriptor to Unintended Control Sphere
<http://cwe.mitre.org/data/definitions/403.html>`_ (2008)
* `Hijacking Apache https by mod_php
<http://www.securityfocus.com/archive/1/348368>`_ (Dec 2003)
Atomicity
---------
2013-01-12 19:30:02 -05:00
Using ``fcntl()`` to set the close-on-exec flag is not safe in a
multithreaded application. If a thread calls ``fork()`` and ``exec()``
between the creation of the file descriptor and the call to
``fcntl(fd, F_SETFD, new_flags)``: the file descriptor will be
inherited by the child process. Modern operating systems offer
functions to set the flag during the creation of the file descriptor,
which avoids the race condition.
Portability
-----------
Python 3.2 added ``socket.SOCK_CLOEXEC`` flag, Python 3.3 added
2013-01-12 19:30:02 -05:00
``os.O_CLOEXEC`` flag and ``os.pipe2()`` function. It is already
possible to set atomically close-on-exec flag in Python 3.3 when
opening a file and creating a pipe or socket.
2013-01-12 19:30:02 -05:00
The problem is that these flags and functions are not portable: only
recent versions of operating systems support them. ``O_CLOEXEC`` and
``SOCK_CLOEXEC`` flags are ignored by old Linux versions and so
``FD_CLOEXEC`` flag must be checked using ``fcntl(fd, F_GETFD)``. If
the kernel ignores ``O_CLOEXEC`` or ``SOCK_CLOEXEC`` flag, a call to
``fcntl(fd, F_SETFD, flags)`` is required to set close-on-exec flag.
2013-01-12 19:30:02 -05:00
.. note::
OpenBSD older 5.2 does not close the file descriptor with
close-on-exec flag set if ``fork()`` is used before ``exec()``, but
it works correctly if ``exec()`` is called without ``fork()``. Try
`openbsd_bug.py <http://hg.python.org/peps/file/tip/pep-0433/openbsd_bug.py>`_.
Scope
-----
Applications still have to close explicitly file descriptors after a
``fork()``. The close-on-exec flag only closes file descriptors after
``exec()``, and so after ``fork()`` + ``exec()``.
2013-01-12 19:30:02 -05:00
This PEP only change the close-on-exec flag of file descriptors
created by the Python standard library, or by modules using the
standard library. Third party modules not using the standard library
should be modified to conform to this PEP. The new
``os.set_cloexec()`` function can be used for example.
Impacted functions:
* ``os.forkpty()``
* ``http.server.CGIHTTPRequestHandler.run_cgi()``
Impacted modules:
* ``multiprocessing``
* ``socketserver``
* ``subprocess``
* ``tempfile``
* ``xmlrpc.server``
* Maybe: ``signal``, ``threading``
XXX Should ``subprocess.Popen`` clear the close-on-exec flag on file
XXX descriptors of the constructor the ``pass_fds`` parameter?
2013-01-12 18:04:29 -05:00
.. note::
See `Close file descriptors after fork`_ for a possible solution
for ``fork()`` without ``exec()``.
2013-01-12 18:04:29 -05:00
Proposal
========
This PEP proposes to add a new optional parameter ``cloexec`` on
functions creating file descriptors in the Python standard library. If
the parameter is ``True``, the close-on-exec flag will be set on the
new file descriptor.
2013-01-18 06:45:30 -05:00
Add a new functions:
2013-01-18 06:45:30 -05:00
* ``os.get_cloexec(fd:int) -> bool``: get the
close-on-exec flag of a file descriptor. Not available on all platforms.
* ``os.set_cloexec(fd:int, cloexec:bool=True)``: set or clear the
close-on-exec flag on a file descriptor. Not available on all platforms.
Add a new optional ``cloexec`` parameter to:
* ``open()``: ``os.fdopen()`` is indirectly modified
* ``os.dup()``, ``os.dup2()``
* ``os.pipe()``
2013-01-12 19:30:02 -05:00
* ``socket.socket()``, ``socket.socketpair()``,
``socket.socket.accept()``
* Maybe also: ``os.open()``, ``os.openpty()``
* TODO:
* ``select.devpoll()``
* ``select.poll()``
* ``select.epoll()``
* ``select.kqueue()``
2013-01-12 19:30:02 -05:00
* ``socket.socket.recvmsg()``: use ``MSG_CMSG_CLOEXEC``,
or ``os.set_cloexec()``
The default value of the ``cloexec`` parameter is ``False`` to keep the
2013-01-12 19:30:02 -05:00
backward compatibility.
2013-01-12 19:30:02 -05:00
The close-on-exec flag will not be set on file descriptors 0 (stdin),
1 (stdout) and 2 (stderr), because these files are expected to be
inherited. It would still be possible to set close-on-exec flag
explicitly using ``os.set_cloexec()``.
2013-01-12 18:04:29 -05:00
Drawbacks:
2013-01-12 19:30:02 -05:00
* Many functions of the Python standard library creating file
descriptors cannot be changed by this proposal, because adding
a ``cloexec`` optional parameter would be surprising and too many
2013-01-12 19:30:02 -05:00
functions would need it. For example, ``os.urandom()`` uses a
temporary file on UNIX, but it calls a function of Windows API on
Windows. Adding a ``cloexec`` parameter to ``os.urandom()`` would
2013-01-25 12:32:01 -05:00
not make sense. See `Enable file descriptor inheritance by default`_
for an incomplete list of functions creating file descriptors.
2013-01-12 19:30:02 -05:00
* Checking if a module creates file descriptors is difficult. For
example, ``os.urandom()`` creates a file descriptor on UNIX to read
``/dev/urandom`` (and closes it at exit), whereas it is implemented
using a function call on Windows. It is not possible to control
close-on-exec flag of the file descriptor used by ``os.urandom()``,
because ``os.urandom()`` API does not allow it.
2013-01-12 18:04:29 -05:00
Alternatives
============
2013-01-18 06:45:30 -05:00
Bikeshedding on the name of the new parameter
---------------------------------------------
* ``inherit``, ``inherited``: closer to Windows definition
* ``sensitive``
* ``sterile``: "Does not produce offspring."
Enable file descriptor inheritance by default
---------------------------------------------
Set the close-on-exec flag by default on new file descriptors created
by Python. This alternative just changes the default value of the new
``cloexec`` parameter.
2013-01-12 19:30:02 -05:00
If a file must be inherited by child processes, ``cloexec=False``
parameter can be used.
``subprocess.Popen`` constructor has an ``pass_fds`` parameter to
2013-01-12 19:30:02 -05:00
specify which file descriptors must be inherited. The close-on-exec
flag of these file descriptors must be changed with
``os.set_cloexec()``.
2013-01-12 18:04:29 -05:00
2013-01-12 19:30:02 -05:00
Example of functions creating file descriptors which will be modified
to set close-on-exec flag:
2013-01-12 18:04:29 -05:00
* ``os.urandom()`` (on UNIX)
* ``curses.window.getwin()``, ``curses.window.putwin()``
* ``mmap.mmap()`` (if ``MAP_ANONYMOUS`` is not defined)
* ``oss.open()``
* ``Modules/main.c``: ``RunStartupFile()``
* ``Python/pythonrun.c``: ``PyRun_SimpleFileExFlags()``
* ``Modules/getpath.c``: ``search_for_exec_prefix()``
* ``Modules/zipimport.c``: ``read_directory()``
* ``Modules/_ssl.c``: ``load_dh_params()``
* ``PC/getpathp.c``: ``calculate_path()``
* ``Python/errors.c``: ``PyErr_ProgramText()``
* ``Python/import.c``: ``imp_load_dynamic()``
* TODO: ``PC/_msi.c``
Many functions are impacted indirectly by this alternative. Examples:
* ``logging.FileHandler``
Advantages of setting close-on-exec flag by default:
* There are far more programs that are bitten by FD inheritance upon
2013-01-12 19:30:02 -05:00
exec (see `Inherited file descriptors issues`_ and `Security`_)
than programs relying on it (see `Applications using inherance of
file descriptors`_).
2013-01-12 18:04:29 -05:00
Drawbacks of setting close-on-exec flag by default:
2013-01-12 19:30:02 -05:00
* The os module is written as a thin wrapper to system calls (to
functions of the C standard library). If atomic flags to set
close-on-exec flag are not supported (see `Appendix: Operating
system support`_), a single Python function call may call 2 or 3
system calls (see `Performances`_ section).
* Extra system calls, if any, may slow down Python: see
`Performances`_.
* It violates the principle of least surprise. Developers using the
os module may expect that Python respects the POSIX standard and so
that close-on-exec flag is not set by default.
2013-01-12 18:04:29 -05:00
Backward compatibility: only a few programs rely on inherance of file
2013-01-12 19:30:02 -05:00
descriptors, and they only pass a few file descriptors, usually just
one. These programs will fail immediatly with ``EBADF`` error, and it
will be simple to fix them: add ``cloexec=False`` parameter or use
2013-01-12 18:04:29 -05:00
``os.set_cloexec(fd, False)``.
The ``subprocess`` module will be changed anyway to clear
2013-01-12 19:30:02 -05:00
close-on-exec flag on file descriptors listed in the ``pass_fds``
parameter of Popen constructor. So it possible that these programs will
2013-01-12 19:30:02 -05:00
not need any fix if they use the ``subprocess`` module.
2013-01-12 18:04:29 -05:00
Add a function to set close-on-exec flag by default
---------------------------------------------------
2013-01-12 19:30:02 -05:00
An alternative is to add also a function to change globally the
default behaviour. It would be possible to set close-on-exec flag for
the whole application including all modules and the Python standard
library. This alternative is based on the `Proposal`_ and adds extra
changes.
2013-01-12 18:04:29 -05:00
New functions, command line argument and environment variable:
2013-01-12 18:04:29 -05:00
* ``sys.getdefaultcloexec() -> bool``: get the default value of the
*cloexec* parameter
* ``sys.setdefaultcloexec()``, ``-e`` command line option, ``PYTHONCLOEXEC``
environment variable (if set): set the default value of the *cloexec*
parameter to ``True``
2013-01-12 18:04:29 -05:00
The major change is that the default value of the ``cloexec`` parameter
2013-01-12 19:30:02 -05:00
is ``sys.getdefaultcloexec()``, instead of ``False``.
2013-01-12 18:04:29 -05:00
When ``sys.setdefaultcloexec()`` is called to set close-on-exec by default, we
have the same drawbacks as the `Enable file descriptor inheritance by default`_
alternative.
2013-01-12 18:04:29 -05:00
There are additionnal drawbacks of having two behaviours depending on
``sys.getdefaultcloexec()`` value:
2013-01-12 19:30:02 -05:00
* It is not more possible to know if the close-on-exec flag will be
set or not just by reading the source code.
2013-01-12 18:04:29 -05:00
Close file descriptors after fork
---------------------------------
2013-01-12 19:30:02 -05:00
This PEP does not fix issues with applications using ``fork()``
without ``exec()``. Python needs a generic process to register
callbacks which would be called after a fork, see `Add an 'atfork'
2013-01-12 19:30:02 -05:00
module`_. Such registry could be used to close file descriptors just
after a ``fork()``.
2013-01-12 18:04:29 -05:00
Drawbacks:
2013-01-12 19:30:02 -05:00
* This alternative does not solve the problem for programs using
``exec()`` without ``fork()``.
* A third party module may call directly the C function ``fork()``
which will not call "atfork" callbacks.
* All functions creating file descriptors must be changed to register
a callback and then unregister their callback when the file is
closed. Or a list of *all* open file descriptors must be
maintained.
* The operating system is a better place than Python to close
automatically file descriptors. For example, it is not easy to
avoid a race condition between closing the file and unregistering
the callback closing the file.
2013-01-12 18:04:29 -05:00
open(): add "e" flag to mode
----------------------------
A new "e" mode would set close-on-exec flag (best-effort).
2013-01-12 19:30:02 -05:00
This alternative only solves the problem for ``open()``.
socket.socket() and os.pipe() do not have a ``mode`` parameter for
2013-01-12 19:30:02 -05:00
example.
2013-01-12 19:30:02 -05:00
Since its version 2.7, the GNU libc supports ``"e"`` flag for
``fopen()``. It uses ``O_CLOEXEC`` if available, or use ``fcntl(fd,
F_SETFD, FD_CLOEXEC)``. With Visual Studio, fopen() accepts a "N"
flag which uses ``O_NOINHERIT``.
2013-01-12 18:04:29 -05:00
Applications using inherance of file descriptors
================================================
2013-01-12 19:30:02 -05:00
Most developers don't know that file descriptors are inherited by
default. Most programs do not rely on inherance of file descriptors.
For example, ``subprocess.Popen`` was changed in Python 3.2 to close
all file descriptors greater than 2 in the child process by default.
No user complained about this behavior change yet.
2013-01-12 19:30:02 -05:00
Network servers using fork may want to pass the client socket to the
child process. For example, on UNIX a CGI server pass the socket
client through file descriptors 0 (stdin) and 1 (stdout) using
``dup2()``. This specific case is not impacted by this PEP because the
close-on-exec flag is never set on file descriptors smaller than 3.
To access a restricted resource like creating a socket listening on a
TCP port lower than 1024 or reading a file containing sensitive data
like passwords, a common practice is: start as the root user, create a
file descriptor, create a child process, pass the file descriptor to
the child process and exit. Security is very important in such use
case: leaking another file descriptor would be a critical security
vulnerability (see `Security`_). The root process may not exit but
monitors the child process instead, and restarts a new child process
and pass the same file descriptor if the previous child process
2013-01-12 18:04:29 -05:00
crashed.
2013-01-12 19:30:02 -05:00
Example of programs taking file descriptors from the parent process
using a command line option:
2013-01-12 18:04:29 -05:00
* gpg: ``--status-fd <fd>``, ``--logger-fd <fd>``, etc.
* openssl: ``-pass fd:<fd>``
* qemu: ``-add-fd <fd>``
* valgrind: ``--log-fd=<fd>``, ``--input-fd=<fd>``, etc.
* xterm: ``-S <fd>``
2013-01-12 19:30:02 -05:00
On Linux, it is possible to use ``"/dev/fd/<fd>"`` filename to pass a
file descriptor to a program expecting a filename.
Performances
============
2013-01-12 19:30:02 -05:00
Setting close-on-exec flag may require additional system calls for
each creation of new file descriptors. The number of additional system
calls depends on the method used to set the flag:
* ``O_NOINHERIT``: no additionnal system call
2013-01-12 19:30:02 -05:00
* ``O_CLOEXEC``: one addition system call, but only at the creation
of the first file descriptor, to check if the flag is supported. If
no, Python has to fallback to the next method.
* ``ioctl(fd, FIOCLEX)``: one addition system call per file
descriptor
* ``fcntl(fd, F_SETFD, flags)``: two addition system calls per file
descriptor, one to get old flags and one to set new flags
On Linux, setting the close-on-flag has a low overhead on performances.
Results of
`bench_cloexec.py <http://hg.python.org/peps/file/tip/pep-0433/bench_cloexec.py>`_
on Linux 3.6:
* close-on-flag not set: 7.8 us
* ``O_CLOEXEC``: 1% slower (7.9 us)
* ``ioctl()``: 3% slower (8.0 us)
* ``fcntl()``: 3% slower (8.0 us)
Implementation
==============
2013-01-18 06:45:30 -05:00
os.get_cloexec(fd)
------------------
2013-01-18 06:45:30 -05:00
Get the close-on-exec flag of a file descriptor.
Pseudo-code::
if os.name == 'nt':
def get_cloexec(fd):
handle = _winapi._get_osfhandle(fd);
flags = _winapi.GetHandleInformation(handle)
return not(flags & _winapi.HANDLE_FLAG_INHERIT)
else:
try:
import fcntl
except ImportError:
pass
else:
def get_cloexec(fd):
flags = fcntl.fcntl(fd, fcntl.F_GETFD)
return bool(flags & fcntl.FD_CLOEXEC)
os.set_cloexec(fd, cloexec=True)
--------------------------------
Set or clear the close-on-exec flag on a file descriptor. The flag
is set after the creation of the file descriptor and so it is not
atomic.
Pseudo-code::
if os.name == 'nt':
def set_cloexec(fd, cloexec=True):
2013-01-18 06:45:30 -05:00
handle = _winapi._get_osfhandle(fd);
mask = _winapi.HANDLE_FLAG_INHERIT
if cloexec:
flags = 0
else:
flags = mask
_winapi.SetHandleInformation(handle, mask, flags)
else:
fnctl = None
ioctl = None
try:
import ioctl
except ImportError:
try:
import fcntl
except ImportError:
pass
if ioctl is not None and hasattr('FIOCLEX', ioctl):
def set_cloexec(fd, cloexec=True):
if cloexec:
ioctl.ioctl(fd, ioctl.FIOCLEX)
else:
ioctl.ioctl(fd, ioctl.FIONCLEX)
elif fnctl is not None:
def set_cloexec(fd, cloexec=True):
flags = fcntl.fcntl(fd, fcntl.F_GETFD)
if cloexec:
flags |= FD_CLOEXEC
else:
flags &= ~FD_CLOEXEC
fcntl.fcntl(fd, fcntl.F_SETFD, flags)
2013-01-12 19:30:02 -05:00
ioctl is preferred over fcntl because it requires only one syscall,
instead of two syscalls for fcntl.
2013-01-12 19:30:02 -05:00
.. note::
``fcntl(fd, F_SETFD, flags)`` only supports one flag
(``FD_CLOEXEC``), so it would be possible to avoid ``fcntl(fd,
F_GETFD)``. But it may drop other flags in the future, and so it is
safer to keep the two functions calls.
.. note::
``fopen()`` function of the GNU libc ignores the error if
``fcntl(fd, F_SETFD, flags)`` failed.
open()
------
* Windows: ``open()`` with ``O_NOINHERIT`` flag [atomic]
* ``open()`` with ``O_CLOEXEC flag`` [atomic]
* ``open()`` + ``os.set_cloexec(fd, True)`` [best-effort]
os.dup()
--------
* ``fcntl(fd, F_DUPFD_CLOEXEC)`` [atomic]
* ``dup()`` + ``os.set_cloexec(fd, True)`` [best-effort]
os.dup2()
---------
* ``dup3()`` with ``O_CLOEXEC`` flag [atomic]
* ``dup2()`` + ``os.set_cloexec(fd, True)`` [best-effort]
os.pipe()
---------
2013-01-18 06:45:30 -05:00
* Windows: ``CreatePipe()`` with ``SECURITY_ATTRIBUTES.bInheritHandle=TRUE``,
or ``_pipe()`` with ``O_NOINHERIT`` flag [atomic]
* ``pipe2()`` with ``O_CLOEXEC`` flag [atomic]
* ``pipe()`` + ``os.set_cloexec(fd, True)`` [best-effort]
socket.socket()
---------------
2013-01-18 06:45:30 -05:00
* Windows: ``WSASocket()`` with ``WSA_FLAG_NO_HANDLE_INHERIT`` flag
* ``socket()`` with ``SOCK_CLOEXEC`` flag [atomic]
* ``socket()`` + ``os.set_cloexec(fd, True)`` [best-effort]
socket.socketpair()
-------------------
* ``socketpair()`` with ``SOCK_CLOEXEC`` flag [atomic]
* ``socketpair()`` + ``os.set_cloexec(fd, True)`` [best-effort]
socket.socket.accept()
----------------------
* ``accept4()`` with ``SOCK_CLOEXEC`` flag [atomic]
* ``accept()`` + ``os.set_cloexec(fd, True)`` [best-effort]
Backward compatibility
======================
2013-01-12 19:30:02 -05:00
There is no backward incompatible change. The default behaviour is
unchanged: the close-on-exec flag is not set by default.
Appendix: Operating system support
==================================
Windows
-------
2013-01-12 19:30:02 -05:00
Windows has an ``O_NOINHERIT`` flag: "Do not inherit in child
processes".
For example, it is supported by ``open()`` and ``_pipe()``.
The value of the flag can be modified using:
``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``.
``CreateProcess()`` has an ``bInheritHandles`` parameter: if it is
2013-01-12 19:30:02 -05:00
FALSE, the handles are not inherited. It is used by
``subprocess.Popen`` with ``close_fds`` option.
fcntl
-----
Functions:
* ``fcntl(fd, F_GETFD)``
* ``fcntl(fd, F_SETFD, flags | FD_CLOEXEC)``
2013-01-12 19:30:02 -05:00
Availability: AIX, Digital UNIX, FreeBSD, HP-UX, IRIX, Linux, Mac OS
X, OpenBSD, Solaris, SunOS, Unicos.
ioctl
-----
Functions:
* ``ioctl(fd, FIOCLEX, 0)`` sets close-on-exec flag
* ``ioctl(fd, FIONCLEX, 0)`` clears close-on-exec flag
Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.
Atomic flags
------------
New flags:
2013-01-12 19:30:02 -05:00
* ``O_CLOEXEC``: available on Linux (2.6.23+), FreeBSD (8.3+),
OpenBSD 5.0, QNX, BeOS, next NetBSD release (6.1?). This flag is
part of POSIX.1-2008.
2013-01-18 06:45:30 -05:00
* ``SOCK_CLOEXEC`` flag for ``socket()`` and ``socketpair()``,
available on Linux 2.6.27+, OpenBSD 5.2, NetBSD 6.0.
* ``WSA_FLAG_NO_HANDLE_INHERIT`` flag for ``WSASocket()``: supported on
Windows 7 with SP1, Windows Server 2008 R2 with SP1, and later
* ``fcntl()``: ``F_DUPFD_CLOEXEC`` flag, available on Linux 2.6.24+,
2013-01-12 19:30:02 -05:00
OpenBSD 5.0, FreeBSD 9.1, NetBSD 6.0. This flag is part of
POSIX.1-2008.
* ``recvmsg()``: ``MSG_CMSG_CLOEXEC``, available on Linux 2.6.23+,
NetBSD 6.0.
2013-01-12 19:30:02 -05:00
On Linux older than 2.6.23, ``O_CLOEXEC`` flag is simply ignored. So
we have to check that the flag is supported by calling ``fcntl()``. If
it does not work, we have to set the flag using ``fcntl()``.
On Linux older than 2.6.27, if the ``SOCK_CLOEXEC`` flag is set in the
socket type, ``socket()`` or ``socketpair()`` fail and ``errno`` is set
to ``EINVAL``.
New functions:
* ``dup3()``: available on Linux 2.6.27+ (and glibc 2.9)
* ``pipe2()``: available on Linux 2.6.27+ (and glibc 2.9)
* ``accept4()``: available on Linux 2.6.28+ (and glibc 2.10)
2013-01-12 19:30:02 -05:00
If ``accept4()`` is called on Linux older than 2.6.28, ``accept4()``
returns ``-1`` (fail) and ``errno`` is set to ``ENOSYS``.
Links
=====
Links:
* `Secure File Descriptor Handling
2013-01-12 19:30:02 -05:00
<http://udrepper.livejournal.com/20407.html>`_ (Ulrich Drepper,
2008)
* `win32_support.py of the Tornado project
<https://bitbucket.org/pvl/gaeseries-tornado/src/c2671cea1842/tornado/win32_support.py>`_:
emulate fcntl(fd, F_SETFD, FD_CLOEXEC) using
``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``
Python issues:
* `open() does not able to set flags, such as O_CLOEXEC
<http://bugs.python.org/issue12105>`_
* `Add "e" mode to open(): close-and-exec (O_CLOEXEC) / O_NOINHERIT
<http://bugs.python.org/issue16850>`_
* `TCP listening sockets created without FD_CLOEXEC flag
<http://bugs.python.org/issue12107>`_
* `Use O_CLOEXEC in the tempfile module
<http://bugs.python.org/issue16860>`_
* `Support accept4() for atomic setting of flags at socket creation
<http://bugs.python.org/issue10115>`_
* `Add an 'atfork' module
<http://bugs.python.org/issue16500>`_
* `Implementation of the PEP 433
<http://bugs.python.org/issue17036>`_
Ruby:
* `Set FD_CLOEXEC for all fds (except 0, 1, 2)
<http://bugs.ruby-lang.org/issues/5041>`_
* `O_CLOEXEC flag missing for Kernel::open
<http://bugs.ruby-lang.org/issues/1291>`_:
`commit reverted
2013-01-12 19:30:02 -05:00
<http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/31643>`_
later
Footnotes
=========
2013-01-12 19:30:02 -05:00
.. [#subprocess_close] On UNIX since Python 3.2, subprocess.Popen()
closes all file descriptors by default: ``close_fds=True``. It
closes file descriptors in range 3 inclusive to ``local_max_fd``
exclusive, where ``local_max_fd`` is ``fcntl(0, F_MAXFD)`` on
NetBSD, or ``sysconf(_SC_OPEN_MAX)`` otherwise. If the error pipe
has a descriptor smaller than 3, ``ValueError`` is raised.