python-peps/pep-0446.txt

PEP: 446
Title: Add new cloexec and blocking parameters to functions creating file descriptors
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 3-July-2013
Python-Version: 3.4


Abstract
========

This PEP proposes new portable parameters and functions to configure the
inherance of file descriptors and the non-blocking flag of sockets.


Rationale
=========

Inherance of file descriptors
-----------------------------

The inherance of a file descriptor in a child process at the execution
of a new program can be configured on each file descriptor using a
*close-on-exec* flag. By default, the close-on-exec flag is not set. On
Windows, file descriptors are not inherited if the ``bInheritHandles``
parameter of the ``CreateProcess()`` function is ``FALSE``, even if the
close-on-exec flag is not set. On UNIX, file descriptors are inherited
by default.

Inherance of file descriptors causes issues. For example, closing a file
descriptor in the parent process does not release the resource (file,
socket, ...), because the file descriptor is still open in the child
process.

Leaking file descriptors is a major security vulnerability. An
untrusted child process can read sensitive data like passwords and
take control of the parent process though leaked file descriptors. It
is for example a known vulnerability to escape from a chroot.


Non-blocking sockets
--------------------

To handle multiple network clients in a single thread, a multiplexing
function like ``select()`` can be used. For best performances, sockets
must be configured as non-blocking.

By default, newly created sockets are blocking. Setting the non-blocking
mode requires extra system calls.


Setting flags at the creation of the file descriptor
----------------------------------------------------

Windows and recent versions of other operating systems like Linux
support setting close-on-exec and blocking flags directly at the
creation of file descriptors and sockets.

Setting these flags at the creation is atomic and avoids extra system
calls.


Proposal
========

New cloexec And blocking Parameters
-----------------------------------

Add a new optional *cloexec* on functions creating file descriptors:

* ``io.FileIO``
* ``io.open()``
* ``open()``
* ``os.dup()``
* ``os.dup2()``
* ``os.fdopen()``
* ``os.open()``
* ``os.openpty()``
* ``os.pipe()``
* ``select.devpoll()``
* ``select.epoll()``
* ``select.kqueue()``

Add new optional *cloexec* and *blocking* parameters to functions
creating sockets:

* ``asyncore.dispatcher.create_socket()``
* ``socket.socket()``
* ``socket.socket.accept()``
* ``socket.socket.dup()``
* ``socket.socket.fromfd``
* ``socket.socketpair()``

The default value of *cloexec* is ``False``, and the default value of
*blocking* is ``True``.

The atomicity is not guaranteed. If the platform does not support
setting close-on-exec and blocking flags at the creation of the file
descriptor, the flags are set using extra system calls.

New Functions
-------------

Add new functions the get and set the close-on-exec flag of a file
descriptor:

* ``os.get_cloexec(fd:int) -> bool``
* ``os.set_cloexec(fd:int, cloexec: bool)``


Other Changes
-------------

The ``subprocess.Popen`` class must clear the close-on-exec flag of file
descriptors of the ``pass_fds`` parameter.

The close-on-exec flag must also be set on private file descriptors and
sockets in the Python standard library. For example, on UNIX,
os.urandom() opens ``/dev/urandom`` to read some random bytes, the file
descriptor is closed at function exit.  The file descriptor is not
expected to be inherited on execution of a new program in a child
process.


Alternatives
============

The PEP 433 is a previous attempt proposing various other alternatives,
but no consensus could be reached.

This PEP is much simpler, more conservative (no backward compatibility
issue) and has a well defined behaviour (the default value of the new
*cloexec* parameter is not configurable).


Links
=====

Python issues:

* `#10115: Support accept4() for atomic setting of flags at socket
  creation <http://bugs.python.org/issue10115>`_
* `#12105: open() does not able to set flags, such as O_CLOEXEC
  <http://bugs.python.org/issue12105>`_
* `#12107: TCP listening sockets created without FD_CLOEXEC flag
  <http://bugs.python.org/issue12107>`_
* `#16850: Add "e" mode to open(): close-and-exec
  (O_CLOEXEC) / O_NOINHERIT <http://bugs.python.org/issue16850>`_
* `#16860: Use O_CLOEXEC in the tempfile module
  <http://bugs.python.org/issue16860>`_
* `#16946: subprocess: _close_open_fd_range_safe() does not set
  close-on-exec flag on Linux < 2.6.23 if O_CLOEXEC is defined
  <http://bugs.python.org/issue16946>`_
* `#17070: PEP 433: Use the new cloexec to improve security and avoid
  bugs <http://bugs.python.org/issue17070>`_

Other links:

* `Secure File Descriptor Handling
  <http://udrepper.livejournal.com/20407.html>`_ (Ulrich Drepper,
  2008)


Copyright
=========

This document has been placed into the public domain.