PEP 446: add an "Overlapped I/O" alternative
This commit is contained in:
Victor Stinner
parent
1043aa17ab
commit
7394d0aceb
108
pep-0446.txt
108
pep-0446.txt
|
|
@ -23,23 +23,31 @@ Rationale
|
|||
Inherance of file descriptors
|
||||
-----------------------------
|
||||
|
||||
The inherance of a file descriptor in a child process at the execution
|
||||
of a new program can be configured on each file descriptor using a
|
||||
*close-on-exec* flag. By default, the close-on-exec flag is not set. On
|
||||
Windows, file descriptors are not inherited if the ``bInheritHandles``
|
||||
parameter of the ``CreateProcess()`` function is ``FALSE``, even if the
|
||||
close-on-exec flag is not set. On UNIX, file descriptors are inherited
|
||||
by default.
|
||||
The inherance of file descriptors in child processes can be configured
|
||||
on each file descriptor using a *close-on-exec* flag. By default, the
|
||||
close-on-exec flag is not set.
|
||||
|
||||
On Windows, file descriptors are not inherited if the
|
||||
``bInheritHandles`` parameter of the ``CreateProcess()`` function is
|
||||
``FALSE``, even if the close-on-exec flag is not set.
|
||||
|
||||
On UNIX, file descriptors with the close-and-exec flag set are closed at
|
||||
the execution of a new program (ex: when calling ``execv()``). The flag
|
||||
has no effect on ``fork()``, all file descriptors are inherited by the
|
||||
child process.
|
||||
|
||||
Issues of the inherance of file descriptors
|
||||
-------------------------------------------
|
||||
|
||||
Inherance of file descriptors causes issues. For example, closing a file
|
||||
descriptor in the parent process does not release the resource (file,
|
||||
socket, ...), because the file descriptor is still open in the child
|
||||
process.
|
||||
|
||||
Leaking file descriptors is a major security vulnerability. An
|
||||
untrusted child process can read sensitive data like passwords and
|
||||
take control of the parent process though leaked file descriptors. It
|
||||
is for example a known vulnerability to escape from a chroot.
|
||||
Leaking file descriptors is also a major security vulnerability. An
|
||||
untrusted child process can read sensitive data like passwords and take
|
||||
control of the parent process though leaked file descriptors. It is for
|
||||
example a known vulnerability to escape from a chroot.
|
||||
|
||||
|
||||
Non-blocking sockets
|
||||
|
|
@ -47,21 +55,24 @@ Non-blocking sockets
|
|||
|
||||
To handle multiple network clients in a single thread, a multiplexing
|
||||
function like ``select()`` can be used. For best performances, sockets
|
||||
must be configured as non-blocking.
|
||||
must be configured as non-blocking. Operations like ``send()`` and
|
||||
``recv()`` return an ``EAGAIN`` or ``EWOULDBLOCK`` error if the
|
||||
operation would block.
|
||||
|
||||
By default, newly created sockets are blocking. Setting the non-blocking
|
||||
mode requires extra system calls.
|
||||
mode requires additional system calls.
|
||||
|
||||
|
||||
Setting flags at the creation of the file descriptor
|
||||
----------------------------------------------------
|
||||
|
||||
Windows and recent versions of other operating systems like Linux
|
||||
support setting close-on-exec and blocking flags directly at the
|
||||
creation of file descriptors and sockets.
|
||||
support setting the close-on-exec flag directly at the creation of file
|
||||
descriptors, and close-on-exec and blocking flags at the creation of
|
||||
sockets.
|
||||
|
||||
Setting these flags at the creation is atomic and avoids extra system
|
||||
calls.
|
||||
Setting these flags at the creation is atomic and avoids additional
|
||||
system calls.
|
||||
|
||||
|
||||
Proposal
|
||||
|
|
@ -95,12 +106,12 @@ creating sockets:
|
|||
* ``socket.socket.fromfd``
|
||||
* ``socket.socketpair()``
|
||||
|
||||
The default value of *cloexec* is ``False``, and the default value of
|
||||
The default value of *cloexec* is ``False`` and the default value of
|
||||
*blocking* is ``True``.
|
||||
|
||||
The atomicity is not guaranteed. If the platform does not support
|
||||
setting close-on-exec and blocking flags at the creation of the file
|
||||
descriptor, the flags are set using extra system calls.
|
||||
descriptor or socket, the flags are set using additional system calls.
|
||||
|
||||
New Functions
|
||||
-------------
|
||||
|
|
@ -120,21 +131,54 @@ descriptors of the ``pass_fds`` parameter.
|
|||
|
||||
The close-on-exec flag must also be set on private file descriptors and
|
||||
sockets in the Python standard library. For example, on UNIX,
|
||||
os.urandom() opens ``/dev/urandom`` to read some random bytes, the file
|
||||
descriptor is closed at function exit. The file descriptor is not
|
||||
expected to be inherited on execution of a new program in a child
|
||||
process.
|
||||
os.urandom() opens ``/dev/urandom`` to read some random bytes and the
|
||||
file descriptor is closed at function exit. The file descriptor is not
|
||||
expected to be inherited by child processes.
|
||||
|
||||
|
||||
Alternatives
|
||||
============
|
||||
Rejected Alternatives
|
||||
=====================
|
||||
|
||||
The PEP 433 is a previous attempt proposing various other alternatives,
|
||||
but no consensus could be reached.
|
||||
PEP 433
|
||||
-------
|
||||
|
||||
This PEP is much simpler, more conservative (no backward compatibility
|
||||
issue) and has a well defined behaviour (the default value of the new
|
||||
*cloexec* parameter is not configurable).
|
||||
The PEP 433 entitled "Easier suppression of file descriptor inheritance"
|
||||
is a previous attempt proposing various other alternatives, but no
|
||||
consensus could be reached.
|
||||
|
||||
This PEP has a well defined behaviour (the default value of the new
|
||||
*cloexec* parameter is not configurable), is more conservative (no
|
||||
backward compatibility issue), and is much simpler.
|
||||
|
||||
|
||||
Add blocking parameter for file descriptors and Windows overlapped I/O
|
||||
----------------------------------------------------------------------
|
||||
|
||||
Windows supports non-blocking operations on files using an extension of
|
||||
the Windows API called "Overlapped I/O". Using this extension requires
|
||||
to modify the Python standard library and applications to pass a
|
||||
``OVERLAPPED`` structure and an event loop to wait for the completion of
|
||||
operations.
|
||||
|
||||
This PEP only tries to expose portable flags on file descriptors and
|
||||
sockets. Supporting overlapped I/O requires an abstraction providing a
|
||||
high-level and portable API for asynchronous operations on files and
|
||||
sockets. Overlapped I/O are out of the scope of this PEP.
|
||||
|
||||
UNIX supports non-blocking files, moreover recent versions of operating
|
||||
systems support setting the non-blocking flag at the creation of a file
|
||||
descriptor. It would be possible to add a new optional *blocking*
|
||||
parameter to Python functions creating file descriptors. On Windows,
|
||||
creating a file descriptor with ``blocking=False`` would raise a
|
||||
``NotImplementedError``. This behaviour is not acceptable for the ``os``
|
||||
module which is designed as a thin wrapper on the C functions of the
|
||||
operating system. If a platform does not support a function, the
|
||||
function should not be available on the platform. For example,
|
||||
the ``os.fork()`` function is not available on Windows.
|
||||
|
||||
For all these reasons, this alternative was rejected. The PEP 3156
|
||||
proposes an abstraction for asynchronous I/O supporting non-blocking
|
||||
files on Windows.
|
||||
|
||||
|
||||
Links
|
||||
|
|
@ -155,8 +199,8 @@ Python issues:
|
|||
* `#16946: subprocess: _close_open_fd_range_safe() does not set
|
||||
close-on-exec flag on Linux < 2.6.23 if O_CLOEXEC is defined
|
||||
<http://bugs.python.org/issue16946>`_
|
||||
* `#17070: PEP 433: Use the new cloexec to improve security and avoid
|
||||
bugs <http://bugs.python.org/issue17070>`_
|
||||
* `#17070: Use the new cloexec to improve security and avoid bugs
|
||||
<http://bugs.python.org/issue17070>`_
|
||||
|
||||
Other links:
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue