PEP 446: add an "Overlapped I/O" alternative

pep-0446.txt

@@ -23,23 +23,31 @@ Rationale
 Inherance of file descriptors
 -----------------------------
 
-The inherance of a file descriptor in a child process at the execution
+The inherance of file descriptors in child processes can be configured
-of a new program can be configured on each file descriptor using a
+on each file descriptor using a *close-on-exec* flag. By default, the
-*close-on-exec* flag. By default, the close-on-exec flag is not set. On
+close-on-exec flag is not set.
-Windows, file descriptors are not inherited if the ``bInheritHandles``
+
-parameter of the ``CreateProcess()`` function is ``FALSE``, even if the
+On Windows, file descriptors are not inherited if the
-close-on-exec flag is not set. On UNIX, file descriptors are inherited
+``bInheritHandles`` parameter of the ``CreateProcess()`` function is
-by default.
+``FALSE``, even if the close-on-exec flag is not set.
+
+On UNIX, file descriptors with the close-and-exec flag set are closed at
+the execution of a new program (ex: when calling ``execv()``). The flag
+has no effect on ``fork()``, all file descriptors are inherited by the
+child process.
+
+Issues of the inherance of file descriptors
+-------------------------------------------
 
 Inherance of file descriptors causes issues. For example, closing a file
 descriptor in the parent process does not release the resource (file,
 socket, ...), because the file descriptor is still open in the child
 process.
 
-Leaking file descriptors is a major security vulnerability. An
+Leaking file descriptors is also a major security vulnerability. An
-untrusted child process can read sensitive data like passwords and
+untrusted child process can read sensitive data like passwords and take
-take control of the parent process though leaked file descriptors. It
+control of the parent process though leaked file descriptors. It is for
-is for example a known vulnerability to escape from a chroot.
+example a known vulnerability to escape from a chroot.
 
 
 Non-blocking sockets
@@ -47,21 +55,24 @@ Non-blocking sockets
 
 To handle multiple network clients in a single thread, a multiplexing
 function like ``select()`` can be used. For best performances, sockets
-must be configured as non-blocking.
+must be configured as non-blocking. Operations like ``send()`` and
+``recv()`` return an ``EAGAIN`` or ``EWOULDBLOCK`` error if the
+operation would block.
 
 By default, newly created sockets are blocking. Setting the non-blocking
-mode requires extra system calls.
+mode requires additional system calls.
 
 
 Setting flags at the creation of the file descriptor
 ----------------------------------------------------
 
 Windows and recent versions of other operating systems like Linux
-support setting close-on-exec and blocking flags directly at the
+support setting the close-on-exec flag directly at the creation of file
-creation of file descriptors and sockets.
+descriptors, and close-on-exec and blocking flags at the creation of
+sockets.
 
-Setting these flags at the creation is atomic and avoids extra system
+Setting these flags at the creation is atomic and avoids additional
-calls.
+system calls.
 
 
 Proposal
@@ -95,12 +106,12 @@ creating sockets:
 * ``socket.socket.fromfd``
 * ``socket.socketpair()``
 
-The default value of *cloexec* is ``False``, and the default value of
+The default value of *cloexec* is ``False`` and the default value of
 *blocking* is ``True``.
 
 The atomicity is not guaranteed. If the platform does not support
 setting close-on-exec and blocking flags at the creation of the file
-descriptor, the flags are set using extra system calls.
+descriptor or socket, the flags are set using additional system calls.
 
 New Functions
 -------------
@@ -120,21 +131,54 @@ descriptors of the ``pass_fds`` parameter.
 
 The close-on-exec flag must also be set on private file descriptors and
 sockets in the Python standard library. For example, on UNIX,
-os.urandom() opens ``/dev/urandom`` to read some random bytes, the file
+os.urandom() opens ``/dev/urandom`` to read some random bytes and the
-descriptor is closed at function exit.  The file descriptor is not
+file descriptor is closed at function exit. The file descriptor is not
-expected to be inherited on execution of a new program in a child
+expected to be inherited by child processes.
-process.
 
 
-Alternatives
+Rejected Alternatives
-============
+=====================
 
-The PEP 433 is a previous attempt proposing various other alternatives,
+PEP 433
-but no consensus could be reached.
+-------
 
-This PEP is much simpler, more conservative (no backward compatibility
+The PEP 433 entitled "Easier suppression of file descriptor inheritance"
-issue) and has a well defined behaviour (the default value of the new
+is a previous attempt proposing various other alternatives, but no
-*cloexec* parameter is not configurable).
+consensus could be reached.
+
+This PEP has a well defined behaviour (the default value of the new
+*cloexec* parameter is not configurable), is more conservative (no
+backward compatibility issue), and is much simpler.
+
+
+Add blocking parameter for file descriptors and Windows overlapped I/O
+----------------------------------------------------------------------
+
+Windows supports non-blocking operations on files using an extension of
+the Windows API called "Overlapped I/O". Using this extension requires
+to modify the Python standard library and applications to pass a
+``OVERLAPPED`` structure and an event loop to wait for the completion of
+operations.
+
+This PEP only tries to expose portable flags on file descriptors and
+sockets. Supporting overlapped I/O requires an abstraction providing a
+high-level and portable API for asynchronous operations on files and
+sockets. Overlapped I/O are out of the scope of this PEP.
+
+UNIX supports non-blocking files, moreover recent versions of operating
+systems support setting the non-blocking flag at the creation of a file
+descriptor. It would be possible to add a new optional *blocking*
+parameter to Python functions creating file descriptors. On Windows,
+creating a file descriptor with ``blocking=False``  would raise a
+``NotImplementedError``. This behaviour is not acceptable for the ``os``
+module which is designed as a thin wrapper on the C functions of the
+operating system. If a platform does not support a function, the
+function should not be available on the platform. For example,
+the ``os.fork()`` function is not available on Windows.
+
+For all these reasons, this alternative was rejected. The PEP 3156
+proposes an abstraction for asynchronous I/O supporting non-blocking
+files on Windows.
 
 
 Links
@@ -155,8 +199,8 @@ Python issues:
 * `#16946: subprocess: _close_open_fd_range_safe() does not set
   close-on-exec flag on Linux < 2.6.23 if O_CLOEXEC is defined
   <http://bugs.python.org/issue16946>`_
-* `#17070: PEP 433: Use the new cloexec to improve security and avoid
+* `#17070: Use the new cloexec to improve security and avoid bugs
-  bugs <http://bugs.python.org/issue17070>`_
+  <http://bugs.python.org/issue17070>`_
 
 Other links: