Moves os.open_for_exec to _imp.open_for_import and simplifies the signature. (#420)

pep-0551.rst

@@ -268,58 +268,53 @@ integrate with these when launching scripts or importing modules.
 The new public C API for the verified open hook is::
 
    # Set the handler
-   typedef PyObject *(*handler_func)(const char *narrow,
+   typedef PyObject *(*hook_func)(PyObject *path)
-                                     const wchar_t *wide)
+   int PyImport_SetOpenForImportHook(void *handler)
-   int PyOS_SetOpenForExecuteHandler(handler_func handler)
 
    # Open a file using the handler
-   PyObject *PyOS_OpenForExec(PyObject *path)
+   PyObject *PyImport_OpenForImport(const char *path)
 
 The new public Python API for the verified open hook is::
 
    # Open a file using the handler
-   os.open_for_exec(pathlike)
+   _imp.open_for_import(path)
 
-The ``os.open_for_exec()`` function is a drop-in replacement for
+The ``_imp.open_for_import()`` function is a drop-in replacement for
-``open(pathlike, 'rb')``. Its default behaviour is to open a file for
+``open(str(pathlike), 'rb')``. Its default behaviour is to open a file
-raw, binary access - any more restrictive behaviour requires the use of
+for raw, binary access - any more restrictive behaviour requires the
-a custom handler. (Aside: since ``importlib`` requires access to this
+use of a custom handler. Only ``str`` arguments are accepted.
-function before the ``os`` module has been imported, it will be
-available on the ``nt``/``posix`` modules, but the intent is that other
-users will access it through the ``os`` module.)
 
-A custom handler may be set by calling ``Py_SetOpenForExecuteHandler()``
+A custom handler may be set by calling ``PyImport_SetOpenForImportHook()``
 from C at any time, including before ``Py_Initialize()``. However, if a
-handler has already been set then the call will fail. When
+hook has already been set then the call will fail. When
-``open_for_exec()`` is called with a handler set, the handler will be
+``open_for_import()`` is called with a hook set, the hook will be passed
-passed the processed narrow or wide path, depending on platform, and its
+the path and its return value will be returned directly. The returned
-return value will be returned directly. The returned object should be an
+object should be an open file-like object that supports reading raw
-open file-like object that supports reading raw bytes. This is
+bytes. This is explicitly intended to allow a ``BytesIO`` instance if
-explicitly intended to allow a ``BytesIO`` instance if the open handler
+the open handler has already had to read the file into memory in order
-has already had to read the file into memory in order to perform
+to perform whatever verification is necessary to determine whether the
-whatever verification is necessary to determine whether the content is
+content is permitted to be executed.
-permitted to be executed.
 
-Note that these handlers can import and call the ``_io.open()`` function
+Note that these hooks can import and call the ``_io.open()`` function on
-on CPython without triggering themselves.
+CPython without triggering themselves.
 
-If the handler determines that the file is not suitable for execution,
+If the hook determines that the file is not suitable for execution, it
-it should raise an exception of its choice, as well as raising any other
+should raise an exception of its choice, as well as raising any other
 auditing events or notifications.
 
 All import and execution functionality involving code from a file will
-be changed to use ``open_for_exec()`` unconditionally. It is important
+be changed to use ``open_for_import()`` unconditionally. It is important
 to note that calls to ``compile()``, ``exec()`` and ``eval()`` do not go
 through this function - an audit hook that includes the code from these
 calls will be added and is the best opportunity to validate code that is
 read from the file. Given the current decoupling between import and
 execution in Python, most imported code will go through both
-``open_for_exec()`` and the log hook for ``compile``, and so care should
+``open_for_import()`` and the log hook for ``compile``, and so care
-be taken to avoid repeating verification steps.
+should be taken to avoid repeating verification steps.
 
 .. note::
-   The use of ``open_for_exec()`` by ``importlib`` is a valuable first
+   The use of ``open_for_import()`` by ``importlib`` is a valuable
-   defence, but should not be relied upon to prevent misuse. In
+   first defence, but should not be relied upon to prevent misuse. In
    particular, it is easy to monkeypatch ``importlib`` in order to
    bypass the call. Auditing hooks are the primary way to achieve
    security transparency, and are essential for detecting attempts to
@@ -341,10 +336,10 @@ call will have any effect. (Including existence tests in
 security-critical code allows another vector to bypass auditing, so it
 is preferable that the function always exist.)
 
-``os.open_for_exec(pathlike)`` should at a minimum always return
+``_imp.open_for_import(path)`` should at a minimum always return
-``_io.open(pathlike, 'rb')``. Code using the function should make no
+``_io.open(path, 'rb')``. Code using the function should make no further
-further assumptions about what may occur, and implementations other than
+assumptions about what may occur, and implementations other than CPython
-CPython are not required to let developers override the behavior of this
+are not required to let developers override the behavior of this
 function with a hook.
 
 Audit Hook Locations
@@ -366,14 +361,14 @@ should be considered part of the rationale for including the hook.
    hooks they are being cleaned up, mainly in case the event is
    triggered unexpectedly. This event cannot be aborted.
    "
-   ``Py_SetOpenForExecuteHandler``, ``setopenforexecutehandler``, "", "
+   ``PyImport_SetOpenForImportHook``, ``setopenforimporthook``, "", "
-   Detects any attempt to set the ``open_for_execute`` handler.
+   Detects any attempt to set the ``open_for_import`` hook.
    "
    "``compile``, ``exec``, ``eval``, ``PyAst_CompileString``,
    ``PyAST_obj2mod``", ``compile``, "``(code, filename_or_none)``", "
    Detect dynamic code compilation, where ``code`` could be a string or
    AST. Note that this will be called for regular imports of source
-   code, including those that were opened with ``open_for_exec``.
+   code, including those that were opened with ``open_for_import``.
    "
    "``exec``, ``eval``, ``run_mod``", ``exec``, "``(code_object,)``", "
    Detect dynamic execution of code objects. This only occurs for
@@ -540,11 +535,11 @@ preventing any other hooks from being added.
 
 **Restrict importable modules**
 
-Also before initialization, ``spython`` will set an open-for-execute
+Also before initialization, ``spython`` will set an open-for-import
-hook that validates all files opened with ``os.open_for_exec``. This
+hook that validates all files opened with ``os.open_for_import``. This
 implementation will require all files to have a ``.py`` suffix (thereby
 blocking the use of cached bytecode), and will raise a custom audit
-event ``spython.open_for_exec`` containing ``(filename,
+event ``spython.open_for_import`` containing ``(filename,
 True_if_allowed)``.
 
 On Windows, the hook will also open the file with flags that prevent any
@@ -622,7 +617,7 @@ for every file in a Python deployment, ideally verified using a private
 certificate. For example, Windows supports embedding signatures in
 executable files and using catalogs for others, and can use DeviceGuard
 [4]_ to validate signatures either automatically or using an
-``open_for_exec`` hook.
+``open_for_import`` hook.
 
 Sysadmins **should** log as many audited events as possible, and
 **should** copy logs off of local machines frequently. Even if logs are
@@ -646,7 +641,7 @@ than to prevent them.
 Sysadmins **should** identify correlations between events, as a change
 to correlated events may indicate misuse. For example, module imports
 will typically trigger the ``import`` auditing event, followed by an
-``open_for_exec`` call and usually a ``compile`` event. Attempts to
+``open_for_import`` call and usually a ``compile`` event. Attempts to
 bypass auditing will often suppress some but not all of these events. So
 if the log contains ``import`` events but not ``compile`` events,
 investigation may be necessary.
@@ -660,14 +655,14 @@ To prevent audit hooks being added on non-production machines, an entry
 point **may** add an audit hook that aborts the ``sys.addloghook`` event
 but otherwise does nothing.
 
-On production machines, a non-validating ``open_for_exec`` hook **may**
+On production machines, a non-validating ``open_for_import`` hook
-be set in C code before ``Py_Initialize`` is called. This prevents later
+**may** be set in C code before ``Py_Initialize`` is called. This
-code from overriding the hook, however, logging the
+prevents later code from overriding the hook, however, logging the
 ``setopenforexecutehandler`` event is useful since no code should ever
-need to call it. Using at least the sample ``open_for_exec`` hook
+need to call it. Using at least the sample ``open_for_import`` hook
 implementation from ``spython`` is recommended.
 
-Since ``importlib``'s use of ``open_for_exec`` may be easily bypassed
+Since ``importlib``'s use of ``open_for_import`` may be easily bypassed
 with monkeypatching, an audit hook **should** be used to detect
 attribute changes on type objects.
 
@@ -719,8 +714,8 @@ section is the first time the word "secure" has been used. Security
 **transparency** does not result in any changed behaviour, so there is
 no appropriate reason for applications to modify their behaviour.
 
-Both application-level APIs ``sys.audit`` and ``os.open_for_exec`` are
+Both application-level APIs ``sys.audit`` and ``_imp.open_for_import``
-always present and functional, regardless of whether the regular
+are always present and functional, regardless of whether the regular
 ``python`` entry point or some alternative entry point is used. Callers
 cannot determine whether any hooks have been added (except by performing
 side-channel analysis), nor do they need to. The calls should be fast