From 4ab33c63f113f0895abd9b2954345c5111adf35d Mon Sep 17 00:00:00 2001 From: Steve Dower Date: Tue, 7 May 2019 15:36:45 -0400 Subject: [PATCH] PEP 578: Update with feedback (#1023) * Note about naming style * Add note about using package name * Avoid embedding null characters * Add socket event argument to socket events --- pep-0578.rst | 77 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 30 deletions(-) diff --git a/pep-0578.rst b/pep-0578.rst index 0fa7b5e1c..f6fcf5443 100644 --- a/pep-0578.rst +++ b/pep-0578.rst @@ -118,9 +118,6 @@ send and receive audit hook messages:: # Raise an event with all auditing hooks int PySys_Audit(const char *event, PyObject *args); - # Internal API used during Py_Finalize() - not publicly accessible - void _Py_ClearAuditHooks(void); - The new Python APIs for receiving and raising audit hooks are:: # Add an auditing hook @@ -133,7 +130,9 @@ The new Python APIs for receiving and raising audit hooks are:: Hooks are added by calling ``PySys_AddAuditHook()`` from C at any time, including before ``Py_Initialize()``, or by calling ``sys.addaudithook()`` from Python code. Hooks cannot be removed or -replaced. +replaced. For CPython, hooks added from C are global, while hooks added +from Python are only for the current interpreter. Global hooks are +executed before interpreter hooks. When events of interest are occurring, code can either call ``PySys_Audit()`` from C (while the GIL is held) or ``sys.audit()``. The @@ -141,41 +140,54 @@ string argument is the name of the event, and the tuple contains arguments. A given event name should have a fixed schema for arguments, which should be considered a public API (for each x.y version release), and thus should only change between feature releases with updated -documentation. +documentation. To minimize overhead and simplify handling in native code +hook implementations, named arguments are not supported. For maximum compatibility, events using the same name as an event in the reference interpreter CPython should make every attempt to use compatible arguments. Including the name or an abbreviation of the implementation in implementation-specific event names will also help prevent collisions. For example, a ``pypy.jit_invoked`` event is clearly -distinguised from an ``ipy.jit_invoked`` event. +distinguished from an ``ipy.jit_invoked`` event. Events raised from +Python modules should include their module or package name in the event +name. + +While event names may be arbitrary UTF-8 strings, for consistency across +implementations it is recommended to use valid Python dotted names and +avoid encoding specific details in the name. For example, an ``import`` +event with the module name ``spam`` as an argument is preferable to a +``spam module imported`` event with no arguments. Avoid using embedded +null characters or you may upset those who implement hooks using C. When an event is audited, each hook is called in the order it was added -with the event name and tuple. If any hook returns with an exception -set, later hooks are ignored and *in general* the Python runtime should -terminate. This is intentional to allow hook implementations to decide -how to respond to any particular event. The typical responses will be to -log the event, abort the operation with an exception, or to immediately -terminate the process with an operating system exit call. +(as much as is possible), passing the event name and arguments. If any +hook returns with an exception set, later hooks are ignored and *in +general* the Python runtime should terminate - exceptions from hooks are +not intended to be handled or treated as expected occurrences. This +allows hook implementations to decide how to respond to any particular +event. The typical responses will be to log the event, abort the +operation with an exception, or to immediately terminate the process with +an operating system exit call. When an event is audited but no hooks have been set, the ``audit()`` function should impose minimal overhead. Ideally, each argument is a -reference to existing data rather than a value calculated just for the +reference to existing data rather than a value calculated just for the auditing call. As hooks may be Python objects, they need to be freed during -``Py_Finalize()``. To do this, we add an internal API -``_Py_ClearAuditHooks()`` that releases any Python hooks and any -memory held. This is an internal function with no public export, and -we recommend it raise its own audit event for all current hooks to -ensure that unexpected calls are observed. +interpreter or runtime finalization. These should not be triggered at +any other time, and should raise an event hook to ensure that any +unexpected calls are observed. Below in `Suggested Audit Hook Locations`_, we recommend some important -operations that should raise audit events. +operations that should raise audit events. In general, events should be +raised at the lowest possible level. Given the choice between raising an +event from Python code or native code, raising from native code should be +preferred. Python implementations should document which operations will raise audit events, along with the event schema. It is intentional that -``sys.addaudithook(print)`` be a trivial way to display all messages. +``sys.addaudithook(print)`` is a trivial way to display all messages. Verified Open Hook ------------------ @@ -186,8 +198,13 @@ execute bit in the permissions field, a verified hash of the file contents to detect potential code tampering, or file system path restrictions. These are an important security mechanism for ensuring that only code that has been approved for a given environment is -executed. Currently, Python has no way to integrate with operating -system support when launching scripts or importing modules. +executed. + +Most kernels offer ways to restrict or audit binaries loaded and executed +by the kernel. File types owned by Python appear as regular data and +these features do not apply. This open hook allows Python embedders to +integrate with operating system support when launching scripts or +importing Python code. The new public C API for the verified open hook is:: @@ -309,8 +326,8 @@ see which operations provide audit events. ``PySys_AddAuditHook``, ``sys.addaudithook``, "", "Detect when new audit hooks are being added. " - ``PyFile_SetOpenCodeHook``, ``setopencodehook``, "", " - Detects any attempt to set the ``open_code`` hook. + ``PyFile_SetOpenCodeHook``, ``cpython.PyFile_SetOpenCodeHook``, " + ", "Detects any attempt to set the ``open_code`` hook. " "``compile``, ``exec``, ``eval``, ``PyAst_CompileString``, ``PyAST_obj2mod``", ``compile``, "``(code, filename_or_none)``", " @@ -328,9 +345,9 @@ see which operations provide audit events. file. All arguments other than the module name may be ``None`` if they are not used or available. " - "``open``", ``open``, "``(path, mode, flags)``", "Detect when a file - is about to be opened. *path* and *mode* are the usual parameters to - ``open`` if available, while *flags* is provided instead of *mode* + "``open``", ``io.open``, "``(path, mode, flags)``", "Detect when a + file is about to be opened. *path* and *mode* are the usual parameters + to ``open`` if available, while *flags* is provided instead of *mode* in some cases. " ``PyEval_SetProfile``, ``sys.setprofile``, "", "Detect when code is @@ -400,9 +417,9 @@ see which operations provide audit events. " "``socket.bind``, ``socket.connect``, ``socket.connect_ex``, ``socket.getaddrinfo``, ``socket.getnameinfo``, ``socket.sendmsg``, - ``socket.sendto``", ``socket.address``, "``(address,)``", "Detect - access to network resources. The address is unmodified from the - original call. + ``socket.sendto``", ``socket.address``, "``(socket, address,)``", " + Detect access to network resources. The address is unmodified from + the original call. " "``member_get``, ``func_get_code``, ``func_get_[kw]defaults`` ",``object.__getattr__``,"``(object, attr)``","Detect access to