iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 551 and PEP 7 (#392)

This commit is contained in:
Steve Dower 2017-09-06 11:27:23 -07:00 committed by GitHub
parent ebbb636b7b
commit f41260ee2d
1 changed files with 96 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

143
pep-0551.rst
View File

@ -269,7 +269,10 @@ The new public C API for the verified open hook is::
# Set the handler # Set the handler
typedef PyObject *(*handler_func)(const char *narrow, typedef PyObject *(*handler_func)(const char *narrow,
const wchar_t *wide) const wchar_t *wide)
int Py_SetOpenForExecuteHandler(handler_func handler) int PyOS_SetOpenForExecuteHandler(handler_func handler)
# Open a file using the handler
PyObject *PyOS_OpenForExec(PyObject *path)
The new public Python API for the verified open hook is:: The new public Python API for the verified open hook is::
@ -356,101 +359,144 @@ should be considered part of the rationale for including the hook.
:widths: 2, 2, 3, 6 :widths: 2, 2, 3, 6
``PySys_AddAuditHook``, ``sys.addaudithook``, "", "Detect when new ``PySys_AddAuditHook``, ``sys.addaudithook``, "", "Detect when new
audit hooks are being added." audit hooks are being added.
"
``_PySys_ClearAuditHooks``, ``sys._clearaudithooks``, "", "Notifies ``_PySys_ClearAuditHooks``, ``sys._clearaudithooks``, "", "Notifies
hooks they are being cleaned up, mainly in case the event is hooks they are being cleaned up, mainly in case the event is
triggered unexpectedly. This event cannot be aborted." triggered unexpectedly. This event cannot be aborted.
"
``Py_SetOpenForExecuteHandler``, ``setopenforexecutehandler``, "", " ``Py_SetOpenForExecuteHandler``, ``setopenforexecutehandler``, "", "
Detects any attempt to set the ``open_for_execute`` handler." Detects any attempt to set the ``open_for_execute`` handler.
"
"``compile``, ``exec``, ``eval``, ``PyAst_CompileString``, "``compile``, ``exec``, ``eval``, ``PyAst_CompileString``,
``PyAST_obj2mod``", ``compile``, "``(code, filename_or_none)``", " ``PyAST_obj2mod``", ``compile``, "``(code, filename_or_none)``", "
Detect dynamic code compilation, where ``code`` could be a string or Detect dynamic code compilation, where ``code`` could be a string or
AST. Note that this will be called for regular imports of source AST. Note that this will be called for regular imports of source
code, including those that were opened with ``open_for_exec``." code, including those that were opened with ``open_for_exec``.
"
"``exec``, ``eval``, ``run_mod``", ``exec``, "``(code_object,)``", " "``exec``, ``eval``, ``run_mod``", ``exec``, "``(code_object,)``", "
Detect dynamic execution of code objects. This only occurs for Detect dynamic execution of code objects. This only occurs for
explicit calls, and is not raised for normal function invocation." explicit calls, and is not raised for normal function invocation.
"
``import``, ``import``, "``(module, filename, sys.path, ``import``, ``import``, "``(module, filename, sys.path,
sys.meta_path, sys.path_hooks)``", "Detect when modules are sys.meta_path, sys.path_hooks)``", "Detect when modules are
imported. This is raised before the module name is resolved to a imported. This is raised before the module name is resolved to a
file. All arguments other than the module name may be ``None`` if file. All arguments other than the module name may be ``None`` if
they are not used or available." they are not used or available.
"
``code_new``, ``code.__new__``, "``(bytecode, filename, name)``", " ``code_new``, ``code.__new__``, "``(bytecode, filename, name)``", "
Detect dynamic creation of code objects. This only occurs for Detect dynamic creation of code objects. This only occurs for
direct instantiation, and is not raised for normal compilation." direct instantiation, and is not raised for normal compilation.
"
``func_new_impl``, ``function.__new__``, "``(code,)``", "Detect
dynamic creation of function objects. This only occurs for direct
instantiation, and is not raised for normal compilation.
"
"``_ctypes.dlopen``, ``_ctypes.LoadLibrary``", ``ctypes.dlopen``, " "``_ctypes.dlopen``, ``_ctypes.LoadLibrary``", ``ctypes.dlopen``, "
``(module_or_path,)``", "Detect when native modules are used." ``(module_or_path,)``", "Detect when native modules are used.
"
``_ctypes._FuncPtr``, ``ctypes.dlsym``, "``(lib_object, name)``", " ``_ctypes._FuncPtr``, ``ctypes.dlsym``, "``(lib_object, name)``", "
Collect information about specific symbols retrieved from native Collect information about specific symbols retrieved from native
modules." modules.
"
``_ctypes._CData``, ``ctypes.cdata``, "``(ptr_as_int,)``", "Detect ``_ctypes._CData``, ``ctypes.cdata``, "``(ptr_as_int,)``", "Detect
when code is accessing arbitrary memory using ``ctypes``" when code is accessing arbitrary memory using ``ctypes``.
"
``id``, ``id``, "``(id_as_int,)``", "Detect when code is accessing ``id``, ``id``, "``(id_as_int,)``", "Detect when code is accessing
the id of objects, which in CPython reveals information about the id of objects, which in CPython reveals information about
memory layout." memory layout.
"
``sys._getframe``, ``sys._getframe``, "``(frame_object,)``", "Detect ``sys._getframe``, ``sys._getframe``, "``(frame_object,)``", "Detect
when code is accessing frames directly" when code is accessing frames directly.
"
``sys._current_frames``, ``sys._current_frames``, "", "Detect when ``sys._current_frames``, ``sys._current_frames``, "", "Detect when
code is accessing frames directly" code is accessing frames directly.
"
``PyEval_SetProfile``, ``sys.setprofile``, "", "Detect when code is ``PyEval_SetProfile``, ``sys.setprofile``, "", "Detect when code is
injecting trace functions. Because of the implementation, exceptions injecting trace functions. Because of the implementation, exceptions
raised from the hook will abort the operation, but will not be raised from the hook will abort the operation, but will not be
raised in Python code. Note that ``threading.setprofile`` eventually raised in Python code. Note that ``threading.setprofile`` eventually
calls this function, so the event will be audited for each thread." calls this function, so the event will be audited for each thread.
"
``PyEval_SetTrace``, ``sys.settrace``, "", "Detect when code is ``PyEval_SetTrace``, ``sys.settrace``, "", "Detect when code is
injecting trace functions. Because of the implementation, exceptions injecting trace functions. Because of the implementation, exceptions
raised from the hook will abort the operation, but will not be raised from the hook will abort the operation, but will not be
raised in Python code. Note that ``threading.settrace`` eventually raised in Python code. Note that ``threading.settrace`` eventually
calls this function, so the event will be audited for each thread." calls this function, so the event will be audited for each thread.
"
``_PyEval_SetAsyncGenFirstiter``, ``sys.set_async_gen_firstiter``, " ``_PyEval_SetAsyncGenFirstiter``, ``sys.set_async_gen_firstiter``, "
", "Detect changes to async generator hooks." ", "Detect changes to async generator hooks.
"
``_PyEval_SetAsyncGenFinalizer``, ``sys.set_async_gen_finalizer``, " ``_PyEval_SetAsyncGenFinalizer``, ``sys.set_async_gen_finalizer``, "
", "Detect changes to async generator hooks." ", "Detect changes to async generator hooks.
"
``_PyEval_SetCoroutineWrapper``, ``sys.set_coroutine_wrapper``, " ``_PyEval_SetCoroutineWrapper``, ``sys.set_coroutine_wrapper``, "
", "Detect changes to the coroutine wrapper." ", "Detect changes to the coroutine wrapper.
"
``Py_SetRecursionLimit``, ``sys.setrecursionlimit``, " ``Py_SetRecursionLimit``, ``sys.setrecursionlimit``, "
``(new_limit,)``", "Detect changes to the recursion limit." ``(new_limit,)``", "Detect changes to the recursion limit.
"
``_PyEval_SetSwitchInterval``, ``sys.setswitchinterval``, " ``_PyEval_SetSwitchInterval``, ``sys.setswitchinterval``, "
``(interval_us,)``", "Detect changes to the switching interval." ``(interval_us,)``", "Detect changes to the switching interval.
"
"``socket.bind``, ``socket.connect``, ``socket.connect_ex``, "``socket.bind``, ``socket.connect``, ``socket.connect_ex``,
``socket.getaddrinfo``, ``socket.getnameinfo``, ``socket.sendmsg``, ``socket.getaddrinfo``, ``socket.getnameinfo``, ``socket.sendmsg``,
``socket.sendto``", ``socket.address``, "``(address,)``", "Detect ``socket.sendto``", ``socket.address``, "``(address,)``", "Detect
access to network resources. The address is unmodified from the access to network resources. The address is unmodified from the
original call." original call.
"
``socket.__init__``, "socket()", "``(family, type, proto)``", " ``socket.__init__``, "socket()", "``(family, type, proto)``", "
Detect creation of sockets. The arguments will be int values." Detect creation of sockets. The arguments will be int values.
"
``socket.gethostname``, ``socket.gethostname``, "", "Detect attempts ``socket.gethostname``, ``socket.gethostname``, "", "Detect attempts
to retrieve the current host name." to retrieve the current host name.
"
``socket.sethostname``, ``socket.sethostname``, "``(name,)``", " ``socket.sethostname``, ``socket.sethostname``, "``(name,)``", "
Detect attempts to change the current host name. The name argument Detect attempts to change the current host name. The name argument
is passed as a bytes object." is passed as a bytes object.
"
"``socket.gethostbyname``, ``socket.gethostbyname_ex``", "``socket.gethostbyname``, ``socket.gethostbyname_ex``",
"``socket.gethostbyname``", "``(name,)``", "Detect host name "``socket.gethostbyname``", "``(name,)``", "Detect host name
resolution. The name argument is a str or bytes object." resolution. The name argument is a str or bytes object.
"
``socket.gethostbyaddr``, ``socket.gethostbyaddr``, " ``socket.gethostbyaddr``, ``socket.gethostbyaddr``, "
``(address,)``", "Detect host resolution. The address argument is a ``(address,)``", "Detect host resolution. The address argument is a
str or bytes object." str or bytes object.
"
``socket.getservbyname``, ``socket.getservbyname``, "``(name, ``socket.getservbyname``, ``socket.getservbyname``, "``(name,
protocol)``", "Detect service resolution. The arguments are str protocol)``", "Detect service resolution. The arguments are str
objects." objects.
``socket.getservbyport``, ``socket.getservbyport``, "``(port, "
"``socket.getservbyport``", ``socket.getservbyport``, "``(port,
protocol)``", "Detect service resolution. The port argument is an protocol)``", "Detect service resolution. The port argument is an
int and protocol is a str." int and protocol is a str.
"
"``member_get``, ``func_get_code``, ``func_get_[kw]defaults``
",``object.__getattr__``,"``(object, attr)``","Detect access to
restricted attributes. This event is raised for any built-in
members that are marked as restricted, and members that may allow
bypassing imports.
"
"``_PyObject_GenericSetAttr``, ``check_set_special_type_attr``, "``_PyObject_GenericSetAttr``, ``check_set_special_type_attr``,
``object_set_class``",``object.__setattr__``,"``(object, attr, ``object_set_class``, ``func_set_code``, ``func_set_[kw]defaults``","
value)``","Detect monkey patching of types and objects. This event ``object.__setattr__``","``(object, attr, value)``","Detect monkey
patching of types and objects. This event
is raised for the ``__class__`` attribute and any attribute on is raised for the ``__class__`` attribute and any attribute on
``type`` objects." ``type`` objects.
``_PyObject_GenericSetAttr``,``object.__delattr__``,"``(object, "
"``_PyObject_GenericSetAttr``",``object.__delattr__``,"``(object,
attr)``","Detect deletion of object attributes. This event is raised attr)``","Detect deletion of object attributes. This event is raised
for any attribute on ``type`` objects." for any attribute on ``type`` objects.
``Unpickler.find_class``,``pickle.find_class``,"``(module_name, "
"``Unpickler.find_class``",``pickle.find_class``,"``(module_name,
global_name)``","Detect imports and global name lookup when global_name)``","Detect imports and global name lookup when
unpickling." unpickling.
"
"``array_new``",``array.__new__``,"``(typecode, initial_value)``", "
Detects creation of array objects.
"
TODO - more hooks in ``_socket``, ``_ssl``, others? TODO - more hooks in ``_socket``, ``_ssl``, others?
* code objects
* function objects
SPython Entry Point SPython Entry Point
=================== ===================
@ -523,16 +569,19 @@ events unless explicitly added, and so they will continue to be allowed.
Performance Impact Performance Impact
================== ==================
**TODO** The important performance impact is the case where events are being
raised but there are no hooks attached. This is the unavoidable case -
once a distributor or sysadmin begins adding audit hooks they have
explicitly chosen to trade performance for functionality. Performance
impact using ``spython`` or with hooks added are not of interest here,
since this is considered opt-in functionality.
Full impact analysis still requires investigation. Preliminary testing Analysis using the ``performance`` tool shows no significant impact,
shows that calling ``sys.audit`` with no hooks added does not with the vast majority of benchmarks showing between 1.05x faster to
significantly affect any existing benchmarks, though targeted 1.05x slower.
microbenchmarks can observe an impact.
Performance impact using ``spython`` or with hooks added are not of
interest here, since this is considered opt-in functionality.
In our opinion, the performance impact of the set of auditing points
described in this PEP is negligible.
Recommendations Recommendations
=============== ===============