diff --git a/pep-0573.rst b/pep-0573.rst index 709019cce..4af053191 100644 --- a/pep-0573.rst +++ b/pep-0573.rst @@ -46,8 +46,6 @@ Process-Global State C-level static variables. Since this is very low-level memory storage, it must be managed carefully. -(XXX Use this term everywhere) - Per-module State ---------------- @@ -129,7 +127,7 @@ This allows "smuggling" values to other subinterpreters via attributes of Moreover, since raising exceptions is a common operation, and heap types will be "viral", ``PyErr_NewException`` will tend to "infect" the module -with "heap-type-ness" – at least if the module decides play well with +with "heap type-ness" – at least if the module decides play well with subinterpreters/isolation. Many modules could go without module state entirely if the exception classes were immutable. @@ -154,24 +152,26 @@ In Python code, the Python-level equivalents may be retrieved as:: import sys - class Foo: def meth(self): instance = self module_globals = globals() - module_object = sys.modules[__name__] - underlying_function = Foo.meth - defining_class = Foo + module_object = sys.modules[__name__] # (1) + underlying_function = Foo.meth # (1) + defining_class = Foo # (1) + defining_class = __class__ # (2) .. note:: The defining class is not ``type(self)``, since ``type(self)`` might be a subclass of ``Foo``. -Implicitly, the last three of those rely on name-based lookup via the function's ``__globals__``: +The statements marked (1) implicitly rely on name-based lookup via the function's ``__globals__``: either the ``Foo`` attribute to access the defining class and Python function object, or ``__name__`` to find the module object in ``sys.modules``. In Python code, this is feasible, as ``__globals__`` is set appropriately when the function definition is executed, and even if the namespace has been manipulated to return a different object, at worst an exception will be raised. +The ``__class__`` closure, (2), is a safer way to get the defining class, but it still relies on ``__closure__`` being set appropriately. + By contrast, extension methods are typically implemented as normal C functions. This means that they only have access to their arguments and C level thread-local and process-global states. Traditionally, many extension modules have stored @@ -239,7 +239,11 @@ Two possible solutions have been proposed to this problem: ``__typeslots__`` [#typeslots-mail]_. This is technically feasible and fast, but quite invasive. -Due to the invasiveness of the latter approach, this PEP proposes adding a MRO walking helper for use in slot method implementations, deferring the more complex alternative as a potential future optimisation. +Due to the invasiveness of the latter approach, this PEP proposes adding an MRO walking +helper for use in slot method implementations, deferring the more complex alternative +as a potential future optimisation. Modules affected by this concern also have the +option of using thread-local state or PEP 567 context variables, or else defining their +own reload-friendly lookup caching scheme. Immutable Exception Types @@ -253,8 +257,6 @@ This pointer may be declared in process-global state. The function will then allocate the object and will keep in mind that already existing exception should not be overwritten. -(XXX: Update function name in the implementation) - The extra indirection makes it possible to make ``PyErr_PrepareImmutableException`` part of the stable ABI by having the Python interpreter, rather than extension code, allocate the ``PyTypeObject``. @@ -314,6 +316,13 @@ will be added:: PyTypeObject *mm_class; /* Passed as 'defining_class' arg to the C func */ } PyCMethodObject; +To allow passing the defining class to the underlying C function, a change +to private API is required, now ``_PyMethodDef_RawFastCallDict`` and +``_PyMethodDef_RawFastCallKeywords`` will receive ``PyTypeObject *cls`` +as one of their arguments. + +A new macro ``PyCFunction_GET_CLASS(cls)`` will be added for easier access to mm_class. + Method construction and calling code and will be updated to honor ``METH_METHOD``. @@ -385,30 +394,51 @@ and return -1. If called with an initialized exception type (``*exc`` is non-NULL), the function will do nothing but incref ``*exc``. -(XXX: Support and test multiple inheritance) - A new flag, ``Py_TPFLAGS_HEAP_IMMUTABLE``, will be added to prevent mutation of the type object. This makes it possible to share the object safely between multiple interpreters. -This flag is checked in setattr (XXX: name of C function?) and blocks +This flag is checked in ``type_setattro`` and blocks setting of attributes when set, similar to built-in types. A new pointer, ``ht_moduleptr``, will be added to heap types to store ``exc``. -(XXX: better name? Update implementation) - On deinitialization of the exception type, ``*exc`` will be set to ``NULL``. -This makes it safe for -``PyErr_PrepareImmutableException`` to check if the exception -was already initialized. +This makes it safe for ``PyErr_PrepareImmutableException`` to check if +the exception was already initialized. + +PyType_offsets +-------------- + +Some extension types are using instances with ``__dict__`` or ``__weakref__`` +allocated. Currently, there is no way of passing offsets of these through +``PyType_Spec``. To allow this, a new structure and a spec slot are proposed. + +A new structure, ``PyType_offsets``, will have two members containing the +offsets of ``__dict__`` and ``__weakref__``:: + + typedef struct { + Py_ssize_t dict; + Py_ssize_t weaklist; + } PyType_offsets; + +The new slot, ``Py_offsets``, will be used to pass a ``PyType_offsets *`` +structure containing the mentioned data. Helpers ------- -XXX: I'd like to port a bunch of modules to see what helpers would be convenient +Getting to per-module state from a heap type is a very common task. To make this +easier, a helper will be added:: -XXX: Helper for getting module state directly from defining class + void *PyType_GetModuleState(PyObject *type) + +This function takes a heap type and on success, it returns pointer to state of the +module that the heap type belongs to. + +On failure, two scenarios may occure. When a type without a module is passed in, +``SystemError`` is set and ``NULL`` returned. If the module is found, pointer +to the state, which may be ``NULL``, is returned without setting any exception. Modules Converted in the Initial Implementation @@ -424,7 +454,40 @@ will be ported to PEP 489 multiphase initialization. Summary of API Changes and Additions ==================================== -XXX, see above for now +New functions: + +* PyType_GetModule +* PyType_DefiningTypeFromSlotFunc +* PyType_GetModuleState +* PyErr_PrepareImmutableException + +New macros: + +* PyCFunction_GET_CLASS + +New types: + +* PyCMethodObject + +New structures: + +* PyType_offsets + +Modified functions: + +* _PyMethodDef_RawFastCallDict now receives ``PyTypeObject *cls``. +* _PyMethodDef_RawFastCallKeywords now receives ``PyTypeObject *cls``. + +Modified structures: + +* _heaptypeobject - added ht_module and ht_moduleptr + +Other changes: + +* METH_METHOD call flag +* defining_class converter in clinic +* Py_TPFLAGS_HEAP_IMMUTABLE flag +* Py_offsets type spec slot Backwards Compatibility @@ -434,7 +497,7 @@ Two new pointers are added to all heap types. All other changes are adding new functions, structures and a type flag. The new ``PyErr_PrepareImmutableException`` function changes encourages -modules to switch from using heap-type Exception classes to static ones, +modules to switch from using heap type Exception classes to immutable ones, and a number of modules will be switched in the initial implementation. This change will prevent adding class attributes to such types. For example, the following will raise AttributeError:: @@ -471,6 +534,9 @@ such as not allowing keyword arguments. As proposed here, methods defined with the ``METH_METHOD`` flag do not support these optimizations. +Optimized calls still have the option of accessing per-module state +the same way slot methods do. + References ==========