PEP 573 updates (#618)

pep-0573.rst

@@ -46,8 +46,6 @@ Process-Global State
 C-level static variables. Since this is very low-level
 memory storage, it must be managed carefully.
 
-(XXX Use this term everywhere)
-
 Per-module State
 ----------------
 
@@ -129,7 +127,7 @@ This allows "smuggling" values to other subinterpreters via attributes of
 
 Moreover, since raising exceptions is a common operation, and heap types
 will be "viral", ``PyErr_NewException`` will tend to "infect" the module
-with "heap-type-ness" – at least if the module decides play well with
+with "heap type-ness" – at least if the module decides play well with
 subinterpreters/isolation.
 Many modules could go without module state
 entirely if the exception classes were immutable.
@@ -154,24 +152,26 @@ In Python code, the Python-level equivalents may be retrieved as::
 
     import sys
 
-    class Foo:
         def meth(self):
             instance = self
             module_globals = globals()
-            module_object = sys.modules[__name__]
+            module_object = sys.modules[__name__]  # (1)
-            underlying_function = Foo.meth
+            underlying_function = Foo.meth         # (1)
-            defining_class = Foo
+            defining_class = Foo                   # (1)
+            defining_class = __class__             # (2)
 
 .. note::
 
     The defining class is not ``type(self)``, since ``type(self)`` might
     be a subclass of ``Foo``.
 
-Implicitly, the last three of those rely on name-based lookup via the function's ``__globals__``:
+The statements marked (1) implicitly rely on name-based lookup via the function's ``__globals__``:
 either the ``Foo`` attribute to access the defining class and Python function object, or ``__name__`` to find the module object in ``sys.modules``.
 In Python code, this is feasible, as ``__globals__`` is set appropriately when the function definition is executed, and
 even if the namespace has been manipulated to return a different object, at worst an exception will be raised.
 
+The ``__class__`` closure, (2), is a safer way to get the defining class, but it still relies on ``__closure__`` being set appropriately.
+
 By contrast, extension methods are typically implemented as normal C functions.
 This means that they only have access to their arguments and C level thread-local
 and process-global states. Traditionally, many extension modules have stored
@@ -239,7 +239,11 @@ Two possible solutions have been proposed to this problem:
       ``__typeslots__`` [#typeslots-mail]_.  This is technically feasible and fast,
       but quite invasive.
 
-Due to the invasiveness of the latter approach, this PEP proposes adding a MRO walking helper for use in slot method implementations, deferring the more complex alternative as a potential future optimisation.
+Due to the invasiveness of the latter approach, this PEP proposes adding an MRO walking
+helper for use in slot method implementations, deferring the more complex alternative
+as a potential future optimisation. Modules affected by this concern also have the
+option of using thread-local state or PEP 567 context variables, or else defining their
+own reload-friendly lookup caching scheme.
 
 
 Immutable Exception Types
@@ -253,8 +257,6 @@ This pointer may be declared in process-global state. The function will then
 allocate the object and will keep in mind that already existing exception
 should not be overwritten.
 
-(XXX: Update function name in the implementation)
-
 The extra indirection makes it possible to make ``PyErr_PrepareImmutableException``
 part of the stable ABI by having the Python interpreter, rather than extension code,
 allocate the ``PyTypeObject``.
@@ -314,6 +316,13 @@ will be added::
         PyTypeObject *mm_class; /* Passed as 'defining_class' arg to the C func */
     } PyCMethodObject;
 
+To allow passing the defining class to the underlying C function, a change
+to private API is required, now ``_PyMethodDef_RawFastCallDict`` and
+``_PyMethodDef_RawFastCallKeywords`` will receive ``PyTypeObject *cls``
+as one of their arguments.
+
+A new macro ``PyCFunction_GET_CLASS(cls)`` will be added for easier access to mm_class. 
+
 Method construction and calling code and will be updated to honor
 ``METH_METHOD``.
 
@@ -385,30 +394,51 @@ and return -1.
 If called with an initialized exception type (``*exc``
 is non-NULL), the function will do nothing but incref ``*exc``.
 
-(XXX: Support and test multiple inheritance)
-
 A new flag, ``Py_TPFLAGS_HEAP_IMMUTABLE``, will be added to prevent
 mutation of the type object. This makes it possible to
 share the object safely between multiple interpreters.
-This flag is checked in setattr (XXX: name of C function?) and blocks
+This flag is checked in ``type_setattro`` and blocks
 setting of attributes when set, similar to built-in types.
 
 A new pointer, ``ht_moduleptr``, will be added to heap types to store ``exc``.
 
-(XXX: better name? Update implementation)
-
 On deinitialization of the exception type, ``*exc`` will be set to ``NULL``.
-This makes it safe for
+This makes it safe for ``PyErr_PrepareImmutableException`` to check if
-``PyErr_PrepareImmutableException`` to check if the exception
+the exception was already initialized.
-was already initialized.
+
+PyType_offsets
+--------------
+
+Some extension types are using instances with ``__dict__`` or ``__weakref__``
+allocated. Currently, there is no way of passing offsets of these through
+``PyType_Spec``. To allow this, a new structure and a spec slot are proposed.
+
+A new structure, ``PyType_offsets``, will have two members containing the
+offsets of ``__dict__`` and ``__weakref__``::
+
+    typedef struct {
+        Py_ssize_t dict;
+        Py_ssize_t weaklist;
+    } PyType_offsets;
+
+The new slot, ``Py_offsets``, will be used to pass a ``PyType_offsets *``
+structure containing the mentioned data.
 
 
 Helpers
 -------
 
-XXX: I'd like to port a bunch of modules to see what helpers would be convenient
+Getting to per-module state from a heap type is a very common task. To make this
+easier, a helper will be added::
 
-XXX: Helper for getting module state directly from defining class
+    void *PyType_GetModuleState(PyObject *type)
+
+This function takes a heap type and on success, it returns pointer to state of the
+module that the heap type belongs to.
+
+On failure, two scenarios may occure. When a type without a module is passed in,
+``SystemError`` is set and ``NULL`` returned. If the module is found, pointer
+to the state, which may be ``NULL``, is returned without setting any exception.
 
 
 Modules Converted in the Initial Implementation
@@ -424,7 +454,40 @@ will be ported to PEP 489 multiphase initialization.
 Summary of API Changes and Additions
 ====================================
 
-XXX, see above for now
+New functions:
+
+* PyType_GetModule
+* PyType_DefiningTypeFromSlotFunc
+* PyType_GetModuleState
+* PyErr_PrepareImmutableException
+
+New macros:
+
+* PyCFunction_GET_CLASS
+
+New types:
+
+* PyCMethodObject 
+
+New structures:
+
+* PyType_offsets
+
+Modified functions:
+
+* _PyMethodDef_RawFastCallDict now receives ``PyTypeObject *cls``.
+* _PyMethodDef_RawFastCallKeywords now receives ``PyTypeObject *cls``.
+
+Modified structures:
+
+* _heaptypeobject - added ht_module and ht_moduleptr
+
+Other changes:
+
+* METH_METHOD call flag
+* defining_class converter in clinic
+* Py_TPFLAGS_HEAP_IMMUTABLE flag
+* Py_offsets type spec slot
 
 
 Backwards Compatibility
@@ -434,7 +497,7 @@ Two new pointers are added to all heap types.
 All other changes are adding new functions, structures and a type flag.
 
 The new ``PyErr_PrepareImmutableException`` function changes encourages
-modules to switch from using heap-type Exception classes to static ones,
+modules to switch from using heap type Exception classes to immutable ones,
 and a number of modules will be switched in the initial implementation.
 This change will prevent adding class attributes to such types.
 For example, the following will raise AttributeError::
@@ -471,6 +534,9 @@ such as not allowing keyword arguments.
 As proposed here, methods defined with the ``METH_METHOD`` flag do not support
 these optimizations.
 
+Optimized calls still have the option of accessing per-module state
+the same way slot methods do.
+
 
 References
 ==========