PEP 620

Elaborate rationale for making structures opaque.

pep-0620.rst

@@ -51,12 +51,25 @@ released in 2009, has been removed in the Python 3.8 development cycle.
 But the change broke too many C extensions and had to be reverted before
 3.8 final release. Finally, the member was removed again in Python 3.9.
 
-Modifying other structures, like ``PyListObject``, cannot be even
-considered right now, since it would break way too many C extensions.
+C extensions rely on the ability to access directly structures members,
+indirectly through the C API, or even directly. Modifying structures
+like ``PyListObject`` cannot be even considered.
 
 The ``PyTypeObject`` structure is the one which evolved the most, simply
 because there was no other way to evolve CPython than modifying it.
 
+In the C API, all Python objects are passed as ``PyObject*``: a pointer
+to a ``PyObject`` structure. Experimenting tagged pointers in CPython is
+blocked by the fact that a C extension can technically dereference a
+``PyObject*`` pointer and access ``PyObject`` members. Small "objects"
+can be stored as a tagged pointer with no concrete ``PyObject``
+structure.
+
+Replacing Python garbage collector with a tracing garbage collector
+would also need to remove ``PyObject.ob_refcnt`` reference counter,
+whereas currently ``Py_INCREF()`` and ``Py_DECREF()`` macros access
+directly to ``PyObject.ob_refcnt``.
+
 Same CPython design since 1990: structures and reference counting
 -----------------------------------------------------------------
 
@@ -78,8 +91,8 @@ changed since the "Initial revision" commit (1990)::
     } object;
 
     typedef struct {
-           OB_VARHEAD
-           object *ob_item[1];
+        OB_VARHEAD
+        object *ob_item[1];
     } tupleobject;
 
 Only names changed: ``object`` was renamed to ``PyObject`` and
@@ -98,8 +111,8 @@ CPython in average. PyPy developers chose to not fork CPython, but start
 from scratch to have more freedom in terms of optimization choices.
 
 PyPy does not use reference counting, but a tracing garbage collector
-which moves objects. Objects can be allocated on the stack, rather than
-always having to be allocated on the heap.
+which moves objects. Objects can be allocated on the stack (or even not
+at all), rather than always having to be allocated on the heap.
 
 Objects layouts are designed with performance in mind. For example, a
 list strategy stores integers directly as integers, rather than objects.
@@ -139,7 +152,9 @@ Hide implementation details
 Hiding implementation details from the C API has multiple advantages:
 
 * It becomes possible to experiment in CPython more advanced
-  optimizations than just micro-optimizations.
+  optimizations than just micro-optimizations. For example, tagged
+  pointers, and replace the garbage collector with a tracing garbage
+  collector which can move objects.
 * Adding new features in CPython becomes easier.
 * PyPy should be able to avoid conversions to CPython objects in more
   cases: keep efficient PyPy objects.
@@ -293,7 +308,7 @@ cast their parameters to ``PyObject*``.
 Python 3.6 requires C compilers to support static inline functions: the
 PEP 7 requires a subset of C99.
 
-**STATUS**: Completed (in Python 3.8)
+**STATUS**: Completed (in Python 3.9)
 
 Macros converted to static inline functions in Python 3.8: