PEP 520 updates after feedback. (#35)

pep-0520.txt

@@ -8,24 +8,28 @@ Type: Standards Track
 Content-Type: text/x-rst
 Created: 7-Jun-2016
 Python-Version: 3.6
-Post-History: 7-Jun-2016, 11-Jun-2016, 20-Jun-2016
+Post-History: 7-Jun-2016, 11-Jun-2016, 20-Jun-2016, 24-Jun-2016
 
 
 Abstract
 ========
 
-When a class is defined using a ``class`` statement, the class body is
+The class definition syntax is ordered by its very nature. Class
-executed within a namespace.  After the execution completes, that
+attributes defined there are thus ordered.  Aside from helping with
-namespace is copied into new ``dict`` and the original definition
+readability, that ordering is sometimes significant.  If it were
-namespace is discarded.  The new copy is stored away as the class's
+automatically available outside the class definition then the
-namespace and is exposed as ``__dict__`` through a read-only proxy.
+attribute order could be used without the need for extra boilerplate
+(such as metaclasses or manually enumerating the attribute order).
+Given that this information already exists, access to the definition
+order of attributes is a reasonable expectation.  However, currently
+Python does not preserve the attribute order from the class
+definition.
 
-This PEP preserves the order in which the attributes in the definition
+This PEP changes that by preserving the order in which attributes
-namespace were added to it, before that namespace is discarded.  This
+are introduced in the class definition body.  That order will now be
-means it reflects the definition order of the class body.  That order
+preserved in the ``__definition_order__`` attribute of the class.
-will now be preserved in the ``__definition_order__`` attribute of the
+This allows introspection of the original definition order, e.g. by
-class.  This allows introspection of the original definition order,
+class decorators.
-e.g. by class decorators.
 
 Additionally, this PEP changes the default class definition namespace
 to ``OrderedDict``.  The long-lived class namespace (``__dict__``) will
@@ -35,14 +39,51 @@ remain a ``dict``.
 Motivation
 ==========
 
-Currently Python does not preserve the order in which attributes are
+The attribute order from a class definition may be useful to tools
-added to the class definition namespace. The namespace used during
+that rely on name order.  However, without the automatic availability
-execution of a class body defaults to ``dict``.  If the metaclass
+of the definition order, those tools must impose extra requirements on
-defines ``__prepare__()`` then the result of calling it is used.  Thus,
+users.  For example, use of such a tool may require that your class use
-before this PEP, to access your class definition namespace you must
+a particular metaclass.  Such requirements are often enough to
-use ``OrderedDict`` along with a metaclass. Then you must preserve the
+discourage use of the tool.
-definition order (from the ``OrderedDict``) yourself.  This has a
+
-couple of problems.
+Some tools that could make use of this PEP include:
+
+* documentation generators
+* testing frameworks
+* CLI frameworks
+* web frameworks
+* config generators
+* data serializers
+* enum factories (my original motivation)
+
+
+Background
+==========
+
+When a class is defined using a ``class`` statement, the class body
+is executed within a namespace.  Currently that namespace defaults to
+``dict``.  If the metaclass defines ``__prepare__()`` then the result
+of calling it is used for the class definition namespace.
+
+After the execution completes, the definition namespace namespace is
+copied into new ``dict``.  Then the original definition namespace is
+discarded.  The new copy is stored away as the class's namespace and
+is exposed as ``__dict__`` through a read-only proxy.
+
+The class attribute definition order is represented by the insertion
+order of names in the *definition* namespace.  Thus, we can have
+access to the definition order by switching the definition namespace
+to an ordered mapping, such as ``collections.OrderedDict``.  This is
+feasible using a metaclass and ``__prepare__``, as described above.
+In fact, exactly this is by far the most common use case for using
+``__prepare__`` (see PEP 487).
+
+At that point, the only missing thing for later access to the
+definition order is storing it on the class before the definition
+namespace is thrown away.  Again, this may be done using a metaclass.
+However, this means that the definition order is preserved only for
+classes that use such a metaclass.  There are two practical problems
+with that:
 
 First, it requires the use of a metaclass.  Metaclasses introduce an
 extra level of complexity to code and in some cases (e.g. conflicts)
@@ -55,8 +96,6 @@ we have such an opportunity by defaulting to ``OrderedDict``.
 Second, only classes that opt in to using the ``OrderedDict``-based
 metaclass will have access to the definition order. This is problematic
 for cases where universal access to the definition order is important.
-One of the original motivating use cases for this PEP is generic class
-decorators that make use of the definition order.
 
 
 Specification
@@ -64,21 +103,21 @@ Specification
 
 Part 1:
 
-* the order in which class attributes are defined is preserved in the
+* all classes have a ``__definition_order__`` attribute
-  new ``__definition_order__`` attribute on each class
+* ``__definition_order__`` is a ``tuple`` of identifiers (or ``None``)
-* "dunder" attributes (e.g. ``__init__``, ``__module__``) are ignored
-* ``__definition_order__`` is a ``tuple`` (or ``None``)
 * ``__definition_order__`` is a read-only attribute
 * ``__definition_order__`` is always set:
 
-  1. if ``__definition_order__`` is defined in the class body then it
+  1. during execution of the class body, the insertion order of names
+     into the class *definition* namespace is stored in a tuple
+  2. if ``__definition_order__`` is defined in the class body then it
      must be a ``tuple`` of identifiers or ``None``; any other value
      will result in ``TypeError``
-  2. classes that do not have a class definition (e.g. builtins) have
+  3. classes that do not have a class definition (e.g. builtins) have
      their ``__definition_order__`` set to ``None``
-  3. classes for which `__prepare__()`` returned something other than
+  4. classes for which `__prepare__()`` returned something other than
      ``OrderedDict`` (or a subclass) have their ``__definition_order__``
-     set to ``None`` (except where #1 applies)
+     set to ``None`` (except where #2 applies)
 
 Part 2:
 
@@ -95,12 +134,7 @@ default behavior::
    class Spam(metaclass=Meta):
        ham = None
        eggs = 5
-       __definition_order__ = tuple(k for k in locals()
+       __definition_order__ = tuple(locals())
-                                    if not (k.startswith('__') and
-                                            k.endswith('__')))
-
-Note that [pep487_] proposes a similar solution, albeit as part of a
-broader proposal.
 
 Why a tuple?
 ------------
@@ -125,22 +159,36 @@ If a use case for a writable (or mutable) ``__definition_order__``
 arises, the restriction may be loosened later.  Presently this seems
 unlikely and furthermore it is usually best to go immutable-by-default.
 
-Note that ``__definition_order__`` is centered on the class definition
+Note that the ability to set ``__definition_order__`` manually allows
+a dynamically created class (e.g. Cython, ``type()``) to still have
+``__definition_order__`` properly set.
+
+Why not "__attribute_order__"?
+------------------------------
+
+``__definition_order__`` is centered on the class definition
 body.  The use cases for dealing with the class namespace (``__dict__``)
 post-definition are a separate matter.  ``__definition_order__`` would
 be a significantly misleading name for a feature focused on more than
 class definition.
 
-See [nick_concern_] for more discussion.
+Why not ignore "dunder" names?
-
+------------------------------
-Why ignore "dunder" names?
---------------------------
 
 Names starting and ending with "__" are reserved for use by the
 interpreter.  In practice they should not be relevant to the users of
 ``__definition_order__``.  Instead, for nearly everyone they would only
 be clutter, causing the same extra work for everyone.
 
+However, dropping dunder names by default may inadvertantly cause
+problems for classes that use dunder names unconventionally. In this
+case it's better to play it safe and preserve *all* the names from
+the class definition.
+
+Note that a couple of dunder names (``__name__`` and ``__qualname__``)
+are injected by default by the compiler.  So they will be included even
+though they are not strictly part of the class definition body.
+
 Why None instead of an empty tuple?
 -----------------------------------
 
@@ -192,6 +240,12 @@ have a roughly equivalent concept of a definition order. So conceivably
 PEP does not introduce any such support. However, it does not prohibit
 it either.
 
+The specific cases:
+
+* builtin types
+* PyType_Ready
+* PyType_FromSpec
+
 
 Compatibility
 =============
@@ -221,6 +275,22 @@ be minimal.  If a Python implementation cannot support switching to
 to ``None``.
 
 
+Open Questions
+==============
+
+* What about `__slots__`?
+
+* Drop the "read-only attribute" requirement?
+
+Per Guido:
+
+    I don't see why it needs to be a read-only attribute. There are
+    very few of those -- in general we let users play around with
+    things unless we have a hard reason to restrict assignment (e.g.
+    the interpreter's internal state could be compromised). I don't
+    see such a hard reason here.
+
+
 Implementation
 ==============
 
@@ -230,8 +300,8 @@ The implementation is found in the tracker. [impl_]
 Alternatives
 ============
 
-cls.__dict__ as OrderedDict
+An Order-preserving cls.__dict__
--------------------------------
+--------------------------------
 
 Instead of storing the definition order in ``__definition_order__``,
 the now-ordered definition namespace could be copied into a new
@@ -240,8 +310,15 @@ the now-ordered definition namespace could be copied into a new
 
 However, using ``OrderedDict`` for ``__dict__`` would obscure the
 relationship with the definition namespace, making it less useful.
-Additionally, doing this would require significant changes to the
+
-semantics of the concrete ``dict`` C-API.
+Additionally, (in the case of ``OrderedDict`` specifically) doing
+this would require significant changes to the semantics of the
+concrete ``dict`` C-API.
+
+There has been some discussion about moving to a compact dict
+implementation which would (mostly) preserve insertion order.  However
+the lack of an explicit ``__definition_order__`` would still remain
+as a pain point.
 
 A "namespace" Keyword Arg for Class Definition
 ----------------------------------------------