Checkpoint. Rewrote (and swapped) the sections on

isinstance/issubclass overriding and the support framework.
Next up: redesign the collection ABCs.

pep-3119.txt

@@ -16,12 +16,12 @@ Abstract
 This is a proposal to add Abstract Base Class (ABC) support to Python
 3000.  It proposes:
 
-* An "ABC support framework" which defines a built-in decorator that
-  can be used to define abstract methods.  A class containing an
-  abstract method that isn't overridden cannot be instantiated.
-
 * A way to overload ``isinstance()`` and ``issubclass()``.
 
+* A new module ``abc`` which serves as an "ABC support framework".  It
+  defines a metaclass for use with ABCs and a decorator that can be
+  used to define abstract methods.
+
 * Specific ABCs for containers and iterators, to be added to the
   collections module.
 
@@ -100,8 +100,8 @@ This PEP proposes a particular strategy for organizing these tests
 known as Abstract Base Classes, or ABC.  ABCs are simply Python
 classes that are added into an object's inheritance tree to signal
 certain features of that object to an external inspector.  Tests are
-done using isinstance(), and the presence of a particular ABC means
+done using ``isinstance()``, and the presence of a particular ABC
-that the test has passed.
+means that the test has passed.
 
 In addition, the ABCs define a minimal set of methods that establish
 the characteristic behavior of the type.  Code that discriminates
@@ -123,25 +123,197 @@ Specification
 
 The specification follows the categories listed in the abstract:
 
-* An "ABC support framework" which defines a built-in decorator that
-  make it easy to define ABCs, and mechanisms to support it.
-
 * A way to overload ``isinstance()`` and ``issubclass()``.
 
+* A new module ``abc`` which serves as an "ABC support framework".  It
+  defines a metaclass for use with ABCs and a decorator that can be
+  used to define abstract methods.
+
 * Specific ABCs for containers and iterators, to be added to the
   collections module.
 
 
-ABC Support Framework
+Overloading ``isinstance()`` and ``issubclass()``
----------------------
+-------------------------------------------------
 
-We define a new built-in decorator, ``@abstractmethod``, to be used to
+During the development of this PEP and of its companion, PEP 3141, we
-declare abstract methods.  A class containing at least one method
+repeatedly faced the choice between standardizing more, fine-grained
-declared with this decorator that hasn't been overridden yet cannot be
+ABCs or fewer, course-grained ones.  For example, at one stage, PEP
-instantiated.  Such a methods may be called from the overriding method
+3141 introduced the following stack of base classes used for complex
-in the subclass (using ``super`` or direct invocation).  For example::
+numbers: MonoidUnderPlus, AdditiveGroup, Ring, Field, Complex (each
+derived from the previous).  And the discussion mentioned several
+other algebraic categorizations that were left out: Algebraic,
+Transcendental, and IntegralDomain, and PrincipalIdealDomain.  In
+earlier versions of the current PEP, we considered the use cases for
+separate classes like Set, ComposableSet, MutableSet, HashableSet,
+MutableComposableSet, HashableComposableSet.
 
-    class A:
+The dilemma here is that we'd rather have fewer ABCs, but then what
+should a user do who needs a less refined ABC?  Consider e.g. the
+plight of a mathematician who wants to define his own kind of
+Transcendental numbers, but also wants float and int to be considered
+Transcendental.  PEP 3141 originally proposed to patch float.__bases__
+for that purpose, but there are some good reasons to keep the built-in
+types immutable (for one, they are shared between all Python
+interpreters running in the same address space, as is used by
+mod_python).
+
+Another example would be someone who wants to define a generic
+function (PEP 3124) for any sequences that has an ``append()`` method.
+The ``Sequence`` ABC (see below) doesn't promise the ``append()``
+method, while ``MutableSequence`` requires not only ``append()`` but
+also various other mutating methods.
+
+To solve these and similar dilemmas, the next section will propose a
+metaclass for use with ABCs that will allow us to add an ABC as a
+"virtual base class" (not the same concept as in C++) to any class,
+including to another ABC.  This allows the standard library to define
+ABCs ``Sequence`` and ``MutableSequence`` and register these as
+virtual base classes for built-in types like ``basestring``, ``tuple``
+and ``list``, so that for example the following conditions are all
+true::
+
+    isinstance([], Sequence)
+    issubclass(list, Sequence)
+    issubclass(list, MutableSequence)
+    isinstance((), Sequence)
+    not issubclass(tuple, MutableSequence)
+    isinstance("", Sequence)
+    issubclass(bytes, MutableSequence)
+
+The primary mechanism proposed here is to allow overloading the
+built-in functions ``isinstance()`` and ``issubclass()``.  The
+overloading works as follows: The call ``isinstance(x, C)`` first
+checks whether ``C.__instancecheck__`` exists, and if so, calls
+``C.__instancecheck__(x)`` instead of its normal implementation.
+Similarly, the call ``issubclass(D, C)`` first checks whether
+``C.__subclasscheck__`` exists, and if so, calls
+``C.__subclasscheck__(D)`` instead of its normal implementation.
+
+Note that the magic names are not ``__isinstance__`` and
+``__issubclass__``; this is because the reversal of the arguments
+could cause confusion, especially for the ``issubclass()`` overloader.
+
+A prototype implementation of this is given in [12]_.
+
+Here is an example with (naively simple) implementations of
+``__instancecheck__`` and ``__subclasscheck__``::
+
+    class ABCMeta(type):
+
+        def __instancecheck__(cls, inst):
+            """Implement isinstance(inst, cls)."""
+            return any(cls.__subclasscheck__(c)
+                       for c in {type(inst), inst.__class__})
+
+        def __subclasscheck__(cls, sub):
+            """Implement issubclass(sub, cls)."""
+            candidates = cls.__dict__.get("__subclass__", set()) | {cls}
+            return any(c in candidates for c in sub.mro())
+
+    class Sequence(metaclass=ABCMeta):
+        __subclass__ = {list, tuple}
+
+    assert issubclass(list, Sequence)
+    assert issubclass(tuple, Sequence)
+
+    class AppendableSequence(Sequence):
+        __subclass__ = {list}
+
+    assert issubclass(list, AppendableSequence)
+    assert isinstance([], AppendableSequence)
+
+    assert not issubclass(tuple, AppendableSequence)
+    assert not isinstance((), AppendableSequence)
+
+The next section proposes a full-fledged implementation.
+
+
+The ``abc`` Module: an ABC Support Framework
+--------------------------------------------
+
+The new standard library module ``abc``, written in pure Python,
+serves as an ABC support framework.  It defines a metaclass
+``ABCMeta`` and a decorator ``@abstractmethod``.  A sample
+implementation is given by [13]_.
+
+The ``ABCMeta`` class overrides ``__instancecheck__`` and
+``__subclasscheck__`` and defines a ``register`` method.  The
+``register`` method takes one argument, which much be a class; after
+the call ``B.register(C)``, the call ``issubclass(C, B)`` will return
+True, by virtue of of ``B.__subclasscheck__(C)`` returning True.
+Also, ``isinstance(x, B)`` is equivalent to ``issubclass(x.__class__,
+B) or issubclass(type(x), B)``.  (It is possible ``type(x)`` and
+``x.__class__`` are not the same object, e.g. when x is a proxy
+object.)
+
+These methods are intended to be be called on classes whose metaclass
+is (derived from) ``ABCMeta``; for example::
+
+    from abc import ABCMeta
+
+    class MyABC(metaclass=ABCMeta):
+        pass
+
+    MyABC.register(tuple)
+
+    assert issubclass(tuple, MyABC)
+    assert isinstance((), MyABC)
+
+The last two asserts are equivalent to the following two::
+
+    assert MyABC.__subclasscheck__(tuple)
+    assert MyABC.__instancecheck__(())
+
+Of course, you can also directly subclass MyABC::
+
+    class MyClass(MyABC):
+        pass
+
+    assert issubclass(MyClass, MyABC)
+    assert isinstance(MyClass(), MyABC)
+
+Also, of course, a tuple is not a ``MyClass``::
+
+    assert not issubclass(tuple, MyClass)
+    assert not isinstance((), MyClass)
+
+You can register another class as a subclass of ``MyClass``::
+
+    MyClass.register(list)
+
+    assert issubclass(list, MyClass)
+    assert issubclass(list, MyABC)
+
+You can also register another ABC::
+
+    class AnotherClass(metaclass=ABCMeta):
+        pass
+
+    AnotherClass.register(basestring)
+
+    MyClass.register(AnotherClass)
+
+    assert isinstance(str, MyABC)
+
+That last assert requires tracing the following superclass-subclass
+relationships::
+
+    MyABC -> MyClass (using regular subclassing)
+    MyClass -> AnotherClass (using registration)
+    AnotherClass -> basestring (using registration)
+    basestring -> str (using regular subclassing)
+
+The ``abc`` module also defines a new decorator, ``@abstractmethod``,
+to be used to declare abstract methods.  A class containing at least
+one method declared with this decorator that hasn't been overridden
+yet cannot be instantiated.  Such a methods may be called from the
+overriding method in the subclass (using ``super`` or direct
+invocation).  For example::
+
+    from abc import ABCMeta, abstractmethod
+
+    class A(metaclass=ABCMeta):
         @abstractmethod
         def foo(self): pass
 
@@ -157,14 +329,17 @@ in the subclass (using ``super`` or direct invocation).  For example::
 
     C()  # works
 
-**Note:** The ``@abstractmethod`` decorator should only be used inside
+**Notes:** The ``@abstractmethod`` decorator should only be used
-a class body.  Dynamically adding abstract methods to a class, or
+inside a class body, and only for classes whose metaclass is (derived
+from) ``ABCMeta``.  Dynamically adding abstract methods to a class, or
 attempting to modify the abstraction status of a method or class once
-it is created, are not supported.
+it is created, are not supported.  The ``@abstractmethod`` only
+affects subclasses derived using regular inheritance; "virtual
+subclasses" registered with the ``register()`` method are not affected.
 
 **Implementation:** The ``@abstractmethod`` decorator sets the
 function attribute ``__isabstractmethod__`` to the value ``True``.
-The ``type.__new__`` method computes the type attribute
+The ``ABCMeta.__new__`` method computes the type attribute
 ``__abstractmethods__`` as the set of all method names that have an
 ``__isabstractmethod__`` attribute whose value is true.  It does this
 by combining the ``__abstractmethods__`` attributes of the base
@@ -174,8 +349,8 @@ of all methods in the new class dict that don't have a true
 ``__isabstractmethod__`` attribute.  If the resulting
 ``__abstractmethods__`` set is non-empty, the class is considered
 abstract, and attempts to instantiate it will raise ``TypeError``.
-(CPython can uses an internal flag ``Py_TPFLAGS_ABSTRACT`` to speed up
+(If this were implemented in CPython, an internal flag
-this check [6]_.)
+``Py_TPFLAGS_ABSTRACT`` could be used to speed up this check [6]_.)
 
 **Discussion:** Unlike C++ or Java, abstract methods as defined here
 may have an implementation.  This implementation can be called via the
@@ -186,77 +361,8 @@ cooperative multiple-inheritance [7]_, [8]_.
 **Open issues:** Should we also provide a standard way to declare
 abstract data attributes?  If so, how should these be spelled?
 Perhaps place ``@abstractattribute`` decorators on properties?  Or use
-an ``@attributes(name1, name2, ...)`` class decorator?
+an ``@attributes(name1, name2, ...)`` class decorator?  Strawman:
-
+let's back off on this; it's easy enough to add this later.
-
-Overloading ``isinstance()`` and ``issubclass()``
--------------------------------------------------
-
-During the development of this PEP and of its companion, PEP 3141, we
-repeatedly faced the choice between standardizing more, fine-grained
-ABCs or fewer, course-grained ones.  For example, at one stage, PEP
-3141 introduced the following stack of base classes used for complex
-numbers: MonoidUnderPlus, AdditiveGroup, Ring, Field, Complex (each
-derived from the previous).  And the discussion mentioned several
-other algebraic categorizations that were left out: Algebraic,
-Transcendental, and IntegralDomain, and PrincipalIdealDomain.  In this
-PEP, we are wondering about the use cases for separate classes like
-Set, ComposableSet, MutableSet, HashableSet, MutableComposableSet,
-HashableComposableSet.
-
-The dilemma here is that we'd rather have fewer ABCs, but then what
-should a user do who needs a less refined ABC?  Consider e.g. the
-plight of a mathematician who wants to define his own kind of
-Transcendental numbers, but also wants float and int to be considered
-Transcendental.  PEP 3141 originally proposed to patch float.__bases__
-for that purpose, but there are some good reasons to keep the built-in
-types immutable (for one, they are shared between all Python
-interpreters running in the same address space, as is used by
-mod_python).
-
-The solution proposed here is to allow overloading the built-in
-functions ``isinstance()`` and ``issubclass()``.  The overloading
-works as follows: The call ``isinstance(x, C)`` first checks whether
-``C.__instancecheck__`` exists, and if so, calls
-``C.__instancecheck__(x)`` instead of its normal implementation.
-Similarly, the call ``issubclass(D, C)`` first checks whether
-``C.__subclasscheck__`` exists, and if so, calls
-``C.__subclasscheck__(D)`` instead of its normal implementation.
-
-Note that the magic names are not ``__isinstance__`` and
-``__issubclass__``; this is because the reversal of the arguments
-could cause confusion, especially for the ``issubclass()`` overloader.
-
-(We could also provide a default implementation of these that
-implements the old algorithms; this would be more regular but would
-have additional backwards compatibility issues, since the old
-algorithms special-case objects not deriving from ``type`` in order to
-support a different kind of overloading of these operations.)
-
-A prototype implementation of this is given in [12]_.
-
-Here is an example with (very simple) implementations of
-``__instancecheck__`` and ``__subclasscheck__``::
-
-    assert issubclass(set, HashableSet)  # Assume this is given
-
-    class ABC:
-
-        def __instancecheck__(cls, inst):
-            """Implement isinstance(inst, cls)."""
-            return any(cls.__subclasscheck__(c)
-                       for c in {type(inst), inst.__class__})
-
-        def __subclasscheck__(cls, sub):
-            """Implement issubclass(sub, cls)."""
-            candidates = cls.__dict__.get("__subclass__", set()) | {cls}
-            return any(c in candidates for c in sub.mro())
-
-    class NoncomposableHashableSet(Set, Hashable, metaclass=ABC):
-        __subclass__ = {HashableSet}
-
-    assert issubclass(set, NoncomposableHashableSet)
-    assert isinstance({1, 2, 3}, NoncomposableHashableSet)
 
 
 ABCs for Containers and Iterators
@@ -831,6 +937,9 @@ References
 .. [12] Make isinstance/issubclass overloadable
    (http://python.org/sf/1708353)
 
+.. [13] ABCMeta sample implementation
+   (http://svn.python.org/view/sandbox/trunk/abc/xyz.py)
+
 
 Copyright
 =========