Update PEP 544 (#644)

* Rename `@runtime` to `@runtime_checkable`
* Postpone making mappings and sequences protocols, this can be done later in Python 3.8
* Update the implementation status
* More strict and precise specification for `isinstance()` and `issubclass()` with protocols.
* Few typos

pep-0544.txt

@@ -199,8 +199,8 @@ approaches related to structural subtyping in Python and other languages:
   Such behavior seems to be a perfect fit for both runtime and static behavior
   of protocols. As discussed in `rationale`_, we propose to add static support
   for such behavior. In addition, to allow users to achieve such runtime
-  behavior for *user-defined* protocols a special ``@runtime`` decorator will
+  behavior for *user-defined* protocols a special ``@runtime_checkable`` decorator
-  be provided, see detailed `discussion`_ below.
+  will be provided, see detailed `discussion`_ below.
 
 * TypeScript [typescript]_ provides support for user-defined classes and
   interfaces. Explicit implementation declaration is not required and
@@ -381,8 +381,7 @@ Explicitly declaring implementation
 
 To explicitly declare that a certain class implements a given protocol,
 it can be used as a regular base class. In this case a class could use
-default implementations of protocol members. ``typing.Sequence`` is a good
+default implementations of protocol members. Static analysis tools are
-example of a protocol with useful default methods. Static analysis tools are
 expected to automatically detect that a class implements a given protocol.
 So while it's possible to subclass a protocol explicitly, it's *not necessary*
 to do so for the sake of type-checking.
@@ -665,14 +664,14 @@ classes. For example::
 One can use multiple inheritance to define an intersection of protocols.
 Example::
 
-  from typing import Sequence, Hashable
+  from typing import Iterable, Hashable
 
-  class HashableFloats(Sequence[float], Hashable, Protocol):
+  class HashableFloats(Iterable[float], Hashable, Protocol):
       pass
 
   def cached_func(args: HashableFloats) -> float:
       ...
-  cached_func((1, 2, 3)) # OK, tuple is both hashable and sequence
+  cached_func((1, 2, 3)) # OK, tuple is both hashable and iterable
 
 If this will prove to be a widely used scenario, then a special
 intersection type construct could be added in future as specified by PEP 483,
@@ -740,8 +739,8 @@ aliases::
 
 .. _discussion:
 
-``@runtime`` decorator and narrowing types by ``isinstance()``
+``@runtime_checkable`` decorator and narrowing types by ``isinstance()``
---------------------------------------------------------------
+------------------------------------------------------------------------
 
 The default semantics is that ``isinstance()`` and ``issubclass()`` fail
 for protocol types. This is in the spirit of duck typing -- protocols
@@ -753,37 +752,57 @@ instance and class checks when this makes sense, similar to how ``Iterable``
 and other ABCs in ``collections.abc`` and ``typing`` already do it,
 but this is limited to non-generic and unsubscripted generic protocols
 (``Iterable`` is statically equivalent to ``Iterable[Any]``).
-The ``typing`` module will define a special ``@runtime`` class decorator
+The ``typing`` module will define a special ``@runtime_checkable`` class decorator
 that provides the same semantics for class and instance checks as for
 ``collections.abc`` classes, essentially making them "runtime protocols"::
 
   from typing import runtime, Protocol
 
-  @runtime
+  @runtime_checkable
-  class Closable(Protocol):
+  class SupportsClose(Protocol):
       def close(self):
           ...
 
-  assert isinstance(open('some/file'), Closable)
+  assert isinstance(open('some/file'), SupportsClose)
-
-Static type checkers will understand ``isinstance(x, Proto)`` and
-``issubclass(C, Proto)`` for protocols defined with this decorator (as they
-already do for ``Iterable`` etc.). Static type checkers will narrow types
-after such checks by the type erased ``Proto`` (i.e. with all variables
-having type ``Any`` and all methods having type ``Callable[..., Any]``).
-Note that ``isinstance(x, Proto[int])`` etc. will always fail in agreement
-with PEP 484. Examples::
-
-  from typing import Iterable, Iterator, Sequence
-
-  def process(items: Iterable[int]) -> None:
-      if isinstance(items, Iterator):
-          # 'items' has type 'Iterator[int]' here
-      elif isinstance(items, Sequence[int]):
-          # Error! Can't use 'isinstance()' with subscripted protocols
 
 Note that instance checks are not 100% reliable statically, this is why
 this behavior is opt-in, see section on `rejected`_ ideas for examples.
+The most type checkers can do is to treat ``isinstance(obj, Iterator)``
+roughly as a simpler way to write
+``hasattr(x, '__iter__') and hasattr(x, '__next__')``. To minimize
+the risks for this feature, the following rules are applied.
+
+**Definitions**:
+
+* *Data, and non-data protocols*: A protocol is called non-data protocol
+  if it only contains methods as members (for example ``Sized``,
+  ``Iterator``, etc). A protocol that contains at least one non-method member
+  (like ``x: int``) is called a data protocol.
+* *Unsafe overlap*: A type ``X`` is called unsafely overlapping with
+  a protocol ``P``, if ``X`` is not a subtype of ``P``, but it is a subtype
+  of the type erased version of ``P`` where all members have type ``Any``.
+  In addition, if at least one element of a union unsafely overlaps with
+  a protocol ``P``, then the whole union is unsafely overlapping with ``P``.
+
+**Specification**:
+
+* A protocol can be used as a second argument in ``isinstance()`` and
+  ``issubclass()`` only if it is explicitly opt-in by ``@runtime_checkable``
+  decorator. This requirement exists because protocol checks are not type safe
+  in case of dynamically set attributes, and because type checkers can only prove
+  that an ``isinstance()`` check is safe only for a given class, not for all its
+  subclasses.
+* ``isinstance()`` can be used with both data and non-data protocols, while
+  ``issubclass()`` can be used only with non-data protocols. This restriction
+  exists because some data attributes can be set on an instance in constructor
+  and this information is not always available on the class object.
+* Type checkers should reject an ``isinstance()`` or ``issubclass()`` call, if
+  there is an unsafe overlap between the type of the first argument and
+  the protocol.
+* Type checkers should be able to select a correct element from a union after
+  a safe ``isinstance()`` or ``issubclass()`` call. For narrowing from non-union
+  types, type checkers can use their best judgement (this is intentionally
+  unspecified, since a precise specification would require intersection types).
 
 
 Using Protocols in Python 2.7 - 3.5
@@ -825,14 +844,12 @@ effects on the core interpreter and standard library except in the
   a protocol or not. Add a class attribute ``_is_protocol = True``
   if that is the case. Verify that a protocol class only has protocol
   base classes in the MRO (except for object).
-* Implement ``@runtime`` that allows ``__subclasshook__()`` performing
+* Implement ``@runtime_checkable`` that allows ``__subclasshook__()``
-  structural instance and subclass checks as in ``collections.abc`` classes.
+  performing structural instance and subclass checks as in ``collections.abc``
+  classes.
 * All structural subtyping checks will be performed by static type checkers,
   such as ``mypy`` [mypy]_. No additional support for protocol validation will
   be provided at runtime.
-* Classes ``Mapping``, ``MutableMapping``, ``Sequence``, and
-  ``MutableSequence`` in ``collections.abc`` module will support structural
-  instance and subclass checks (like e.g. ``collections.abc.Iterable``).
 
 
 Changes in the typing module
@@ -849,8 +866,6 @@ The following classes in ``typing`` module will be protocols:
 * ``Container``
 * ``Collection``
 * ``Reversible``
-* ``Sequence``, ``MutableSequence``
-* ``Mapping``, ``MutableMapping``
 * ``ContextManager``, ``AsyncContextManager``
 * ``SupportsAbs`` (and other ``Supports*`` classes)
 
@@ -1026,11 +1041,10 @@ be considered "non-protocol". Therefore, it was decided to not introduce
 "non-protocol" methods.
 
 There is only one downside to this: it will require some boilerplate for
-implicit subtypes of ``Mapping`` and few other "large" protocols. But, this
+implicit subtypes of "large" protocols. But, this doesn't apply to "built-in"
-applies to few "built-in" protocols (like ``Mapping`` and ``Sequence``) and
+protocols that are all "small" (i.e. have only few abstract methods).
-people are already subclassing them. Also, such style is discouraged for
+Also, such style is discouraged for user-defined protocols. It is recommended
-user-defined protocols. It is recommended to create compact protocols and
+to create compact protocols and combine them.
-combine them.
 
 
 Make protocols interoperable with other approaches
@@ -1103,7 +1117,7 @@ Another potentially problematic case is assignment of attributes
           self.x = 0
 
   c = C()
-  isinstance(c1, P)  # False
+  isinstance(c, P)  # False
   c.initialize()
   isinstance(c, P)  # True
 
@@ -1149,7 +1163,7 @@ This was rejected for the following reasons:
   ABCs from ``typing`` module. If we prohibit explicit subclassing of these
   ABCs, then quite a lot of code will break.
 
-* Convenience: There are existing protocol-like ABCs (that will be turned
+* Convenience: There are existing protocol-like ABCs (that may be turned
   into protocols) that have many useful "mix-in" (non-abstract) methods.
   For example in the case of ``Sequence`` one only needs to implement
   ``__getitem__`` and ``__len__`` in an explicit subclass, and one gets
@@ -1301,33 +1315,16 @@ confusions.
 Backwards Compatibility
 =======================
 
-This PEP is almost fully backwards compatible. Few collection classes such as
+This PEP is fully backwards compatible.
-``Sequence`` and ``Mapping`` will be turned into runtime protocols, therefore
-results of ``isinstance()`` checks are going to change in some edge cases.
-For example, a class that implements the ``Sequence`` protocol but does not
-explicitly inherit from ``Sequence`` currently returns ``False`` in
-corresponding instance and class checks. With this PEP implemented, such
-checks will return ``True``.
 
 
 Implementation
 ==============
 
-A working implementation of this PEP for ``mypy`` type checker is found on
+The ``mypy`` type checker fully supports protocols (modulo a few
-GitHub repo at https://github.com/ilevkivskyi/mypy/tree/protocols,
+known bugs). This includes treating all the builtin protocols, such as
-corresponding ``typeshed`` stubs for more flavor are found at
+``Iterable`` structurally. The runtime implementation of protocols is
-https://github.com/ilevkivskyi/typeshed/tree/protocols. Installation steps::
+available in ``typing_extensions`` module on PyPI.
-
-  git clone --recurse-submodules https://github.com/ilevkivskyi/mypy/
-  cd mypy && git checkout protocols && cd typeshed
-  git remote add proto https://github.com/ilevkivskyi/typeshed
-  git fetch proto && git checkout proto/protocols
-  cd .. && git add typeshed && sudo python3 -m pip install -U .
-
-The runtime implementation of protocols in ``typing`` module is
-found at https://github.com/ilevkivskyi/typehinting/tree/protocols.
-The version of ``collections.abc`` with structural behavior for mappings and
-sequences is found at https://github.com/ilevkivskyi/cpython/tree/protocols.
 
 
 References