diff --git a/pep-0544.txt b/pep-0544.txt index e75d78302..e8023ae11 100644 --- a/pep-0544.txt +++ b/pep-0544.txt @@ -199,8 +199,8 @@ approaches related to structural subtyping in Python and other languages: Such behavior seems to be a perfect fit for both runtime and static behavior of protocols. As discussed in `rationale`_, we propose to add static support for such behavior. In addition, to allow users to achieve such runtime - behavior for *user-defined* protocols a special ``@runtime`` decorator will - be provided, see detailed `discussion`_ below. + behavior for *user-defined* protocols a special ``@runtime_checkable`` decorator + will be provided, see detailed `discussion`_ below. * TypeScript [typescript]_ provides support for user-defined classes and interfaces. Explicit implementation declaration is not required and @@ -381,8 +381,7 @@ Explicitly declaring implementation To explicitly declare that a certain class implements a given protocol, it can be used as a regular base class. In this case a class could use -default implementations of protocol members. ``typing.Sequence`` is a good -example of a protocol with useful default methods. Static analysis tools are +default implementations of protocol members. Static analysis tools are expected to automatically detect that a class implements a given protocol. So while it's possible to subclass a protocol explicitly, it's *not necessary* to do so for the sake of type-checking. @@ -665,14 +664,14 @@ classes. For example:: One can use multiple inheritance to define an intersection of protocols. Example:: - from typing import Sequence, Hashable + from typing import Iterable, Hashable - class HashableFloats(Sequence[float], Hashable, Protocol): + class HashableFloats(Iterable[float], Hashable, Protocol): pass def cached_func(args: HashableFloats) -> float: ... - cached_func((1, 2, 3)) # OK, tuple is both hashable and sequence + cached_func((1, 2, 3)) # OK, tuple is both hashable and iterable If this will prove to be a widely used scenario, then a special intersection type construct could be added in future as specified by PEP 483, @@ -740,8 +739,8 @@ aliases:: .. _discussion: -``@runtime`` decorator and narrowing types by ``isinstance()`` --------------------------------------------------------------- +``@runtime_checkable`` decorator and narrowing types by ``isinstance()`` +------------------------------------------------------------------------ The default semantics is that ``isinstance()`` and ``issubclass()`` fail for protocol types. This is in the spirit of duck typing -- protocols @@ -753,37 +752,57 @@ instance and class checks when this makes sense, similar to how ``Iterable`` and other ABCs in ``collections.abc`` and ``typing`` already do it, but this is limited to non-generic and unsubscripted generic protocols (``Iterable`` is statically equivalent to ``Iterable[Any]``). -The ``typing`` module will define a special ``@runtime`` class decorator +The ``typing`` module will define a special ``@runtime_checkable`` class decorator that provides the same semantics for class and instance checks as for ``collections.abc`` classes, essentially making them "runtime protocols":: from typing import runtime, Protocol - @runtime - class Closable(Protocol): + @runtime_checkable + class SupportsClose(Protocol): def close(self): ... - assert isinstance(open('some/file'), Closable) - -Static type checkers will understand ``isinstance(x, Proto)`` and -``issubclass(C, Proto)`` for protocols defined with this decorator (as they -already do for ``Iterable`` etc.). Static type checkers will narrow types -after such checks by the type erased ``Proto`` (i.e. with all variables -having type ``Any`` and all methods having type ``Callable[..., Any]``). -Note that ``isinstance(x, Proto[int])`` etc. will always fail in agreement -with PEP 484. Examples:: - - from typing import Iterable, Iterator, Sequence - - def process(items: Iterable[int]) -> None: - if isinstance(items, Iterator): - # 'items' has type 'Iterator[int]' here - elif isinstance(items, Sequence[int]): - # Error! Can't use 'isinstance()' with subscripted protocols + assert isinstance(open('some/file'), SupportsClose) Note that instance checks are not 100% reliable statically, this is why this behavior is opt-in, see section on `rejected`_ ideas for examples. +The most type checkers can do is to treat ``isinstance(obj, Iterator)`` +roughly as a simpler way to write +``hasattr(x, '__iter__') and hasattr(x, '__next__')``. To minimize +the risks for this feature, the following rules are applied. + +**Definitions**: + +* *Data, and non-data protocols*: A protocol is called non-data protocol + if it only contains methods as members (for example ``Sized``, + ``Iterator``, etc). A protocol that contains at least one non-method member + (like ``x: int``) is called a data protocol. +* *Unsafe overlap*: A type ``X`` is called unsafely overlapping with + a protocol ``P``, if ``X`` is not a subtype of ``P``, but it is a subtype + of the type erased version of ``P`` where all members have type ``Any``. + In addition, if at least one element of a union unsafely overlaps with + a protocol ``P``, then the whole union is unsafely overlapping with ``P``. + +**Specification**: + +* A protocol can be used as a second argument in ``isinstance()`` and + ``issubclass()`` only if it is explicitly opt-in by ``@runtime_checkable`` + decorator. This requirement exists because protocol checks are not type safe + in case of dynamically set attributes, and because type checkers can only prove + that an ``isinstance()`` check is safe only for a given class, not for all its + subclasses. +* ``isinstance()`` can be used with both data and non-data protocols, while + ``issubclass()`` can be used only with non-data protocols. This restriction + exists because some data attributes can be set on an instance in constructor + and this information is not always available on the class object. +* Type checkers should reject an ``isinstance()`` or ``issubclass()`` call, if + there is an unsafe overlap between the type of the first argument and + the protocol. +* Type checkers should be able to select a correct element from a union after + a safe ``isinstance()`` or ``issubclass()`` call. For narrowing from non-union + types, type checkers can use their best judgement (this is intentionally + unspecified, since a precise specification would require intersection types). Using Protocols in Python 2.7 - 3.5 @@ -825,14 +844,12 @@ effects on the core interpreter and standard library except in the a protocol or not. Add a class attribute ``_is_protocol = True`` if that is the case. Verify that a protocol class only has protocol base classes in the MRO (except for object). -* Implement ``@runtime`` that allows ``__subclasshook__()`` performing - structural instance and subclass checks as in ``collections.abc`` classes. +* Implement ``@runtime_checkable`` that allows ``__subclasshook__()`` + performing structural instance and subclass checks as in ``collections.abc`` + classes. * All structural subtyping checks will be performed by static type checkers, such as ``mypy`` [mypy]_. No additional support for protocol validation will be provided at runtime. -* Classes ``Mapping``, ``MutableMapping``, ``Sequence``, and - ``MutableSequence`` in ``collections.abc`` module will support structural - instance and subclass checks (like e.g. ``collections.abc.Iterable``). Changes in the typing module @@ -849,8 +866,6 @@ The following classes in ``typing`` module will be protocols: * ``Container`` * ``Collection`` * ``Reversible`` -* ``Sequence``, ``MutableSequence`` -* ``Mapping``, ``MutableMapping`` * ``ContextManager``, ``AsyncContextManager`` * ``SupportsAbs`` (and other ``Supports*`` classes) @@ -1026,11 +1041,10 @@ be considered "non-protocol". Therefore, it was decided to not introduce "non-protocol" methods. There is only one downside to this: it will require some boilerplate for -implicit subtypes of ``Mapping`` and few other "large" protocols. But, this -applies to few "built-in" protocols (like ``Mapping`` and ``Sequence``) and -people are already subclassing them. Also, such style is discouraged for -user-defined protocols. It is recommended to create compact protocols and -combine them. +implicit subtypes of "large" protocols. But, this doesn't apply to "built-in" +protocols that are all "small" (i.e. have only few abstract methods). +Also, such style is discouraged for user-defined protocols. It is recommended +to create compact protocols and combine them. Make protocols interoperable with other approaches @@ -1103,7 +1117,7 @@ Another potentially problematic case is assignment of attributes self.x = 0 c = C() - isinstance(c1, P) # False + isinstance(c, P) # False c.initialize() isinstance(c, P) # True @@ -1149,7 +1163,7 @@ This was rejected for the following reasons: ABCs from ``typing`` module. If we prohibit explicit subclassing of these ABCs, then quite a lot of code will break. -* Convenience: There are existing protocol-like ABCs (that will be turned +* Convenience: There are existing protocol-like ABCs (that may be turned into protocols) that have many useful "mix-in" (non-abstract) methods. For example in the case of ``Sequence`` one only needs to implement ``__getitem__`` and ``__len__`` in an explicit subclass, and one gets @@ -1301,33 +1315,16 @@ confusions. Backwards Compatibility ======================= -This PEP is almost fully backwards compatible. Few collection classes such as -``Sequence`` and ``Mapping`` will be turned into runtime protocols, therefore -results of ``isinstance()`` checks are going to change in some edge cases. -For example, a class that implements the ``Sequence`` protocol but does not -explicitly inherit from ``Sequence`` currently returns ``False`` in -corresponding instance and class checks. With this PEP implemented, such -checks will return ``True``. +This PEP is fully backwards compatible. Implementation ============== -A working implementation of this PEP for ``mypy`` type checker is found on -GitHub repo at https://github.com/ilevkivskyi/mypy/tree/protocols, -corresponding ``typeshed`` stubs for more flavor are found at -https://github.com/ilevkivskyi/typeshed/tree/protocols. Installation steps:: - - git clone --recurse-submodules https://github.com/ilevkivskyi/mypy/ - cd mypy && git checkout protocols && cd typeshed - git remote add proto https://github.com/ilevkivskyi/typeshed - git fetch proto && git checkout proto/protocols - cd .. && git add typeshed && sudo python3 -m pip install -U . - -The runtime implementation of protocols in ``typing`` module is -found at https://github.com/ilevkivskyi/typehinting/tree/protocols. -The version of ``collections.abc`` with structural behavior for mappings and -sequences is found at https://github.com/ilevkivskyi/cpython/tree/protocols. +The ``mypy`` type checker fully supports protocols (modulo a few +known bugs). This includes treating all the builtin protocols, such as +``Iterable`` structurally. The runtime implementation of protocols is +available in ``typing_extensions`` module on PyPI. References