Update PEP 544 (#644)
* Rename `@runtime` to `@runtime_checkable` * Postpone making mappings and sequences protocols, this can be done later in Python 3.8 * Update the implementation status * More strict and precise specification for `isinstance()` and `issubclass()` with protocols. * Few typos
This commit is contained in:
Ivan Levkivskyi
Guido van Rossum
parent
86c2708d7d
commit
9c06059758
127
pep-0544.txt
127
pep-0544.txt
|
|
@ -199,8 +199,8 @@ approaches related to structural subtyping in Python and other languages:
|
|||
Such behavior seems to be a perfect fit for both runtime and static behavior
|
||||
of protocols. As discussed in `rationale`_, we propose to add static support
|
||||
for such behavior. In addition, to allow users to achieve such runtime
|
||||
behavior for *user-defined* protocols a special ``@runtime`` decorator will
|
||||
be provided, see detailed `discussion`_ below.
|
||||
behavior for *user-defined* protocols a special ``@runtime_checkable`` decorator
|
||||
will be provided, see detailed `discussion`_ below.
|
||||
|
||||
* TypeScript [typescript]_ provides support for user-defined classes and
|
||||
interfaces. Explicit implementation declaration is not required and
|
||||
|
|
@ -381,8 +381,7 @@ Explicitly declaring implementation
|
|||
|
||||
To explicitly declare that a certain class implements a given protocol,
|
||||
it can be used as a regular base class. In this case a class could use
|
||||
default implementations of protocol members. ``typing.Sequence`` is a good
|
||||
example of a protocol with useful default methods. Static analysis tools are
|
||||
default implementations of protocol members. Static analysis tools are
|
||||
expected to automatically detect that a class implements a given protocol.
|
||||
So while it's possible to subclass a protocol explicitly, it's *not necessary*
|
||||
to do so for the sake of type-checking.
|
||||
|
|
@ -665,14 +664,14 @@ classes. For example::
|
|||
One can use multiple inheritance to define an intersection of protocols.
|
||||
Example::
|
||||
|
||||
from typing import Sequence, Hashable
|
||||
from typing import Iterable, Hashable
|
||||
|
||||
class HashableFloats(Sequence[float], Hashable, Protocol):
|
||||
class HashableFloats(Iterable[float], Hashable, Protocol):
|
||||
pass
|
||||
|
||||
def cached_func(args: HashableFloats) -> float:
|
||||
...
|
||||
cached_func((1, 2, 3)) # OK, tuple is both hashable and sequence
|
||||
cached_func((1, 2, 3)) # OK, tuple is both hashable and iterable
|
||||
|
||||
If this will prove to be a widely used scenario, then a special
|
||||
intersection type construct could be added in future as specified by PEP 483,
|
||||
|
|
@ -740,8 +739,8 @@ aliases::
|
|||
|
||||
.. _discussion:
|
||||
|
||||
``@runtime`` decorator and narrowing types by ``isinstance()``
|
||||
--------------------------------------------------------------
|
||||
``@runtime_checkable`` decorator and narrowing types by ``isinstance()``
|
||||
------------------------------------------------------------------------
|
||||
|
||||
The default semantics is that ``isinstance()`` and ``issubclass()`` fail
|
||||
for protocol types. This is in the spirit of duck typing -- protocols
|
||||
|
|
@ -753,37 +752,57 @@ instance and class checks when this makes sense, similar to how ``Iterable``
|
|||
and other ABCs in ``collections.abc`` and ``typing`` already do it,
|
||||
but this is limited to non-generic and unsubscripted generic protocols
|
||||
(``Iterable`` is statically equivalent to ``Iterable[Any]``).
|
||||
The ``typing`` module will define a special ``@runtime`` class decorator
|
||||
The ``typing`` module will define a special ``@runtime_checkable`` class decorator
|
||||
that provides the same semantics for class and instance checks as for
|
||||
``collections.abc`` classes, essentially making them "runtime protocols"::
|
||||
|
||||
from typing import runtime, Protocol
|
||||
|
||||
@runtime
|
||||
class Closable(Protocol):
|
||||
@runtime_checkable
|
||||
class SupportsClose(Protocol):
|
||||
def close(self):
|
||||
...
|
||||
|
||||
assert isinstance(open('some/file'), Closable)
|
||||
|
||||
Static type checkers will understand ``isinstance(x, Proto)`` and
|
||||
``issubclass(C, Proto)`` for protocols defined with this decorator (as they
|
||||
already do for ``Iterable`` etc.). Static type checkers will narrow types
|
||||
after such checks by the type erased ``Proto`` (i.e. with all variables
|
||||
having type ``Any`` and all methods having type ``Callable[..., Any]``).
|
||||
Note that ``isinstance(x, Proto[int])`` etc. will always fail in agreement
|
||||
with PEP 484. Examples::
|
||||
|
||||
from typing import Iterable, Iterator, Sequence
|
||||
|
||||
def process(items: Iterable[int]) -> None:
|
||||
if isinstance(items, Iterator):
|
||||
# 'items' has type 'Iterator[int]' here
|
||||
elif isinstance(items, Sequence[int]):
|
||||
# Error! Can't use 'isinstance()' with subscripted protocols
|
||||
assert isinstance(open('some/file'), SupportsClose)
|
||||
|
||||
Note that instance checks are not 100% reliable statically, this is why
|
||||
this behavior is opt-in, see section on `rejected`_ ideas for examples.
|
||||
The most type checkers can do is to treat ``isinstance(obj, Iterator)``
|
||||
roughly as a simpler way to write
|
||||
``hasattr(x, '__iter__') and hasattr(x, '__next__')``. To minimize
|
||||
the risks for this feature, the following rules are applied.
|
||||
|
||||
**Definitions**:
|
||||
|
||||
* *Data, and non-data protocols*: A protocol is called non-data protocol
|
||||
if it only contains methods as members (for example ``Sized``,
|
||||
``Iterator``, etc). A protocol that contains at least one non-method member
|
||||
(like ``x: int``) is called a data protocol.
|
||||
* *Unsafe overlap*: A type ``X`` is called unsafely overlapping with
|
||||
a protocol ``P``, if ``X`` is not a subtype of ``P``, but it is a subtype
|
||||
of the type erased version of ``P`` where all members have type ``Any``.
|
||||
In addition, if at least one element of a union unsafely overlaps with
|
||||
a protocol ``P``, then the whole union is unsafely overlapping with ``P``.
|
||||
|
||||
**Specification**:
|
||||
|
||||
* A protocol can be used as a second argument in ``isinstance()`` and
|
||||
``issubclass()`` only if it is explicitly opt-in by ``@runtime_checkable``
|
||||
decorator. This requirement exists because protocol checks are not type safe
|
||||
in case of dynamically set attributes, and because type checkers can only prove
|
||||
that an ``isinstance()`` check is safe only for a given class, not for all its
|
||||
subclasses.
|
||||
* ``isinstance()`` can be used with both data and non-data protocols, while
|
||||
``issubclass()`` can be used only with non-data protocols. This restriction
|
||||
exists because some data attributes can be set on an instance in constructor
|
||||
and this information is not always available on the class object.
|
||||
* Type checkers should reject an ``isinstance()`` or ``issubclass()`` call, if
|
||||
there is an unsafe overlap between the type of the first argument and
|
||||
the protocol.
|
||||
* Type checkers should be able to select a correct element from a union after
|
||||
a safe ``isinstance()`` or ``issubclass()`` call. For narrowing from non-union
|
||||
types, type checkers can use their best judgement (this is intentionally
|
||||
unspecified, since a precise specification would require intersection types).
|
||||
|
||||
|
||||
Using Protocols in Python 2.7 - 3.5
|
||||
|
|
@ -825,14 +844,12 @@ effects on the core interpreter and standard library except in the
|
|||
a protocol or not. Add a class attribute ``_is_protocol = True``
|
||||
if that is the case. Verify that a protocol class only has protocol
|
||||
base classes in the MRO (except for object).
|
||||
* Implement ``@runtime`` that allows ``__subclasshook__()`` performing
|
||||
structural instance and subclass checks as in ``collections.abc`` classes.
|
||||
* Implement ``@runtime_checkable`` that allows ``__subclasshook__()``
|
||||
performing structural instance and subclass checks as in ``collections.abc``
|
||||
classes.
|
||||
* All structural subtyping checks will be performed by static type checkers,
|
||||
such as ``mypy`` [mypy]_. No additional support for protocol validation will
|
||||
be provided at runtime.
|
||||
* Classes ``Mapping``, ``MutableMapping``, ``Sequence``, and
|
||||
``MutableSequence`` in ``collections.abc`` module will support structural
|
||||
instance and subclass checks (like e.g. ``collections.abc.Iterable``).
|
||||
|
||||
|
||||
Changes in the typing module
|
||||
|
|
@ -849,8 +866,6 @@ The following classes in ``typing`` module will be protocols:
|
|||
* ``Container``
|
||||
* ``Collection``
|
||||
* ``Reversible``
|
||||
* ``Sequence``, ``MutableSequence``
|
||||
* ``Mapping``, ``MutableMapping``
|
||||
* ``ContextManager``, ``AsyncContextManager``
|
||||
* ``SupportsAbs`` (and other ``Supports*`` classes)
|
||||
|
||||
|
|
@ -1026,11 +1041,10 @@ be considered "non-protocol". Therefore, it was decided to not introduce
|
|||
"non-protocol" methods.
|
||||
|
||||
There is only one downside to this: it will require some boilerplate for
|
||||
implicit subtypes of ``Mapping`` and few other "large" protocols. But, this
|
||||
applies to few "built-in" protocols (like ``Mapping`` and ``Sequence``) and
|
||||
people are already subclassing them. Also, such style is discouraged for
|
||||
user-defined protocols. It is recommended to create compact protocols and
|
||||
combine them.
|
||||
implicit subtypes of "large" protocols. But, this doesn't apply to "built-in"
|
||||
protocols that are all "small" (i.e. have only few abstract methods).
|
||||
Also, such style is discouraged for user-defined protocols. It is recommended
|
||||
to create compact protocols and combine them.
|
||||
|
||||
|
||||
Make protocols interoperable with other approaches
|
||||
|
|
@ -1103,7 +1117,7 @@ Another potentially problematic case is assignment of attributes
|
|||
self.x = 0
|
||||
|
||||
c = C()
|
||||
isinstance(c1, P) # False
|
||||
isinstance(c, P) # False
|
||||
c.initialize()
|
||||
isinstance(c, P) # True
|
||||
|
||||
|
|
@ -1149,7 +1163,7 @@ This was rejected for the following reasons:
|
|||
ABCs from ``typing`` module. If we prohibit explicit subclassing of these
|
||||
ABCs, then quite a lot of code will break.
|
||||
|
||||
* Convenience: There are existing protocol-like ABCs (that will be turned
|
||||
* Convenience: There are existing protocol-like ABCs (that may be turned
|
||||
into protocols) that have many useful "mix-in" (non-abstract) methods.
|
||||
For example in the case of ``Sequence`` one only needs to implement
|
||||
``__getitem__`` and ``__len__`` in an explicit subclass, and one gets
|
||||
|
|
@ -1301,33 +1315,16 @@ confusions.
|
|||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
This PEP is almost fully backwards compatible. Few collection classes such as
|
||||
``Sequence`` and ``Mapping`` will be turned into runtime protocols, therefore
|
||||
results of ``isinstance()`` checks are going to change in some edge cases.
|
||||
For example, a class that implements the ``Sequence`` protocol but does not
|
||||
explicitly inherit from ``Sequence`` currently returns ``False`` in
|
||||
corresponding instance and class checks. With this PEP implemented, such
|
||||
checks will return ``True``.
|
||||
This PEP is fully backwards compatible.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
A working implementation of this PEP for ``mypy`` type checker is found on
|
||||
GitHub repo at https://github.com/ilevkivskyi/mypy/tree/protocols,
|
||||
corresponding ``typeshed`` stubs for more flavor are found at
|
||||
https://github.com/ilevkivskyi/typeshed/tree/protocols. Installation steps::
|
||||
|
||||
git clone --recurse-submodules https://github.com/ilevkivskyi/mypy/
|
||||
cd mypy && git checkout protocols && cd typeshed
|
||||
git remote add proto https://github.com/ilevkivskyi/typeshed
|
||||
git fetch proto && git checkout proto/protocols
|
||||
cd .. && git add typeshed && sudo python3 -m pip install -U .
|
||||
|
||||
The runtime implementation of protocols in ``typing`` module is
|
||||
found at https://github.com/ilevkivskyi/typehinting/tree/protocols.
|
||||
The version of ``collections.abc`` with structural behavior for mappings and
|
||||
sequences is found at https://github.com/ilevkivskyi/cpython/tree/protocols.
|
||||
The ``mypy`` type checker fully supports protocols (modulo a few
|
||||
known bugs). This includes treating all the builtin protocols, such as
|
||||
``Iterable`` structurally. The runtime implementation of protocols is
|
||||
available in ``typing_extensions`` module on PyPI.
|
||||
|
||||
|
||||
References
|
||||
|
|
|
|||
Loading…
Reference in New Issue