From 4205604fc84a20c29cee1b696501a2350aa6134f Mon Sep 17 00:00:00 2001 From: Ivan Levkivskyi Date: Wed, 24 May 2017 22:38:49 +0200 Subject: [PATCH] Updates to PEP 544: Protocols (#255) * Add covariant mutable overriding and overriding variance to rejected ideas * Update the notes on runtime implementation * Add one more argument for prohibiting variance overrides --- pep-0544.txt | 117 +++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 103 insertions(+), 14 deletions(-) diff --git a/pep-0544.txt b/pep-0544.txt index b70949217..028a3107b 100644 --- a/pep-0544.txt +++ b/pep-0544.txt @@ -507,24 +507,29 @@ non-protocol generic types:: ``Protocol[T, S, ...]`` is allowed as a shorthand for ``Protocol, Generic[T, S, ...]``. -Declaring variance is not necessary for protocol classes, since it can be -inferred from a protocol definition. Examples:: +User-defined generic protocols support explicitly declared variance. +Type checkers will warn if the inferred variance is different from +the declared variance. Examples:: - class Box(Protocol[T]): - def content(self) -> T: + T = TypeVar('T') + T_co = TypeVar('T_co', covariant=True) + T_contra = TypeVar('T_contra', contravariant=True) + + class Box(Protocol[T_co]): + def content(self) -> T_co: ... box: Box[float] second_box: Box[int] - box = second_box # This is OK due to the inferred covariance of 'Box'. + box = second_box # This is OK due to the covariance of 'Box'. - class Sender(Protocol[T]): - def send(self, data: T) -> int: + class Sender(Protocol[T_contra]): + def send(self, data: T_contra) -> int: ... sender: Sender[float] new_sender: Sender[int] - new_sender = sender # OK, type checker finds that 'Sender' is contravariant. + new_sender = sender # OK, 'Sender' is contravariant. class Proto(Protocol[T]): attr: T # this class is invariant, since it has a mutable attribute @@ -533,6 +538,16 @@ inferred from a protocol definition. Examples:: another_var: Proto[int] var = another_var # Error! 'Proto[float]' is incompatible with 'Proto[int]'. +Note that unlike nominal classes, de-facto covariant protocols cannot be +declared as invariant, since this can break transitivity of subtyping +(see `rejected`_ ideas for details). For example:: + + T = TypeVar('T') + + class AnotherBox(Protocol[T]): # Error, this protocol is covariant in T, + def content(self) -> T: # not invariant. + ... + Recursive protocols ------------------- @@ -562,7 +577,7 @@ Continuing the previous example:: def walk(graph: Traversable) -> None: ... - tree: Tree[float] = Tree(0, []) + tree: Tree[float] = Tree() walk(tree) # OK, 'Tree[float]' is a subtype of 'Traversable' @@ -771,17 +786,21 @@ Implementation details The runtime implementation could be done in pure Python without any effects on the core interpreter and standard library except in the -``typing`` module: +``typing`` module, and a minor update to ``collections.abc``: * Define class ``typing.Protocol`` similar to ``typing.Generic``. * Implement metaclass functionality to detect whether a class is - a protocol or not. Add a class attribute ``__protocol__ = True`` + a protocol or not. Add a class attribute ``_is_protocol = True`` if that is the case. Verify that a protocol class only has protocol base classes in the MRO (except for object). -* Implement ``@runtime`` that adds all attributes to ``__subclasshook__()``. +* Implement ``@runtime`` that allows ``__subclasshook__()`` performing + structural instance and subclass checks as in ``collections.abc`` classes. * All structural subtyping checks will be performed by static type checkers, such as ``mypy`` [mypy]_. No additional support for protocol validation will be provided at runtime. +* Classes ``Mapping``, ``MutableMapping``, ``Sequence``, and + ``MutableSequence`` in ``collections.abc`` module will support structural + instance and subclass checks (like e.g. ``collections.abc.Iterable``). Changes in the typing module @@ -879,8 +898,8 @@ reasons: Python runtime, which won't happen. -Allow protocols subclassing normal classes ------------------------------------------- +Protocols subclassing normal classes +------------------------------------ The main rationale to prohibit this is to preserve transitivity of subtyping, consider this example:: @@ -1118,6 +1137,74 @@ This was rejected for the following reasons: it has an unsafe override. +Covariant subtyping of mutable attributes +----------------------------------------- + +Rejected because covariant subtyping of mutable attributes is not safe. +Consider this example:: + + class P(Protocol): + x: float + + def f(arg: P) -> None: + arg.x = 0.42 + + class C: + x: int + + c = C() + f(c) # Would typecheck if covariant subtyping + # of mutable attributes were allowed + c.x >> 1 # But this fails at runtime + +It was initially proposed to allow this for practical reasons, but it was +subsequently rejected, since this may mask some hard to spot bugs. + + +Overriding inferred variance of protocol classes +------------------------------------------------ + +It was proposed to allow declaring protocols as invariant if they are actually +covariant or contravariant (as it is possible for nominal classes, see PEP 484). +However, it was decided not to do this because of several downsides: + +* Declared protocol invariance breaks transitivity of sub-typing. Consider + this situation:: + + T = TypeVar('T') + + class P(Protocol[T]): # Declared as invariant + def meth(self) -> T: + ... + class C: + def meth(self) -> float: + ... + class D(C): + def meth(self) -> int: + ... + + Now we have that ``D`` is a subtype of ``C``, and ``C`` is a subtype of + ``P[float]``. But ``D`` is *not* a subtype of ``P[float]`` since ``D`` + implements ``P[int]``, and ``P`` is invariant. There is a possibility + to "cure" this by looking for protocol implementations in MROs but this + will be too complex in a general case, and this "cure" requires abandoning + simple idea of purely structural subtyping for protocols. + +* Subtyping checks will always require type inference for protocols. In the + above example a user may complain: "Why did you infer ``P[int]`` for + my ``D``? It implements ``P[float]``!". Normally, inference can be overruled + by an explicit annotation, but here this will require explicit subclassing, + defeating the purpose of using protocols. + +* Allowing overriding variance will make impossible more detailed error + messages in type checkers citing particular conflicts in member + type signatures. + +* Finally, explicit is better than implicit in this case. Requiring user to + declare correct variance will simplify understanding the code and will avoid + unexpected errors at the point of use. + + Support adapters and adaptation ------------------------------- @@ -1179,6 +1266,8 @@ https://github.com/ilevkivskyi/typeshed/tree/protocols. Installation steps:: The runtime implementation of protocols in ``typing`` module is found at https://github.com/ilevkivskyi/typehinting/tree/protocols. +The version of ``collections.abc`` with structural behavior for mappings and +sequences is found at https://github.com/ilevkivskyi/cpython/tree/protocols. References