From 4aa98027b3b96b6eee0ee614665150b733b58869 Mon Sep 17 00:00:00 2001 From: Philippe PRADOS Date: Fri, 20 Sep 2019 09:47:02 +0200 Subject: [PATCH] Add PEP-0604 to describes an extension to Python language, which aims to add a complementary (#1171) syntax to write ``Union[X,Y]`` and ``Optional[X]`` easier. --- pep-0604.rst | 320 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 320 insertions(+) create mode 100644 pep-0604.rst diff --git a/pep-0604.rst b/pep-0604.rst new file mode 100644 index 000000000..01ebe22e2 --- /dev/null +++ b/pep-0604.rst @@ -0,0 +1,320 @@ +PEP: 604 +Title: Complementary syntax for ``Union[]`` and ``Optional[]`` +Author: Philippe PRADOS +Sponsor: Chris Angelico +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 28-Aug-2019 +Python-Version: 3.9 + + +Introduction +============ + +This PEP describes an extension to Python language, which aims to add a complementary +syntax to write ``Union[X,Y]`` and ``Optional[X]`` easier. + + +Motivation +========== + +The PEP484 [1]_ and PEP526 [2]_ propose a generic syntax to add typing to variables, +parameters and function returns. +The PEP585 [3]_ proposes to `expose parameters to generics at runtime `_. +MyPy [4]_ accepts a syntax which looks like something like this: + +:: + + annotation: name_type + name_type: NAME (args)? + args: '[' paramslist ']' + paramslist: annotation (',' annotation)* [','] + +- To describe a disjunction, the user must use ``Union[X,Y]``. + +- To describe an optional value, the user must use ``Optional[X]``. + +The verbosity of this syntax does not help the adoption. + +Proposal +======== + +Inspired by Scala language [5]_, this proposal adds two operators in the root ``type``: + +Strong proposition +------------------ +Add operator ``__or__()`` in the root ``type``. + +With this new operator, it is possible to write ``int | str`` in place of ``Union[int,str]``. + +This proposition uses the standard meaning of the ``|`` operator. + +Optional proposition 1 +---------------------- +Add operator ``__invert__()`` in the root ``type``. + +With this new operator, it is possible to write ``~int`` in place of ``Optional[int]``. + +This proposition uses this operator because it is present in the language and it's conform to the +`usage of tilde `_ + +So, the new syntax for annotations will be: + +:: + + annotation: ( name_type | or_type | invert_type ) + or_type: name_type '|' annotation + invert_type: '~' annotation + name_type: NAME (args)? + args: '[' paramslist ']' + paramslist: annotation (',' annotation)* [','] + +Optional proposition 2 +---------------------- +Then, it is possible to extend ``isinstance()`` and ``issubclass()`` +to accept this new syntax: + +:: + + isinstance(int, int | str) + +Examples +======== + +Here are some examples of what we can do with this feature. + +:: + + # in place of + # def f(list: List[Union[int, str]], param: Optional[int]) -> Union[float, str] + def f(list: List[int | str], param: ~int) -> float | str: + pass + + f([1,"abc"],None) + + assert str | int == Union[str,int] + assert str | int | float == Union[str, int, float] + assert ~str == Optional[str] + + assert isinstance("", int | str) + assert issubclass(int, int | str) + +Once the Python language is extended, MyPy [3]_ must be updated to accept this new syntax. + +Incompatible changes +==================== +In some situations, some exceptions will not be raised as expected. + + +Dissenting Opinion +================== + +- `Discussion in python-ideas `_ + +1. Add a new operator for ``Union[type1|type2]``? +-------------------------------------------------- + +- CONS: This is not a new proposal. If I recall correctly, it was proposed way back at the very beginning of the + type-hinting discussion, and there has been at least one closed feature request for it: + `Issue 387 `_ + + - It is maybe too late to change this, many people are already get used to current notation. + + - *This PEP propose to add a new notation, not to replace the notation* + + - This syntax is difficult to google, if someone encounters it in code + - It is still not possible to use ``|`` for unions because of built-in types. (This would require a corresponding + slot in type which is a non-starter) + + - *The proposed implementation do it* + + - There are currently no volunteer to implement this in mypy + + - *An implementation is proposed now (One patch for CPython and one for MyPy).* + + - "but as @ilevkivskyi pointed out, that is not an option (at least until Python 4)." + + - *Is it time now ?* + +- PRO: It's similar of Scala +- PRO: Seems like ``foo | None`` is just as readable +- PRO: Which means you couldn't use this feature in Python 3.7, much less 2.7. I'm not sure it maintaining backward + compatibility in typing and in mypy is still as important today as it was 5 years ago, but I'm pretty sure it hasn't + been abandoned entirely. +- CONS: add operator introducing a dependency to typing in builtins +- CONS: supporting this would likely break compatibility with existing code that overloads ``|`` for class objects + using a metaclass. We could perhaps work around this by making ``|`` inside an annotation context different from + the regular ``|`` operator. + + - *A work around is to use* ``Union[type1,type2]`` *in this case* + +- CONS: You need ``__ror__`` as well as ``__or__`` + - *No, in this situation, Python auto invoke ``__or__`` in case of ``__ror__``.* +- CONS: as breaking the backport (in that ``typing.py`` can easily be backported but core ``types`` can't) + + - There are several things in the typing syntax that require a certain minimum version. E.g. type annotations require + Python 3 (whereas type comments work in Python 2 too), type annotations on variables (PEP 526) require 3.6+, + ``from __future__ import annotations`` (PEP 563) requires 3.7+. + +- PRO: I mean that at run-time, ``int|str`` might return a very simple object in 3.9, rather than everything that + you'd need to grab from importing ``typing``. Wondering if doing so would close off the possibility of, in 3.12 or + something, making it a more directly usable "type union" that has other value. +- CONS: if Python itself doesn't have to be changed, we'd still need to implement it in mypy, Pyre, PyCharm, + Pytype, and who knows what else. + + - *A proposed patch of mypy is just 20 lines of codes* + +- If yes, + +Change only the PEP484 (Type hints) to accept the syntax ``type1 | type2`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- PRO: The PEP563 [6]_ (Postponed Evaluation of Annotations) is enough to accept this proposition +- CONS: The Resolving type hints at runtime says: “For code which uses annotations for other purposes, a + regular ``eval(ann, globals, locals)`` call is enough to resolve the annotation.". Without add a new + operator ``__or__`` in type ``type``, it's not possible to resolve type hints at runtime. + +:: + + >>> from __future__ import annotations + >>> def foo() -> int | str: pass + ... + >>> eval(foo.__annotations__['return']) + Traceback (most recent call last): + File "", line 1, in + File "", line 1, in + TypeError: unsupported operand type(s) for |: 'type' and 'type' + + - CONS: Without operator, it's not possible to write + +:: + + >>> u = int | str + >>> u + typing.Union[int, str] + +Use ``(int, str)`` in place of ``Union[int,str]`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +- PRO: This doesn't have compatibility issues and it's similar to ``isinstance(foo, (int, str))`` +- PRO: Either better than breaking backward compatibility by adding new operator methods to the type ``type``. +- CONS: In most languages with similar-ish type syntax, ``(int, str)`` means ``Tuple[int, str]``, + not ``Union[int, str]``. + +Use ``{int, str}`` in place of ``Union[int,str]`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +- PRO: big advantage of ``{int, str}`` over ``int|str``. It doesn't require adding anything to ``type``, + and we don't need to introduce a new lightweight builtin union type. + +2. Add a new operator for ``Optional[type]`` ? +---------------------------------------------- + +- CONS: ``foo | None`` is short and readable +- CONS: ``foo | None`` it's 3 fewer characters than ``Optional[foo]``, or 30 fewer if you include the full + removal of ``from typing import Optional``. the additional gain of ``~foo`` is only 6 characters. +- PRO: help the readability, with a lot of parameters: + +:: + + def f(source: str | None, destination: str | None, param: int | None):... + def f(source: ~str, destination: ~str, param: ~int):... + +- PRO: I'm currently working on annotating a very large codebase, and ``Optional[T]`` is so frequent that I + think ``T | None`` would not be enough of an improvement. +- PRO: Adding a default ``__or__`` overload to ``type`` seems a reasonable price to pay in 3.9, and + ditto for ``__invert__``. Type checkers can support this in older Python versions using PEP 563 or in type + comments or in "forward references" (types hidden in string literals). +- CONS: The ``~`` is easy to be missed (at least by human readers) and the meaning not obvious. +- PRO: Also, Python's typing system is a lot easier to grasp if you're familiar with an established modern-typed + language (Swift, Scala, Haskell, F#, etc.), and they also use ``Optional[T]`` (or ``optional`` or ``Maybe t`` + or some other spelling of the same idea) all over be place—so often that many of them have added shortcuts + like ``T?`` to make it easier to write and less intrusive to read. + +- if yes, + +Add operator ``__revert__`` in type type to use syntax like ``~int`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +- CONS: ``~`` is not automatically readable + + - *like ``:`` to separate variable and typing.* + +- CONS: ``~`` means complement, which is a completely different thing from ``|None``. ``~int`` seems like it + would actually harm comprehension instead of helping. +- PRO: the slight abuse of ``~int`` meaning "maybe int" is pretty plausible (consider how "approximately equal" + is written mathematically). +- PRO: `Possibly relevant for tilde `_ +- CONS: With ``~`` there probably won't be a confusion in that sense, but someone reading it for the first time will + definitely need to look it up (which is fine i.m.o.). + + - *Like the first time someone reading the annotation* + +:: + + def f(a=int):... + def f(a:int):... + +Add operator ``__add__`` in type type to use syntax like ``+int`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +- PRO: ``+foo`` definitely seems to say "foo, plus something else" to me much more than ``~foo``. +- CONS: ``+foo`` is less intuitive than ``~foo`` for ``Optional`` + +Like Kotlin, add a new ``?`` operator to use syntax like ``int?`` or ``?int`` ? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- CONS: It's not compatible with IPython and Jupyter Lab ``?smth`` displays help for symbol ``smth`` +- CONS: With default arguments, ``?=`` looks... not great + +:: + + def f(source: str?=def_src, destination: str?=MISSING, param: int?=1): ... + +3. Extend ``isinstance()`` and ``issubclass()`` to accept ``Union`` ? +--------------------------------------------------------------------- + +:: + + isinstance(x, str | int) ==> "is x an instance of str or int" + +- PRO: if they were permitted, then instance checks could use an extremely clean-looking notation for "any of these": +- PRO: The implementation can use the tuple present in ``Union`` parameter, without create a new instance. +- CONS: Why not accept this syntax in ``except`` ? + +Reference Implementation +======================== + +A proposed implementation for `cpython is here +`_. +A proposed implementation for `mypy is here +`_. + + +References +========== + +.. [1] PEP484, + https://www.python.org/dev/peps/pep-0484/ +.. [2] PEP526, + https://www.python.org/dev/peps/pep-0526/ +.. [3] PEP585, + https://www.python.org/dev/peps/pep-0585/ +.. [4] MyPy + http://mypy-lang.org/ +.. [5] Scala Union Types + https://dotty.epfl.ch/docs/reference/new-types/union-types.html +.. [6] PEP563, + https://www.python.org/dev/peps/pep-0563/ + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: