python-peps/pep-0604.rst

PEP: 604
Title: Complementary syntax for ``Union[]`` and ``Optional[]``
Author: Philippe PRADOS <python@prados.fr>
Sponsor: Chris Angelico <rosuav@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Aug-2019
Python-Version: 3.9


Introduction
============

This PEP describes an extension to Python language, which aims to add a complementary
syntax to write ``Union[X,Y]`` easier.


Motivation
==========

The PEP484 [1]_ and PEP526 [2]_ propose a generic syntax to add typing to variables,
parameters and function returns.
The PEP585 [3]_ proposes to `expose parameters to generics at runtime <https://www.python.org/dev/peps/pep-0585/#id7>`_.
MyPy [4]_ accepts a syntax which looks like something like this:

::

    annotation: name_type
    name_type: NAME (args)?
    args: '[' paramslist ']'
    paramslist: annotation (',' annotation)* [',']

- To describe a disjunction, the user must use ``Union[X,Y]``.

The verbosity of this syntax does not help the adoption.

Proposal
========

Inspired by Scala language [5]_, this proposal adds operator ``__or__()`` in the root ``type``.
With this new operator, it is possible to write ``int | str`` in place of ``Union[int,str]``.
This proposition uses the standard meaning of the ``|`` operator.
Then, it is possible to extend ``isinstance()`` and ``issubclass()``
to accept this new syntax:

::

  isinstance(int, int | str)

Examples
========

Here are some examples of what we can do with this feature.

::

    # in place of
    # def f(list: List[Union[int, str]], param: Optional[int]) -> Union[float, str]
    def f(list: List[int | str], param: int | None) -> float | str:
        pass

    f([1,"abc"],None)

    assert str | int == Union[str,int]
    assert str | int | float == Union[str, int, float]

    assert isinstance("", int | str)
    assert issubclass(int, int | str)

Once the Python language is extended, MyPy [3]_ must be updated to accept this new syntax.

Incompatible changes
====================
In some situations, some exceptions will not be raised as expected.

If some metaclass overload the ``__or__`` operator, the user must resolve the ambiguities with ``Union``.
::

    >>> class M(type):
    ... 	def __or__(self,other): return "Hello"
    ...
    >>> class C(metaclass=M):pass
    ...
    >>> C | int
    'Hello'
    >>> int | C
    typing.Union[int, __main__.C]
    >>> Union[C,int]
    typing.Union[__main__.C, int]

Dissenting Opinion
==================

- `Discussion in python-ideas <https://mail.python.org/archives/list/python-ideas@python.org/thread/FCTXGDT2NNKRJQ6CDEPWUXHVG2AAQZZY/>`_
- `Discussion in typing-sig <https://mail.python.org/archives/list/typing-sig@python.org/thread/D5HCB4NT4S3WSK33WI26WZSFEXCEMNHN/>`_

1. Add a new operator for ``Union[type1|type2]``?
--------------------------------------------------

- CONS: This is not a new proposal. If I recall correctly, it was proposed way back at the very beginning of the
  type-hinting discussion, and there has been at least one closed feature request for it:
  `Issue 387 <https://github.com/python/typing/issues/387>`_

  - It is maybe too late to change this, many people are already get used to current notation.

    - *This PEP propose to add a new notation, not to replace the notation*

  - This syntax is difficult to google, if someone encounters it in code
  - It is still not possible to use ``|`` for unions because of built-in types. (This would require a corresponding
    slot in type which is a non-starter)

    - *The proposed implementation do it*

  - There are currently no volunteer to implement this in mypy

    - *An implementation is proposed now (One patch for CPython and one for MyPy).*

  - "but as @ilevkivskyi pointed out, that is not an option (at least until Python 4)."

    - *Is it time now ?*

- PRO: It's similar of Scala
- PRO: Seems like ``foo | None`` is just as readable
- PRO: Which means you couldn't use this feature in Python 3.7, much less 2.7. I'm not sure it maintaining backward
  compatibility in typing and in mypy is still as important today as it was 5 years ago, but I'm pretty sure it hasn't
  been abandoned entirely.
- CONS: add operator introducing a dependency to typing in builtins
- CONS:  supporting this would likely break compatibility with existing code that overloads ``|`` for class objects
  using a metaclass. We could perhaps work around this by making ``|`` inside an annotation context different from
  the regular ``|`` operator.

  - *A work around is to use* ``Union[type1,type2]`` *in this case*

- CONS: You need ``__ror__`` as well as ``__or__``
  - *No, in this situation, Python auto invoke ``__or__`` in case of ``__ror__``.*
- CONS: as breaking the backport (in that ``typing.py`` can easily be backported but core ``types`` can't)

  - There are several things in the typing syntax that require a certain minimum version. E.g. type annotations require
    Python 3 (whereas type comments work in Python 2 too), type annotations on variables (PEP 526) require 3.6+,
    ``from __future__ import annotations`` (PEP 563) requires 3.7+.

- PRO: I mean that at run-time, ``int|str`` might return a very simple object in 3.9, rather than everything that
  you'd need to grab from importing ``typing``. Wondering if doing so would close off the possibility of, in 3.12 or
  something, making it a more directly usable "type union" that has other value.
- CONS: if Python itself doesn't have to be changed, we'd still need to implement it in mypy, Pyre, PyCharm,
  Pytype, and who knows what else.

  - *A proposed patch of mypy is just 20 lines of codes*

- If yes,

Change only the PEP484 (Type hints) to accept the syntax ``type1 | type2`` ?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- PRO: The PEP563 [6]_ (Postponed Evaluation of Annotations) is enough to accept this proposition
- CONS: The Resolving type hints at runtime says: “For code which uses annotations for other purposes, a
  regular ``eval(ann, globals, locals)`` call is enough to resolve the annotation.". Without add a new
  operator ``__or__`` in type ``type``, it's not possible to resolve type hints at runtime.

::

    >>> from __future__ import annotations
    >>> def foo() -> int | str: pass
    ...
    >>> eval(foo.__annotations__['return'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1, in <module>
    TypeError: unsupported operand type(s) for |: 'type' and 'type'

  - CONS: Without operator, it's not possible to write

::

    >>> u = int | str
    >>> u
    typing.Union[int, str]

Use ``(int, str)`` in place of ``Union[int,str]`` ?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- PRO: This doesn't have compatibility issues and it's similar to ``isinstance(foo, (int, str))``
- PRO: Either better than breaking backward compatibility by adding new operator methods to the type ``type``.
- CONS: In most languages with similar-ish type syntax, ``(int, str)`` means ``Tuple[int, str]``,
  not ``Union[int, str]``.

Use ``{int, str}`` in place of ``Union[int,str]`` ?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- PRO: big advantage of ``{int, str}`` over ``int|str``. It doesn't require adding anything to ``type``,
  and we don't need to introduce a new lightweight builtin union type.

2. Extend ``isinstance()`` and ``issubclass()`` to accept ``Union`` ?
---------------------------------------------------------------------

::

    isinstance(x, str | int) ==> "is x an instance of str or int"

- PRO: if they were permitted, then instance checks could use an extremely clean-looking notation for "any of these":
- PRO: The implementation can use the tuple present in ``Union`` parameter, without create a new instance.

Reference Implementation
========================

A proposed implementation for `cpython is here
<https://github.com/pprados/cpython/tree/update_isinstance>`_.
A proposed implementation for `mypy is here
<https://github.com/pprados/mypy/tree/add_INVERT_to_types>`_.


References
==========

.. [1] PEP484,
   https://www.python.org/dev/peps/pep-0484/
.. [2] PEP526,
   https://www.python.org/dev/peps/pep-0526/
.. [3] PEP585,
   https://www.python.org/dev/peps/pep-0585/
.. [4] MyPy
   http://mypy-lang.org/
.. [5] Scala Union Types
   https://dotty.epfl.ch/docs/reference/new-types/union-types.html
.. [6] PEP563,
   https://www.python.org/dev/peps/pep-0563/

Copyright
=========

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: