Add PEP-0604 to describes an extension to Python language, which aims to add a complementary (#1171)
syntax to write ``Union[X,Y]`` and ``Optional[X]`` easier.
This commit is contained in:
Philippe PRADOS
Chris Angelico
parent
a84501956f
commit
4aa98027b3
|
|
@ -0,0 +1,320 @@
|
|||
PEP: 604
|
||||
Title: Complementary syntax for ``Union[]`` and ``Optional[]``
|
||||
Author: Philippe PRADOS <python@prados.fr>
|
||||
Sponsor: Chris Angelico <rosuav@gmail.com>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 28-Aug-2019
|
||||
Python-Version: 3.9
|
||||
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
This PEP describes an extension to Python language, which aims to add a complementary
|
||||
syntax to write ``Union[X,Y]`` and ``Optional[X]`` easier.
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
||||
The PEP484 [1]_ and PEP526 [2]_ propose a generic syntax to add typing to variables,
|
||||
parameters and function returns.
|
||||
The PEP585 [3]_ proposes to `expose parameters to generics at runtime <https://www.python.org/dev/peps/pep-0585/#id7>`_.
|
||||
MyPy [4]_ accepts a syntax which looks like something like this:
|
||||
|
||||
::
|
||||
|
||||
annotation: name_type
|
||||
name_type: NAME (args)?
|
||||
args: '[' paramslist ']'
|
||||
paramslist: annotation (',' annotation)* [',']
|
||||
|
||||
- To describe a disjunction, the user must use ``Union[X,Y]``.
|
||||
|
||||
- To describe an optional value, the user must use ``Optional[X]``.
|
||||
|
||||
The verbosity of this syntax does not help the adoption.
|
||||
|
||||
Proposal
|
||||
========
|
||||
|
||||
Inspired by Scala language [5]_, this proposal adds two operators in the root ``type``:
|
||||
|
||||
Strong proposition
|
||||
------------------
|
||||
Add operator ``__or__()`` in the root ``type``.
|
||||
|
||||
With this new operator, it is possible to write ``int | str`` in place of ``Union[int,str]``.
|
||||
|
||||
This proposition uses the standard meaning of the ``|`` operator.
|
||||
|
||||
Optional proposition 1
|
||||
----------------------
|
||||
Add operator ``__invert__()`` in the root ``type``.
|
||||
|
||||
With this new operator, it is possible to write ``~int`` in place of ``Optional[int]``.
|
||||
|
||||
This proposition uses this operator because it is present in the language and it's conform to the
|
||||
`usage of tilde <https://www.thecut.com/article/why-the-internet-tilde-is-our-most-perfect-tool-for-snark.html>`_
|
||||
|
||||
So, the new syntax for annotations will be:
|
||||
|
||||
::
|
||||
|
||||
annotation: ( name_type | or_type | invert_type )
|
||||
or_type: name_type '|' annotation
|
||||
invert_type: '~' annotation
|
||||
name_type: NAME (args)?
|
||||
args: '[' paramslist ']'
|
||||
paramslist: annotation (',' annotation)* [',']
|
||||
|
||||
Optional proposition 2
|
||||
----------------------
|
||||
Then, it is possible to extend ``isinstance()`` and ``issubclass()``
|
||||
to accept this new syntax:
|
||||
|
||||
::
|
||||
|
||||
isinstance(int, int | str)
|
||||
|
||||
Examples
|
||||
========
|
||||
|
||||
Here are some examples of what we can do with this feature.
|
||||
|
||||
::
|
||||
|
||||
# in place of
|
||||
# def f(list: List[Union[int, str]], param: Optional[int]) -> Union[float, str]
|
||||
def f(list: List[int | str], param: ~int) -> float | str:
|
||||
pass
|
||||
|
||||
f([1,"abc"],None)
|
||||
|
||||
assert str | int == Union[str,int]
|
||||
assert str | int | float == Union[str, int, float]
|
||||
assert ~str == Optional[str]
|
||||
|
||||
assert isinstance("", int | str)
|
||||
assert issubclass(int, int | str)
|
||||
|
||||
Once the Python language is extended, MyPy [3]_ must be updated to accept this new syntax.
|
||||
|
||||
Incompatible changes
|
||||
====================
|
||||
In some situations, some exceptions will not be raised as expected.
|
||||
|
||||
|
||||
Dissenting Opinion
|
||||
==================
|
||||
|
||||
- `Discussion in python-ideas <https://mail.python.org/archives/list/python-ideas@python.org/thread/FCTXGDT2NNKRJQ6CDEPWUXHVG2AAQZZY/>`_
|
||||
|
||||
1. Add a new operator for ``Union[type1|type2]``?
|
||||
--------------------------------------------------
|
||||
|
||||
- CONS: This is not a new proposal. If I recall correctly, it was proposed way back at the very beginning of the
|
||||
type-hinting discussion, and there has been at least one closed feature request for it:
|
||||
`Issue 387 <https://github.com/python/typing/issues/387>`_
|
||||
|
||||
- It is maybe too late to change this, many people are already get used to current notation.
|
||||
|
||||
- *This PEP propose to add a new notation, not to replace the notation*
|
||||
|
||||
- This syntax is difficult to google, if someone encounters it in code
|
||||
- It is still not possible to use ``|`` for unions because of built-in types. (This would require a corresponding
|
||||
slot in type which is a non-starter)
|
||||
|
||||
- *The proposed implementation do it*
|
||||
|
||||
- There are currently no volunteer to implement this in mypy
|
||||
|
||||
- *An implementation is proposed now (One patch for CPython and one for MyPy).*
|
||||
|
||||
- "but as @ilevkivskyi pointed out, that is not an option (at least until Python 4)."
|
||||
|
||||
- *Is it time now ?*
|
||||
|
||||
- PRO: It's similar of Scala
|
||||
- PRO: Seems like ``foo | None`` is just as readable
|
||||
- PRO: Which means you couldn't use this feature in Python 3.7, much less 2.7. I'm not sure it maintaining backward
|
||||
compatibility in typing and in mypy is still as important today as it was 5 years ago, but I'm pretty sure it hasn't
|
||||
been abandoned entirely.
|
||||
- CONS: add operator introducing a dependency to typing in builtins
|
||||
- CONS: supporting this would likely break compatibility with existing code that overloads ``|`` for class objects
|
||||
using a metaclass. We could perhaps work around this by making ``|`` inside an annotation context different from
|
||||
the regular ``|`` operator.
|
||||
|
||||
- *A work around is to use* ``Union[type1,type2]`` *in this case*
|
||||
|
||||
- CONS: You need ``__ror__`` as well as ``__or__``
|
||||
- *No, in this situation, Python auto invoke ``__or__`` in case of ``__ror__``.*
|
||||
- CONS: as breaking the backport (in that ``typing.py`` can easily be backported but core ``types`` can't)
|
||||
|
||||
- There are several things in the typing syntax that require a certain minimum version. E.g. type annotations require
|
||||
Python 3 (whereas type comments work in Python 2 too), type annotations on variables (PEP 526) require 3.6+,
|
||||
``from __future__ import annotations`` (PEP 563) requires 3.7+.
|
||||
|
||||
- PRO: I mean that at run-time, ``int|str`` might return a very simple object in 3.9, rather than everything that
|
||||
you'd need to grab from importing ``typing``. Wondering if doing so would close off the possibility of, in 3.12 or
|
||||
something, making it a more directly usable "type union" that has other value.
|
||||
- CONS: if Python itself doesn't have to be changed, we'd still need to implement it in mypy, Pyre, PyCharm,
|
||||
Pytype, and who knows what else.
|
||||
|
||||
- *A proposed patch of mypy is just 20 lines of codes*
|
||||
|
||||
- If yes,
|
||||
|
||||
Change only the PEP484 (Type hints) to accept the syntax ``type1 | type2`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- PRO: The PEP563 [6]_ (Postponed Evaluation of Annotations) is enough to accept this proposition
|
||||
- CONS: The Resolving type hints at runtime says: “For code which uses annotations for other purposes, a
|
||||
regular ``eval(ann, globals, locals)`` call is enough to resolve the annotation.". Without add a new
|
||||
operator ``__or__`` in type ``type``, it's not possible to resolve type hints at runtime.
|
||||
|
||||
::
|
||||
|
||||
>>> from __future__ import annotations
|
||||
>>> def foo() -> int | str: pass
|
||||
...
|
||||
>>> eval(foo.__annotations__['return'])
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "<string>", line 1, in <module>
|
||||
TypeError: unsupported operand type(s) for |: 'type' and 'type'
|
||||
|
||||
- CONS: Without operator, it's not possible to write
|
||||
|
||||
::
|
||||
|
||||
>>> u = int | str
|
||||
>>> u
|
||||
typing.Union[int, str]
|
||||
|
||||
Use ``(int, str)`` in place of ``Union[int,str]`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
- PRO: This doesn't have compatibility issues and it's similar to ``isinstance(foo, (int, str))``
|
||||
- PRO: Either better than breaking backward compatibility by adding new operator methods to the type ``type``.
|
||||
- CONS: In most languages with similar-ish type syntax, ``(int, str)`` means ``Tuple[int, str]``,
|
||||
not ``Union[int, str]``.
|
||||
|
||||
Use ``{int, str}`` in place of ``Union[int,str]`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
- PRO: big advantage of ``{int, str}`` over ``int|str``. It doesn't require adding anything to ``type``,
|
||||
and we don't need to introduce a new lightweight builtin union type.
|
||||
|
||||
2. Add a new operator for ``Optional[type]`` ?
|
||||
----------------------------------------------
|
||||
|
||||
- CONS: ``foo | None`` is short and readable
|
||||
- CONS: ``foo | None`` it's 3 fewer characters than ``Optional[foo]``, or 30 fewer if you include the full
|
||||
removal of ``from typing import Optional``. the additional gain of ``~foo`` is only 6 characters.
|
||||
- PRO: help the readability, with a lot of parameters:
|
||||
|
||||
::
|
||||
|
||||
def f(source: str | None, destination: str | None, param: int | None):...
|
||||
def f(source: ~str, destination: ~str, param: ~int):...
|
||||
|
||||
- PRO: I'm currently working on annotating a very large codebase, and ``Optional[T]`` is so frequent that I
|
||||
think ``T | None`` would not be enough of an improvement.
|
||||
- PRO: Adding a default ``__or__`` overload to ``type`` seems a reasonable price to pay in 3.9, and
|
||||
ditto for ``__invert__``. Type checkers can support this in older Python versions using PEP 563 or in type
|
||||
comments or in "forward references" (types hidden in string literals).
|
||||
- CONS: The ``~`` is easy to be missed (at least by human readers) and the meaning not obvious.
|
||||
- PRO: Also, Python's typing system is a lot easier to grasp if you're familiar with an established modern-typed
|
||||
language (Swift, Scala, Haskell, F#, etc.), and they also use ``Optional[T]`` (or ``optional<T>`` or ``Maybe t``
|
||||
or some other spelling of the same idea) all over be place—so often that many of them have added shortcuts
|
||||
like ``T?`` to make it easier to write and less intrusive to read.
|
||||
|
||||
- if yes,
|
||||
|
||||
Add operator ``__revert__`` in type type to use syntax like ``~int`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
- CONS: ``~`` is not automatically readable
|
||||
|
||||
- *like ``:`` to separate variable and typing.*
|
||||
|
||||
- CONS: ``~`` means complement, which is a completely different thing from ``|None``. ``~int`` seems like it
|
||||
would actually harm comprehension instead of helping.
|
||||
- PRO: the slight abuse of ``~int`` meaning "maybe int" is pretty plausible (consider how "approximately equal"
|
||||
is written mathematically).
|
||||
- PRO: `Possibly relevant for tilde <https://www.thecut.com/article/why-the-internet-tilde-is-our-most-perfect-tool-for-snark.html>`_
|
||||
- CONS: With ``~`` there probably won't be a confusion in that sense, but someone reading it for the first time will
|
||||
definitely need to look it up (which is fine i.m.o.).
|
||||
|
||||
- *Like the first time someone reading the annotation*
|
||||
|
||||
::
|
||||
|
||||
def f(a=int):...
|
||||
def f(a:int):...
|
||||
|
||||
Add operator ``__add__`` in type type to use syntax like ``+int`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
- PRO: ``+foo`` definitely seems to say "foo, plus something else" to me much more than ``~foo``.
|
||||
- CONS: ``+foo`` is less intuitive than ``~foo`` for ``Optional``
|
||||
|
||||
Like Kotlin, add a new ``?`` operator to use syntax like ``int?`` or ``?int`` ?
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- CONS: It's not compatible with IPython and Jupyter Lab ``?smth`` displays help for symbol ``smth``
|
||||
- CONS: With default arguments, ``?=`` looks... not great
|
||||
|
||||
::
|
||||
|
||||
def f(source: str?=def_src, destination: str?=MISSING, param: int?=1): ...
|
||||
|
||||
3. Extend ``isinstance()`` and ``issubclass()`` to accept ``Union`` ?
|
||||
---------------------------------------------------------------------
|
||||
|
||||
::
|
||||
|
||||
isinstance(x, str | int) ==> "is x an instance of str or int"
|
||||
|
||||
- PRO: if they were permitted, then instance checks could use an extremely clean-looking notation for "any of these":
|
||||
- PRO: The implementation can use the tuple present in ``Union`` parameter, without create a new instance.
|
||||
- CONS: Why not accept this syntax in ``except`` ?
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
A proposed implementation for `cpython is here
|
||||
<https://github.com/pprados/cpython/tree/updage_isinstance>`_.
|
||||
A proposed implementation for `mypy is here
|
||||
<https://github.com/pprados/mypy/tree/add_INVERT_to_types>`_.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [1] PEP484,
|
||||
https://www.python.org/dev/peps/pep-0484/
|
||||
.. [2] PEP526,
|
||||
https://www.python.org/dev/peps/pep-0526/
|
||||
.. [3] PEP585,
|
||||
https://www.python.org/dev/peps/pep-0585/
|
||||
.. [4] MyPy
|
||||
http://mypy-lang.org/
|
||||
.. [5] Scala Union Types
|
||||
https://dotty.epfl.ch/docs/reference/new-types/union-types.html
|
||||
.. [6] PEP563,
|
||||
https://www.python.org/dev/peps/pep-0563/
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
|
||||
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
sentence-end-double-space: t
|
||||
fill-column: 70
|
||||
coding: utf-8
|
||||
End:
|
||||
Loading…
Reference in New Issue