2021-02-26 00:34:11 -05:00
|
|
|
|
PEP: 655
|
|
|
|
|
Title: Marking individual TypedDict items as required or potentially-missing
|
|
|
|
|
Author: David Foster <david at dafoster.net>
|
|
|
|
|
Sponsor: Guido van Rossum <guido at python.org>
|
2022-02-21 18:47:19 -05:00
|
|
|
|
Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/53XVOD5ZUKJ263MWA6AUPEA6J7LBBLNV/
|
2024-02-16 01:27:36 -05:00
|
|
|
|
Status: Final
|
2021-02-26 00:34:11 -05:00
|
|
|
|
Type: Standards Track
|
2022-10-06 20:36:39 -04:00
|
|
|
|
Topic: Typing
|
2021-02-26 00:34:11 -05:00
|
|
|
|
Created: 30-Jan-2021
|
2021-10-08 17:24:19 -04:00
|
|
|
|
Python-Version: 3.11
|
2022-01-28 12:11:29 -05:00
|
|
|
|
Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021, 17-Jan-2022, 28-Jan-2022
|
2022-03-22 11:49:37 -04:00
|
|
|
|
Resolution: https://mail.python.org/archives/list/python-dev@python.org/message/AJEDNVC3FXM5QXNNW5CR4UCT4KI5XVUE/
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. canonical-typing-spec:: :ref:`typing:required-notrequired`
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Abstract
|
|
|
|
|
========
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
:pep:`589` defines notation
|
|
|
|
|
for declaring a TypedDict with all required keys and notation for defining
|
|
|
|
|
a TypedDict with :pep:`all potentially-missing keys <589#totality>`, however it
|
|
|
|
|
does not provide a mechanism to declare some keys as required and others
|
|
|
|
|
as potentially-missing. This PEP introduces two new notations:
|
|
|
|
|
``Required[]``, which can be used on individual items of a
|
2021-02-26 00:34:11 -05:00
|
|
|
|
TypedDict to mark them as required, and
|
2022-03-09 12:57:04 -05:00
|
|
|
|
``NotRequired[]``, which can be used on individual items
|
2021-02-26 00:34:11 -05:00
|
|
|
|
to mark them as potentially-missing.
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
This PEP makes no Python grammar changes. Correct usage
|
|
|
|
|
of required and potentially-missing keys of TypedDicts is intended to be
|
|
|
|
|
enforced only by static type checkers and need not be enforced by
|
|
|
|
|
Python itself at runtime.
|
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Motivation
|
|
|
|
|
==========
|
|
|
|
|
|
|
|
|
|
It is not uncommon to want to define a TypedDict with some keys that are
|
|
|
|
|
required and others that are potentially-missing. Currently the only way
|
|
|
|
|
to define such a TypedDict is to declare one TypedDict with one value
|
|
|
|
|
for ``total`` and then inherit it from another TypedDict with a
|
|
|
|
|
different value for ``total``:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class _MovieBase(TypedDict): # implicitly total=True
|
|
|
|
|
title: str
|
|
|
|
|
|
|
|
|
|
class Movie(_MovieBase, total=False):
|
|
|
|
|
year: int
|
|
|
|
|
|
|
|
|
|
Having to declare two different TypedDict types for this purpose is
|
|
|
|
|
cumbersome.
|
|
|
|
|
|
2022-02-14 22:26:57 -05:00
|
|
|
|
This PEP introduces two new type qualifiers, ``typing.Required`` and
|
|
|
|
|
``typing.NotRequired``, which allow defining a *single* TypedDict with
|
|
|
|
|
a mix of both required and potentially-missing keys:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[int]
|
|
|
|
|
|
2022-02-25 10:21:02 -05:00
|
|
|
|
This PEP also makes it possible to define TypedDicts in the
|
|
|
|
|
:pep:`alternative functional syntax <589#alternative-syntax>`
|
|
|
|
|
with a mix of required and potentially-missing keys,
|
|
|
|
|
which is not currently possible at all because the alternative syntax does
|
|
|
|
|
not support inheritance:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
Actor = TypedDict('Actor', {
|
|
|
|
|
'name': str,
|
|
|
|
|
# "in" is a keyword, so the functional syntax is necessary
|
|
|
|
|
'in': NotRequired[List[str]],
|
|
|
|
|
})
|
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Rationale
|
|
|
|
|
=========
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
One might think it unusual to propose notation that prioritizes marking
|
2022-02-25 10:21:02 -05:00
|
|
|
|
*required* keys rather than *potentially-missing* keys, as is
|
2021-02-26 00:34:11 -05:00
|
|
|
|
customary in other languages like TypeScript:
|
|
|
|
|
|
2022-02-19 17:25:43 -05:00
|
|
|
|
.. code-block:: typescript
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
interface Movie {
|
|
|
|
|
title: string;
|
|
|
|
|
year?: number; // ? marks potentially-missing keys
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
The difficulty is that the best word for marking a potentially-missing
|
2022-01-20 11:18:41 -05:00
|
|
|
|
key, ``Optional[]``, is already used in Python for a completely
|
2021-02-26 00:34:11 -05:00
|
|
|
|
different purpose: marking values that could be either of a particular
|
|
|
|
|
type or ``None``. In particular the following does not work:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
...
|
|
|
|
|
year: Optional[int] # means int|None, not potentially-missing!
|
|
|
|
|
|
|
|
|
|
Attempting to use any synonym of “optional” to mark potentially-missing
|
2022-01-20 11:18:41 -05:00
|
|
|
|
keys (like ``Missing[]``) would be too similar to ``Optional[]``
|
2021-02-26 00:34:11 -05:00
|
|
|
|
and be easy to confuse with it.
|
|
|
|
|
|
|
|
|
|
Thus it was decided to focus on positive-form phrasing for required keys
|
2022-01-20 11:18:41 -05:00
|
|
|
|
instead, which is straightforward to spell as ``Required[]``.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Nevertheless it is common for folks wanting to extend a regular
|
|
|
|
|
(``total=True``) TypedDict to only want to add a small number of
|
|
|
|
|
potentially-missing keys, which necessitates a way to mark keys that are
|
|
|
|
|
*not* required and potentially-missing, and so we also allow the
|
2022-01-20 11:18:41 -05:00
|
|
|
|
``NotRequired[]`` form for that case.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Specification
|
|
|
|
|
=============
|
|
|
|
|
|
|
|
|
|
The ``typing.Required`` type qualifier is used to indicate that a
|
|
|
|
|
variable declared in a TypedDict definition is a required key:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict, total=False):
|
|
|
|
|
title: Required[str]
|
|
|
|
|
year: int
|
|
|
|
|
|
|
|
|
|
Additionally the ``typing.NotRequired`` type qualifier is used to
|
|
|
|
|
indicate that a variable declared in a TypedDict definition is a
|
|
|
|
|
potentially-missing key:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict): # implicitly total=True
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[int]
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
It is an error to use ``Required[]`` or ``NotRequired[]`` in any
|
2021-02-26 00:34:11 -05:00
|
|
|
|
location that is not an item of a TypedDict.
|
2022-03-09 12:57:04 -05:00
|
|
|
|
Type checkers must enforce this restriction.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
It is valid to use ``Required[]`` and ``NotRequired[]`` even for
|
2021-02-26 00:34:11 -05:00
|
|
|
|
items where it is redundant, to enable additional explicitness if desired:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: Required[str] # redundant
|
|
|
|
|
year: NotRequired[int]
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
It is an error to use both ``Required[]`` and ``NotRequired[]`` at the
|
|
|
|
|
same time:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[Required[int]] # ERROR
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
Type checkers must enforce this restriction.
|
|
|
|
|
The runtime implementations of ``Required[]`` and ``NotRequired[]``
|
|
|
|
|
may also enforce this restriction.
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
2022-02-25 10:21:02 -05:00
|
|
|
|
The :pep:`alternative functional syntax <589#alternative-syntax>`
|
2022-01-20 11:18:41 -05:00
|
|
|
|
for TypedDict also supports
|
|
|
|
|
``Required[]`` and ``NotRequired[]``:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]})
|
|
|
|
|
|
|
|
|
|
|
2022-02-14 22:26:57 -05:00
|
|
|
|
Interaction with ``total=False``
|
|
|
|
|
--------------------------------
|
|
|
|
|
|
|
|
|
|
Any :pep:`589`-style TypedDict declared with ``total=False`` is equivalent
|
|
|
|
|
to a TypedDict with an implicit ``total=True`` definition with all of its
|
|
|
|
|
keys marked as ``NotRequired[]``.
|
|
|
|
|
|
|
|
|
|
Therefore:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class _MovieBase(TypedDict): # implicitly total=True
|
|
|
|
|
title: str
|
|
|
|
|
|
|
|
|
|
class Movie(_MovieBase, total=False):
|
|
|
|
|
year: int
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
is equivalent to:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class _MovieBase(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
|
|
|
|
|
class Movie(_MovieBase):
|
|
|
|
|
year: NotRequired[int]
|
|
|
|
|
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
Interaction with ``Annotated[]``
|
|
|
|
|
-----------------------------------
|
|
|
|
|
|
|
|
|
|
``Required[]`` and ``NotRequired[]`` can be used with ``Annotated[]``,
|
|
|
|
|
in any nesting order:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[Annotated[int, ValueRange(-9999, 9999)]] # ok
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: Annotated[NotRequired[int], ValueRange(-9999, 9999)] # ok
|
|
|
|
|
|
2022-01-28 12:11:29 -05:00
|
|
|
|
In particular allowing ``Annotated[]`` to be the outermost annotation
|
|
|
|
|
for an item allows better interoperability with non-typing uses of
|
|
|
|
|
annotations, which may always want ``Annotated[]`` as the outermost annotation.
|
|
|
|
|
[3]_
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
2022-02-14 22:26:57 -05:00
|
|
|
|
Runtime behavior
|
|
|
|
|
----------------
|
|
|
|
|
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
Interaction with ``get_type_hints()``
|
2022-02-14 22:26:57 -05:00
|
|
|
|
'''''''''''''''''''''''''''''''''''''
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
|
|
|
|
``typing.get_type_hints(...)`` applied to a TypedDict will by default
|
|
|
|
|
strip out any ``Required[]`` or ``NotRequired[]`` type qualifiers,
|
|
|
|
|
since these qualifiers are expected to be inconvenient for code
|
|
|
|
|
casually introspecting type annotations.
|
|
|
|
|
|
|
|
|
|
``typing.get_type_hints(..., include_extras=True)`` however
|
|
|
|
|
*will* retain ``Required[]`` and ``NotRequired[]`` type qualifiers,
|
|
|
|
|
for advanced code introspecting type annotations that
|
|
|
|
|
wishes to preserve *all* annotations in the original source:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[int]
|
2022-01-21 06:03:51 -05:00
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
assert get_type_hints(Movie) == \
|
|
|
|
|
{'title': str, 'year': int}
|
|
|
|
|
assert get_type_hints(Movie, include_extras=True) == \
|
|
|
|
|
{'title': str, 'year': NotRequired[int]}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Interaction with ``get_origin()`` and ``get_args()``
|
2022-02-14 22:26:57 -05:00
|
|
|
|
''''''''''''''''''''''''''''''''''''''''''''''''''''
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
|
|
|
|
``typing.get_origin()`` and ``typing.get_args()`` will be updated to
|
|
|
|
|
recognize ``Required[]`` and ``NotRequired[]``:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
assert get_origin(Required[int]) is Required
|
|
|
|
|
assert get_args(Required[int]) == (int,)
|
2022-01-21 06:03:51 -05:00
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
assert get_origin(NotRequired[int]) is NotRequired
|
|
|
|
|
assert get_args(NotRequired[int]) == (int,)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Interaction with ``__required_keys__`` and ``__optional_keys__``
|
2022-02-14 22:26:57 -05:00
|
|
|
|
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
|
|
|
|
An item marked with ``Required[]`` will always appear
|
|
|
|
|
in the ``__required_keys__`` for its enclosing TypedDict. Similarly an item
|
|
|
|
|
marked with ``NotRequired[]`` will always appear in ``__optional_keys__``.
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
assert Movie.__required_keys__ == frozenset({'title'})
|
|
|
|
|
assert Movie.__optional_keys__ == frozenset({'year'})
|
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Backwards Compatibility
|
|
|
|
|
=======================
|
|
|
|
|
|
|
|
|
|
No backward incompatible changes are made by this PEP.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
How to Teach This
|
|
|
|
|
=================
|
|
|
|
|
|
|
|
|
|
To define a TypedDict where most keys are required and some are
|
|
|
|
|
potentially-missing, define a single TypedDict as normal
|
2022-02-14 22:26:57 -05:00
|
|
|
|
(without the ``total`` keyword)
|
2022-01-20 11:18:41 -05:00
|
|
|
|
and mark those few keys that are potentially-missing with ``NotRequired[]``.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
To define a TypedDict where most keys are potentially-missing and a few are
|
|
|
|
|
required, define a ``total=False`` TypedDict
|
2022-01-20 11:18:41 -05:00
|
|
|
|
and mark those few keys that are required with ``Required[]``.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
If some items accept ``None`` in addition to a regular value, it is
|
2022-03-09 12:57:04 -05:00
|
|
|
|
recommended that the ``TYPE|None`` notation be preferred over
|
2021-02-26 00:34:11 -05:00
|
|
|
|
``Optional[TYPE]`` for marking such item values, to avoid using
|
2022-01-20 11:18:41 -05:00
|
|
|
|
``Required[]`` or ``NotRequired[]`` alongside ``Optional[]``
|
2021-02-26 00:34:11 -05:00
|
|
|
|
within the same TypedDict definition:
|
|
|
|
|
|
|
|
|
|
Yes:
|
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: good
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
from __future__ import annotations # for Python 3.7-3.9
|
|
|
|
|
|
|
|
|
|
class Dog(TypedDict):
|
|
|
|
|
name: str
|
|
|
|
|
owner: NotRequired[str|None]
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
Okay (required for Python 3.5.3-3.6):
|
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: maybe
|
2022-01-20 11:18:41 -05:00
|
|
|
|
|
|
|
|
|
class Dog(TypedDict):
|
|
|
|
|
name: str
|
|
|
|
|
owner: 'NotRequired[str|None]'
|
|
|
|
|
|
|
|
|
|
No:
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: bad
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
class Dog(TypedDict):
|
|
|
|
|
name: str
|
|
|
|
|
# ick; avoid using both Optional and NotRequired
|
|
|
|
|
owner: NotRequired[Optional[str]]
|
|
|
|
|
|
2022-01-28 12:11:29 -05:00
|
|
|
|
Usage in Python <3.11
|
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
|
|
If your code supports Python <3.11 and wishes to use ``Required[]`` or
|
|
|
|
|
``NotRequired[]`` then it should use ``typing_extensions.TypedDict`` rather
|
|
|
|
|
than ``typing.TypedDict`` because the latter will not understand
|
2022-02-21 18:47:19 -05:00
|
|
|
|
``(Not)Required[]``. In particular ``__required_keys__`` and
|
2022-01-28 12:11:29 -05:00
|
|
|
|
``__optional_keys__`` on the resulting TypedDict type will not be correct:
|
|
|
|
|
|
|
|
|
|
Yes (Python 3.11+ only):
|
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: good
|
2022-01-28 12:11:29 -05:00
|
|
|
|
|
|
|
|
|
from typing import NotRequired, TypedDict
|
|
|
|
|
|
|
|
|
|
class Dog(TypedDict):
|
|
|
|
|
name: str
|
|
|
|
|
owner: NotRequired[str|None]
|
|
|
|
|
|
|
|
|
|
Yes (Python <3.11 and 3.11+):
|
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: good
|
2022-01-28 12:11:29 -05:00
|
|
|
|
|
|
|
|
|
from __future__ import annotations # for Python 3.7-3.9
|
|
|
|
|
|
|
|
|
|
from typing_extensions import NotRequired, TypedDict # for Python <3.11 with (Not)Required
|
|
|
|
|
|
|
|
|
|
class Dog(TypedDict):
|
|
|
|
|
name: str
|
|
|
|
|
owner: NotRequired[str|None]
|
|
|
|
|
|
|
|
|
|
No (Python <3.11 and 3.11+):
|
|
|
|
|
|
2024-02-16 01:27:36 -05:00
|
|
|
|
.. code-block::
|
|
|
|
|
:class: bad
|
2022-01-28 12:11:29 -05:00
|
|
|
|
|
|
|
|
|
from typing import TypedDict # oops: should import from typing_extensions instead
|
|
|
|
|
from typing_extensions import NotRequired
|
2022-02-21 18:47:19 -05:00
|
|
|
|
|
2022-01-28 12:11:29 -05:00
|
|
|
|
class Movie(TypedDict):
|
|
|
|
|
title: str
|
|
|
|
|
year: NotRequired[int]
|
2022-02-21 18:47:19 -05:00
|
|
|
|
|
2022-01-28 12:11:29 -05:00
|
|
|
|
assert Movie.__required_keys__ == frozenset({'title', 'year'}) # yikes
|
|
|
|
|
assert Movie.__optional_keys__ == frozenset() # yikes
|
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Reference Implementation
|
|
|
|
|
========================
|
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
The `mypy <http://www.mypy-lang.org/>`__
|
2022-01-28 12:11:29 -05:00
|
|
|
|
`0.930 <https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html>`__,
|
|
|
|
|
`pyright <https://github.com/Microsoft/pyright>`__
|
|
|
|
|
`1.1.117 <https://github.com/microsoft/pyright/commit/7ed245b1845173090c6404e49912e8cbfb3417c8>`__,
|
|
|
|
|
and `pyanalyze <https://github.com/quora/pyanalyze>`__
|
|
|
|
|
`0.4.0 <https://pyanalyze.readthedocs.io/en/latest/changelog.html#version-0-4-0-november-18-2021>`__
|
2022-01-20 11:18:41 -05:00
|
|
|
|
type checkers support ``Required`` and ``NotRequired``.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2022-01-20 11:18:41 -05:00
|
|
|
|
A reference implementation of the runtime component is provided in the
|
|
|
|
|
`typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__
|
|
|
|
|
module.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rejected Ideas
|
|
|
|
|
==============
|
|
|
|
|
|
|
|
|
|
Special syntax around the *key* of a TypedDict item
|
|
|
|
|
---------------------------------------------------
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict):
|
|
|
|
|
opt1?: str # may not exist, but if exists, value is string
|
2022-01-20 11:18:41 -05:00
|
|
|
|
opt2: Optional[str] # always exists, but may have None value
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
This notation would require Python grammar changes and it is not
|
2022-01-20 11:18:41 -05:00
|
|
|
|
believed that marking TypedDict items as required or potentially-missing
|
|
|
|
|
would meet the high bar required to make such grammar changes.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict):
|
|
|
|
|
Optional[opt1]: str # may not exist, but if exists, value is string
|
2022-01-20 11:18:41 -05:00
|
|
|
|
opt2: Optional[str] # always exists, but may have None value
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
This notation causes ``Optional[]`` to take on different meanings depending
|
2022-01-20 11:18:41 -05:00
|
|
|
|
on where it is positioned, which is inconsistent and confusing.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Also, “let’s just not put funny syntax before the colon.” [1]_
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Marking required or potentially-missing keys with an operator
|
|
|
|
|
-------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
We could use unary ``+`` as shorthand to mark a required key, unary
|
|
|
|
|
``-`` to mark a potentially-missing key, or unary ``~`` to mark a key
|
|
|
|
|
with opposite-of-normal totality:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict, total=False):
|
2022-01-20 11:18:41 -05:00
|
|
|
|
req1: +int # + means a required key, or Required[]
|
2021-02-26 00:34:11 -05:00
|
|
|
|
opt1: str
|
|
|
|
|
req2: +float
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict):
|
|
|
|
|
req1: int
|
2022-01-20 11:18:41 -05:00
|
|
|
|
opt1: -str # - means a potentially-missing key, or NotRequired[]
|
2021-02-26 00:34:11 -05:00
|
|
|
|
req2: float
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict):
|
|
|
|
|
req1: int
|
|
|
|
|
opt1: ~str # ~ means a opposite-of-normal-totality key
|
|
|
|
|
req2: float
|
|
|
|
|
|
|
|
|
|
Such operators could be implemented on ``type`` via the ``__pos__``,
|
|
|
|
|
``__neg__`` and ``__invert__`` special methods without modifying the
|
|
|
|
|
grammar.
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
It was decided that it would be prudent to introduce long-form notation
|
2022-01-20 11:18:41 -05:00
|
|
|
|
(i.e. ``Required[]`` and ``NotRequired[]``) before introducing
|
2022-03-09 12:57:04 -05:00
|
|
|
|
any short-form notation. Future PEPs may reconsider introducing this
|
|
|
|
|
or other short-form notation options.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
Note when reconsidering introducing this short-form notation that
|
2022-01-20 11:18:41 -05:00
|
|
|
|
``+``, ``-``, and ``~`` already have existing meanings in the Python
|
|
|
|
|
typing world: covariant, contravariant, and invariant:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
>>> from typing import TypeVar
|
|
|
|
|
>>> (TypeVar('T', covariant=True), TypeVar('U', contravariant=True), TypeVar('V'))
|
|
|
|
|
(+T, -U, ~V)
|
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Marking absence of a value with a special constant
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
|
|
|
|
|
We could introduce a new type-level constant which signals the absence
|
|
|
|
|
of a value when used as a union member, similar to JavaScript’s
|
|
|
|
|
``undefined`` type, perhaps called ``Missing``:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict):
|
|
|
|
|
req1: int
|
|
|
|
|
opt1: str|Missing
|
|
|
|
|
req2: float
|
|
|
|
|
|
|
|
|
|
Such a ``Missing`` constant could also be used for other scenarios such
|
|
|
|
|
as the type of a variable which is only conditionally defined:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyClass:
|
|
|
|
|
attr: int|Missing
|
2022-02-21 18:47:19 -05:00
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
def __init__(self, set_attr: bool) -> None:
|
|
|
|
|
if set_attr:
|
|
|
|
|
self.attr = 10
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
def foo(set_attr: bool) -> None:
|
|
|
|
|
if set_attr:
|
|
|
|
|
attr = 10
|
|
|
|
|
reveal_type(attr) # int|Missing
|
|
|
|
|
|
|
|
|
|
Misalignment with how unions apply to values
|
|
|
|
|
''''''''''''''''''''''''''''''''''''''''''''
|
|
|
|
|
|
|
|
|
|
However this use of ``...|Missing``, equivalent to
|
|
|
|
|
``Union[..., Missing]``, doesn’t align well with what a union normally
|
|
|
|
|
means: ``Union[...]`` always describes the type of a *value* that is
|
|
|
|
|
present. By contrast missingness or non-totality is a property of a
|
|
|
|
|
*variable* instead. Current precedent for marking properties of a
|
|
|
|
|
variable include ``Final[...]`` and ``ClassVar[...]``, which the
|
|
|
|
|
proposal for ``Required[...]`` is aligned with.
|
|
|
|
|
|
|
|
|
|
Misalignment with how unions are subdivided
|
|
|
|
|
'''''''''''''''''''''''''''''''''''''''''''
|
|
|
|
|
|
|
|
|
|
Furthermore the use of ``Union[..., Missing]`` doesn’t align with the
|
|
|
|
|
usual ways that union values are broken down: Normally you can eliminate
|
|
|
|
|
components of a union type using ``isinstance`` checks:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Packet:
|
|
|
|
|
data: Union[str, bytes]
|
|
|
|
|
|
|
|
|
|
def send_data(packet: Packet) -> None:
|
|
|
|
|
if isinstance(packet.data, str):
|
|
|
|
|
reveal_type(packet.data) # str
|
|
|
|
|
packet_bytes = packet.data.encode('utf-8')
|
|
|
|
|
else:
|
|
|
|
|
reveal_type(packet.data) # bytes
|
|
|
|
|
packet_bytes = packet.data
|
|
|
|
|
socket.send(packet_bytes)
|
|
|
|
|
|
|
|
|
|
However if we were to allow ``Union[..., Missing]`` you’d either have to
|
|
|
|
|
eliminate the ``Missing`` case with ``hasattr`` for object attributes:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class Packet:
|
|
|
|
|
data: Union[str, Missing]
|
|
|
|
|
|
|
|
|
|
def send_data(packet: Packet) -> None:
|
|
|
|
|
if hasattr(packet, 'data'):
|
|
|
|
|
reveal_type(packet.data) # str
|
|
|
|
|
packet_bytes = packet.data.encode('utf-8')
|
|
|
|
|
else:
|
|
|
|
|
reveal_type(packet.data) # Missing? error?
|
|
|
|
|
packet_bytes = b''
|
|
|
|
|
socket.send(packet_bytes)
|
|
|
|
|
|
|
|
|
|
or a check against ``locals()`` for local variables:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
def send_data(packet_data: Optional[str]) -> None:
|
|
|
|
|
packet_bytes: Union[str, Missing]
|
|
|
|
|
if packet_data is not None:
|
|
|
|
|
packet_bytes = packet.data.encode('utf-8')
|
2022-02-21 18:47:19 -05:00
|
|
|
|
|
2021-02-26 00:34:11 -05:00
|
|
|
|
if 'packet_bytes' in locals():
|
|
|
|
|
reveal_type(packet_bytes) # bytes
|
|
|
|
|
socket.send(packet_bytes)
|
|
|
|
|
else:
|
|
|
|
|
reveal_type(packet_bytes) # Missing? error?
|
|
|
|
|
|
|
|
|
|
or a check via other means, such as against ``globals()`` for global
|
|
|
|
|
variables:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
warning: Union[str, Missing]
|
|
|
|
|
import sys
|
|
|
|
|
if sys.version_info < (3, 6):
|
|
|
|
|
warning = 'Your version of Python is unsupported!'
|
|
|
|
|
|
|
|
|
|
if 'warning' in globals():
|
|
|
|
|
reveal_type(warning) # str
|
|
|
|
|
print(warning)
|
|
|
|
|
else:
|
|
|
|
|
reveal_type(warning) # Missing? error?
|
|
|
|
|
|
|
|
|
|
Weird and inconsistent. ``Missing`` is not really a value at all; it’s
|
|
|
|
|
an absence of definition and such an absence should be treated
|
|
|
|
|
specially.
|
|
|
|
|
|
|
|
|
|
Difficult to implement
|
|
|
|
|
''''''''''''''''''''''
|
|
|
|
|
|
|
|
|
|
Eric Traut from the Pyright type checker team has stated that
|
2022-03-09 12:57:04 -05:00
|
|
|
|
implementing a ``Union[..., Missing]``-style notation would be
|
2021-02-26 00:34:11 -05:00
|
|
|
|
difficult. [2]_
|
|
|
|
|
|
|
|
|
|
Introduces a second null-like value into Python
|
|
|
|
|
'''''''''''''''''''''''''''''''''''''''''''''''
|
|
|
|
|
|
|
|
|
|
Defining a new ``Missing`` type-level constant would be very close to
|
|
|
|
|
introducing a new ``Missing`` value-level constant at runtime, creating
|
|
|
|
|
a second null-like runtime value in addition to ``None``. Having two
|
|
|
|
|
different null-like constants in Python (``None`` and ``Missing``) would
|
|
|
|
|
be confusing. Many newcomers to JavaScript already have difficulty
|
|
|
|
|
distinguishing between its analogous constants ``null`` and
|
|
|
|
|
``undefined``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Replace Optional with Nullable. Repurpose Optional to mean “optional item”.
|
|
|
|
|
---------------------------------------------------------------------------
|
|
|
|
|
|
2022-03-09 12:57:04 -05:00
|
|
|
|
``Optional[]`` is too ubiquitous to deprecate, although use of it
|
|
|
|
|
*may* fade over time in favor of the ``T|None`` notation specified by :pep:`604`.
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Change Optional to mean “optional item” in certain contexts instead of “nullable”
|
|
|
|
|
---------------------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
Consider the use of a special flag on a TypedDict definition to alter
|
|
|
|
|
the interpretation of ``Optional`` inside the TypedDict to mean
|
|
|
|
|
“optional item” rather than its usual meaning of “nullable”:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict, optional_as_missing=True):
|
|
|
|
|
req1: int
|
|
|
|
|
opt1: Optional[str]
|
|
|
|
|
|
|
|
|
|
or:
|
|
|
|
|
|
|
|
|
|
::
|
|
|
|
|
|
|
|
|
|
class MyThing(TypedDict, optional_as_nullable=False):
|
|
|
|
|
req1: int
|
|
|
|
|
opt1: Optional[str]
|
|
|
|
|
|
|
|
|
|
This would add more confusion for users because it would mean that in
|
2022-01-20 11:18:41 -05:00
|
|
|
|
*some* contexts the meaning of ``Optional[]`` is different than in
|
2021-02-26 00:34:11 -05:00
|
|
|
|
other contexts, and it would be easy to overlook the flag.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Various synonyms for “potentially-missing item”
|
|
|
|
|
-----------------------------------------------
|
|
|
|
|
|
|
|
|
|
- Omittable – too easy to confuse with optional
|
|
|
|
|
- OptionalItem, OptionalKey – two words; too easy to confuse with
|
|
|
|
|
optional
|
|
|
|
|
- MayExist, MissingOk – two words
|
|
|
|
|
- Droppable – too similar to Rust’s ``Drop``, which means something
|
|
|
|
|
different
|
|
|
|
|
- Potential – too vague
|
|
|
|
|
- Open – sounds like applies to an entire structure rather then to an
|
|
|
|
|
item
|
|
|
|
|
- Excludable
|
|
|
|
|
- Checked
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
References
|
|
|
|
|
==========
|
|
|
|
|
|
|
|
|
|
.. [1] https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/
|
|
|
|
|
|
|
|
|
|
.. [2] https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/
|
|
|
|
|
|
2022-01-28 12:11:29 -05:00
|
|
|
|
.. [3] https://bugs.python.org/issue46491
|
2021-02-26 00:34:11 -05:00
|
|
|
|
|
|
|
|
|
Copyright
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
This document is placed in the public domain or under the
|
|
|
|
|
CC0-1.0-Universal license, whichever is more permissive.
|