iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 655: Integrate feedback from circa Feb 2021 (#2248)

This commit is contained in:
David Foster 2022-01-20 11:18:41 -05:00 committed by GitHub
parent b16058c427
commit ab7cef905e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 144 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

182
pep-0655.rst
View File

@ -6,24 +6,22 @@ Discussions-To: typing-sig at python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Requires: 604
Created: 30-Jan-2021
Python-Version: 3.11
Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021
Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021, 17-Jan-2022
Abstract
========
`PEP 589 <https://www.python.org/dev/peps/pep-0589/>`__ defines syntax
:pep:`589` defines syntax
for declaring a TypedDict with all required keys and syntax for defining
a TypedDict with `all potentially-missing
keys <https://www.python.org/dev/peps/pep-0589/#totality>`__ however it
a TypedDict with :pep:`all potentially-missing keys <589#totality>` however it
does not provide any syntax to declare some keys as required and others
as potentially-missing. This PEP introduces two new syntaxes:
``Required[...]`` which can be used on individual items of a
``Required[]`` which can be used on individual items of a
TypedDict to mark them as required, and
``NotRequired[...]`` which can be used on individual items
``NotRequired[]`` which can be used on individual items
to mark them as potentially-missing.
@ -63,7 +61,7 @@ customary in other languages like TypeScript:
}
The difficulty is that the best word for marking a potentially-missing
key, ``Optional[...]``, is already used in Python for a completely
key, ``Optional[]``, is already used in Python for a completely
different purpose: marking values that could be either of a particular
type or ``None``. In particular the following does not work:
@ -74,17 +72,17 @@ type or ``None``. In particular the following does not work:
year: Optional[int] # means int|None, not potentially-missing!
Attempting to use any synonym of “optional” to mark potentially-missing
keys (like ``Missing[...]``) would be too similar to ``Optional[...]``
keys (like ``Missing[]``) would be too similar to ``Optional[]``
and be easy to confuse with it.
Thus it was decided to focus on positive-form phrasing for required keys
instead, which is straightforward to spell as ``Required[...]``.
instead, which is straightforward to spell as ``Required[]``.
Nevertheless it is common for folks wanting to extend a regular
(``total=True``) TypedDict to only want to add a small number of
potentially-missing keys, which necessitates a way to mark keys that are
*not* required and potentially-missing, and so we also allow the
``NotRequired[...]`` form for that case.
``NotRequired[]`` form for that case.
Specification
@ -109,10 +107,10 @@ potentially-missing key:
title: str
year: NotRequired[int]
It is an error to use ``Required[...]`` or ``NotRequired[...]`` in any
It is an error to use ``Required[]`` or ``NotRequired[]`` in any
location that is not an item of a TypedDict.
It is valid to use ``Required[...]`` and ``NotRequired[...]`` even for
It is valid to use ``Required[]`` and ``NotRequired[]`` even for
items where it is redundant, to enable additional explicitness if desired:
::
@ -121,6 +119,96 @@ items where it is redundant, to enable additional explicitness if desired:
title: Required[str] # redundant
year: NotRequired[int]
It is an error to use both ``Required[]`` and ``NotRequired[]`` at the
same time:
::
class Movie(TypedDict):
title: str
year: NotRequired[Required[int]] # ERROR
The :pep:`alternative syntax <589#alternative-syntax>`
for TypedDict also supports
``Required[]`` and ``NotRequired[]``:
::
Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]})
Interaction with ``Annotated[]``
-----------------------------------
``Required[]`` and ``NotRequired[]`` can be used with ``Annotated[]``,
in any nesting order:
::
class Movie(TypedDict):
title: str
year: NotRequired[Annotated[int, ValueRange(-9999, 9999)]] # ok
::
class Movie(TypedDict):
title: str
year: Annotated[NotRequired[int], ValueRange(-9999, 9999)] # ok
Interaction with ``get_type_hints()``
-------------------------------------
``typing.get_type_hints(...)`` applied to a TypedDict will by default
strip out any ``Required[]`` or ``NotRequired[]`` type qualifiers,
since these qualifiers are expected to be inconvenient for code
casually introspecting type annotations.
``typing.get_type_hints(..., include_extras=True)`` however
*will* retain ``Required[]`` and ``NotRequired[]`` type qualifiers,
for advanced code introspecting type annotations that
wishes to preserve *all* annotations in the original source:
::
class Movie(TypedDict):
title: str
year: NotRequired[int]
assert get_type_hints(Movie) == \
{'title': str, 'year': int}
assert get_type_hints(Movie, include_extras=True) == \
{'title': str, 'year': NotRequired[int]}
Interaction with ``get_origin()`` and ``get_args()``
----------------------------------------------------
``typing.get_origin()`` and ``typing.get_args()`` will be updated to
recognize ``Required[]`` and ``NotRequired[]``:
::
assert get_origin(Required[int]) is Required
assert get_args(Required[int]) == (int,)
assert get_origin(NotRequired[int]) is NotRequired
assert get_args(NotRequired[int]) == (int,)
Interaction with ``__required_keys__`` and ``__optional_keys__``
----------------------------------------------------------------
An item marked with ``Required[]`` will always appear
in the ``__required_keys__`` for its enclosing TypedDict. Similarly an item
marked with ``NotRequired[]`` will always appear in ``__optional_keys__``.
::
assert Movie.__required_keys__ == frozenset({'title'})
assert Movie.__optional_keys__ == frozenset({'year'})
Backwards Compatibility
=======================
@ -133,16 +221,16 @@ How to Teach This
To define a TypedDict where most keys are required and some are
potentially-missing, define a single TypedDict as normal
and mark those few keys that are potentially-missing with ``NotRequired[...]``.
and mark those few keys that are potentially-missing with ``NotRequired[]``.
To define a TypedDict where most keys are potentially-missing and a few are
required, define a ``total=False`` TypedDict
and mark those few keys that are required with ``Required[...]``.
and mark those few keys that are required with ``Required[]``.
If some items accept ``None`` in addition to a regular value, it is
recommended that the ``TYPE|None`` syntax be preferred over
``Optional[TYPE]`` for marking such item values, to avoid using
``Required[...]`` or ``NotRequired[...]`` alongside ``Optional[...]``
``Required[]`` or ``NotRequired[]`` alongside ``Optional[]``
within the same TypedDict definition:
Yes:
@ -155,7 +243,15 @@ Yes:
name: str
owner: NotRequired[str|None]
Avoid (unless Python 3.5-3.6):
Okay (required for Python 3.5.3-3.6):
::
class Dog(TypedDict):
name: str
owner: 'NotRequired[str|None]'
No:
::
@ -168,15 +264,15 @@ Avoid (unless Python 3.5-3.6):
Reference Implementation
========================
The goal is to be able to make the following statement:
The `mypy <http://www.mypy-lang.org/>`__
`0.930 <https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html>`__
and `pyright <https://github.com/Microsoft/pyright>`__
`1.1.117 <https://github.com/microsoft/pyright/commit/7ed245b1845173090c6404e49912e8cbfb3417c8>`__
type checkers support ``Required`` and ``NotRequired``.
The `mypy <http://www.mypy-lang.org/>`__ type checker supports
``Required`` and ``NotRequired``. A reference implementation of the
runtime component is provided in the
`typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__
module.
The mypy implementation is currently still being worked on.
A reference implementation of the runtime component is provided in the
`typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__
module.
Rejected Ideas
@ -189,19 +285,20 @@ Special syntax around the *key* of a TypedDict item
class MyThing(TypedDict):
opt1?: str # may not exist, but if exists, value is string
opt2: Optional[str] # always exists, but may have null value
opt2: Optional[str] # always exists, but may have None value
or:
This syntax would require Python grammar changes and it is not
believed that marking TypedDict items as required or potentially-missing
would meet the high bar required to make such grammar changes.
::
class MyThing(TypedDict):
Optional[opt1]: str # may not exist, but if exists, value is string
opt2: Optional[str] # always exists, but may have null value
opt2: Optional[str] # always exists, but may have None value
These syntaxes would require Python grammar changes and it is not
believed that marking TypedDict items as required or potentially-missing
would meet the high bar required to make such grammar changes.
This syntax causes ``Optional[]`` to take on different meanings depending
on where it is positioned, which is inconsistent and confusing.
Also, “let’s just not put funny syntax before the colon.” [1]_
@ -216,13 +313,13 @@ with opposite-of-normal totality:
::
class MyThing(TypedDict, total=False):
req1: +int # + means a required key, or Required[...]
req1: +int # + means a required key, or Required[]
opt1: str
req2: +float
class MyThing(TypedDict):
req1: int
opt1: -str # - means a potentially-missing key, or NotRequired[...]
opt1: -str # - means a potentially-missing key, or NotRequired[]
req2: float
class MyThing(TypedDict):
@ -235,10 +332,20 @@ Such operators could be implemented on ``type`` via the ``__pos__``,
grammar.
It was decided that it would be prudent to introduce longform syntax
(i.e. ``Required[...]`` and ``NotRequired[...]``) before introducing
(i.e. ``Required[]`` and ``NotRequired[]``) before introducing
any shortform syntax. Future PEPs may reconsider introducing this
or other shortform syntax options.
Note when reconsidering introducing this shortform syntax that
``+``, ``-``, and ``~`` already have existing meanings in the Python
typing world: covariant, contravariant, and invariant:
::
>>> from typing import TypeVar
>>> (TypeVar('T', covariant=True), TypeVar('U', contravariant=True), TypeVar('V'))
(+T, -U, ~V)
Marking absence of a value with a special constant
--------------------------------------------------
@ -379,9 +486,8 @@ distinguishing between its analogous constants ``null`` and
Replace Optional with Nullable. Repurpose Optional to mean “optional item”.
---------------------------------------------------------------------------
``Optional[...]`` is too ubiquitous to deprecate. Although use of it
*may* fade over time in favor of the ``T|None`` syntax specified by `PEP
604 <https://www.python.org/dev/peps/pep-0604/>`__.
``Optional[]`` is too ubiquitous to deprecate. Although use of it
*may* fade over time in favor of the ``T|None`` syntax specified by :pep:`604`.
Change Optional to mean “optional item” in certain contexts instead of “nullable”
@ -406,7 +512,7 @@ or:
opt1: Optional[str]
This would add more confusion for users because it would mean that in
*some* contexts the meaning of ``Optional[...]`` is different than in
*some* contexts the meaning of ``Optional[]`` is different than in
other contexts, and it would be easy to overlook the flag.