PEP 728: Incorporate feedback from discussions (#3680)
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This commit is contained in:
parent
c6c71c52e5
commit
290f47b4f4
|
@ -15,11 +15,11 @@ Post-History: `09-Feb-2024 <https://discuss.python.org/t/pep-728-typeddict-with-
|
|||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes a way to type extra items for :class:`~typing.TypedDict` using
|
||||
a reserved ``__extra__`` key. This addresses the need to define a subset of
|
||||
keys that might appear in a ``dict`` while permitting additional items of a
|
||||
specified type, and the need to create a closed TypedDict type with ``__extra__:
|
||||
Never``.
|
||||
This PEP proposes a way to limit extra items for :class:`~typing.TypedDict`
|
||||
using a ``closed`` argument and to type them with the special ``__extra_items__``
|
||||
key. This addresses the need to define closed TypedDict type or to type a subset
|
||||
of keys that might appear in a ``dict`` while permitting additional items of a
|
||||
specified type.
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
@ -143,25 +143,39 @@ This proposal aims to support a similar feature without introducing general
|
|||
intersection of types or syntax changes, offering a natural extension to the
|
||||
existing type consistency rules.
|
||||
|
||||
We propose that we give the dunder attribute ``__extra__`` a special meaning:
|
||||
When it is defined on a TypedDict type, extra items are allowed, and their types
|
||||
should be compatible with the value type of ``__extra__``. Different from index
|
||||
signatures, the types of known items do not need to be consistent with the value
|
||||
type of ``__extra__``.
|
||||
We propose that we add an argument ``closed`` to TypedDict. Similar to
|
||||
``total``, only a literal ``True`` or ``False`` value is allowed. When
|
||||
``closed=True`` is used in the TypedDict type definition, we give the dunder
|
||||
attribute ``__extra_items__`` a special meaning: extra items are allowed, and
|
||||
their types should be compatible with the value type of ``__extra_items__``.
|
||||
|
||||
If ``closed=True`` is set, but there is no ``__extra_items__`` key, the
|
||||
TypedDict is treated as if it contained an item ``__extra_items__: Never``.
|
||||
|
||||
Note that ``__extra_items__`` on the same TypedDict type definition will remain
|
||||
as a regular item if ``closed=True`` is not used.
|
||||
|
||||
Different from index signatures, the types of the known items do not need to be
|
||||
consistent with the value type of ``__extra_items__``.
|
||||
|
||||
There are some advantages to this approach:
|
||||
|
||||
- Inheritance works naturally. ``__extra__`` defined on a TypedDict will also
|
||||
be available to its subclasses.
|
||||
- Inheritance works naturally. ``__extra_items__`` defined on a TypedDict will
|
||||
also be available to its subclasses.
|
||||
|
||||
- We can build on top of the `type consistency rules defined in the typing spec
|
||||
<https://typing.readthedocs.io/en/latest/spec/typeddict.html#type-consistency>`__.
|
||||
``__extra__`` can be treated as a pseudo-item in terms of type consistency.
|
||||
``__extra_items__`` can be treated as a pseudo-item in terms of type
|
||||
consistency.
|
||||
|
||||
- There is no need to introduce a syntax to specify the type of the extra items.
|
||||
- There is no need to introduce a grammar change to specify the type of the
|
||||
extra items.
|
||||
|
||||
- We can precisely type the extra items without making ``__extra__`` the union
|
||||
of known items.
|
||||
- We can precisely type the extra items without making ``__extra_items__`` the
|
||||
union of known items.
|
||||
|
||||
- We do not lose backwards compatibility as ``__extra_items__`` still can be
|
||||
used as a regular key.
|
||||
|
||||
Specification
|
||||
=============
|
||||
|
@ -169,21 +183,24 @@ Specification
|
|||
This specification is structured to parallel :pep:`589` to highlight changes to
|
||||
the original TypedDict specification.
|
||||
|
||||
Extra items are treated as non-required items having the same type of
|
||||
``__extra__`` whose keys are allowed when determining
|
||||
If ``closed=True`` is specified, extra items are treated as non-required items
|
||||
having the same type of ``__extra_items__`` whose keys are allowed when
|
||||
determining
|
||||
`supported and unsupported operations
|
||||
<https://typing.readthedocs.io/en/latest/spec/typeddict.html#supported-and-unsupported-operations>`__.
|
||||
|
||||
Using TypedDict Types
|
||||
---------------------
|
||||
|
||||
For a TypedDict type that has the ``__extra__`` key, during construction, the
|
||||
value type of each unknown item is expected to be non-required and compatible
|
||||
with the value type of ``__extra__``. For example::
|
||||
Assuming that ``closed=True`` is used in the TypedDict type definition.
|
||||
|
||||
class Movie(TypedDict):
|
||||
For a TypedDict type that has the special ``__extra_items__`` key, during
|
||||
construction, the value type of each unknown item is expected to be non-required
|
||||
and compatible with the value type of ``__extra_items__``. For example::
|
||||
|
||||
class Movie(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: bool
|
||||
__extra_items__: bool
|
||||
|
||||
a: Movie = {"name": "Blade Runner", "novel_adaptation": True} # OK
|
||||
b: Movie = {
|
||||
|
@ -191,48 +208,72 @@ with the value type of ``__extra__``. For example::
|
|||
"year": 1982, # Not OK. 'int' is incompatible with 'bool'
|
||||
}
|
||||
|
||||
In this example, ``__extra__: bool`` does not mean that ``Movie`` has a required
|
||||
string key ``"__extra__"`` whose value type is ``bool``. Instead, it specifies that
|
||||
keys other than "name" have a value type of ``bool`` and are non-required.
|
||||
In this example, ``__extra_items__: bool`` does not mean that ``Movie`` has a
|
||||
required string key ``"__extra_items__"`` whose value type is ``bool``. Instead,
|
||||
it specifies that keys other than "name" have a value type of ``bool`` and are
|
||||
non-required.
|
||||
|
||||
The alternative inline syntax is also supported::
|
||||
|
||||
Movie = TypedDict("Movie", {"name": str, "__extra__": bool})
|
||||
Movie = TypedDict("Movie", {"name": str, "__extra_items__": bool}, closed=True)
|
||||
|
||||
Accessing extra keys is allowed. Type checkers must infer its value type from
|
||||
the value type of ``__extra__``::
|
||||
the value type of ``__extra_items__``::
|
||||
|
||||
def f(movie: Movie) -> None:
|
||||
reveal_type(movie["name"]) # Revealed type is 'str'
|
||||
reveal_type(movie["novel_adaptation"]) # Revealed type is 'bool'
|
||||
|
||||
Interaction with PEP 705
|
||||
------------------------
|
||||
When a TypedDict type defines ``__extra_items__`` without ``closed=True``,
|
||||
``closed`` defaults to ``False`` and the key is assumed to be a regular key::
|
||||
|
||||
When ``__extra__`` is annotated with ``ReadOnly[]``, the extra items on the
|
||||
TypedDict have the properties of read-only items. This affects subclassing
|
||||
according to the inheritance rules specified in :pep:`PEP 705 <705#Inheritance>`.
|
||||
class Movie(TypedDict):
|
||||
name: str
|
||||
__extra_items__: bool
|
||||
|
||||
a: Movie = {"name": "Blade Runner", "novel_adaptation": True} # Not OK. Unexpected key 'novel_adaptation'
|
||||
b: Movie = {
|
||||
"name": "Blade Runner",
|
||||
"__extra_items__": True, # OK
|
||||
}
|
||||
|
||||
Notably, a subclass of the TypedDict type may redeclare the value type of
|
||||
``__extra__`` or of additional non-extra items if the TypedDict type declares
|
||||
``__extra__`` to be read-only.
|
||||
For such non-closed TypedDict types, it is assumed that they allow non-required
|
||||
extra items of value type ``ReadOnly[object]`` during inheritance or type
|
||||
consistency checks. However, extra keys found during construction should still
|
||||
be rejected by the type checker.
|
||||
|
||||
More details are discussed in the later sections.
|
||||
``closed`` is not inherited through subclassing::
|
||||
|
||||
class MovieBase(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra_items__: ReadOnly[str | None]
|
||||
|
||||
class Movie(MovieBase):
|
||||
__extra_items__: str # A regular key
|
||||
|
||||
a: Movie = {"name": "Blade Runner", "__extra_items__": None} # Not OK. 'None' is incompatible with 'str'
|
||||
b: Movie = {"name": "Blade Runner", "other_extra_key": None} # OK
|
||||
|
||||
Here, ``"__extra_items__"`` in ``a`` is a regular key defined on ``Movie`` where
|
||||
its value type is narrowed from ``ReadOnly[str | None]`` to ``str``,
|
||||
``"other_extra_key"`` in ``b`` is an extra key whose value type must be
|
||||
consistent with the value type of ``"__extra_items__"`` defined on
|
||||
``MovieBase``.
|
||||
|
||||
Interaction with Totality
|
||||
-------------------------
|
||||
|
||||
It is an error to use ``Required[]`` or ``NotRequired[]`` with the special
|
||||
``__extra__`` item. ``total=False`` and ``total=True`` have no effect on
|
||||
``__extra__`` itself.
|
||||
``__extra_items__`` item. ``total=False`` and ``total=True`` have no effect on
|
||||
``__extra_items__`` itself.
|
||||
|
||||
The extra items are non-required, regardless of the totality of the TypedDict.
|
||||
Operations that are available to ``NotRequired`` items should also be available
|
||||
to the extra items::
|
||||
|
||||
class Movie(TypedDict):
|
||||
class Movie(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: int
|
||||
__extra_items__: int
|
||||
|
||||
def f(movie: Movie) -> None:
|
||||
del movie["name"] # Not OK
|
||||
|
@ -245,50 +286,75 @@ For type checking purposes, ``Unpack[TypedDict]`` with extra items should be
|
|||
treated as its equivalent in regular parameters, and the existing rules for
|
||||
function parameters still apply::
|
||||
|
||||
class Movie(TypedDict):
|
||||
class Movie(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: int
|
||||
__extra_items__: int
|
||||
|
||||
def f(**kwargs: Unpack[Movie]) -> None: ...
|
||||
|
||||
# Should be equivalent to
|
||||
def f(*, name: str, **kwargs: int) -> None: ...
|
||||
|
||||
Interaction with PEP 705
|
||||
------------------------
|
||||
|
||||
When the special ``__extra_items__`` item is annotated with ``ReadOnly[]``, the
|
||||
extra items on the TypedDict have the properties of read-only items. This
|
||||
interacts with inheritance rules specified in :pep:`PEP 705 <705#Inheritance>`.
|
||||
|
||||
Notably, if the TypedDict type declares ``__extra_items__`` to be read-only, a
|
||||
subclass of the TypedDict type may redeclare ``__extra_items__``'s value type or
|
||||
additional non-extra items' value type.
|
||||
|
||||
Because a non-closed TypedDict type implicitly allows non-required extra items
|
||||
of value type ``ReadOnly[object]``, its subclass can override the special
|
||||
``__extra_items__`` with more specific types.
|
||||
|
||||
More details are discussed in the later sections.
|
||||
|
||||
Inheritance
|
||||
-----------
|
||||
|
||||
``__extra__`` is inherited the same way as a regular ``key: value_type`` item.
|
||||
As with the other keys, the same rules from
|
||||
When the TypedDict type is defined as ``closed=False`` (the default),
|
||||
``__extra_items__`` should behave and be inherited the same way a regular key
|
||||
would. A regular ``__extra_items__`` key can coexist with the special
|
||||
``__extra_items__`` and both should be inherited when subclassing.
|
||||
|
||||
We assume that ``closed=True`` whenever ``__extra_items__`` is mentioned for the
|
||||
rest of this section.
|
||||
|
||||
``__extra_items__`` is inherited the same way as a regular ``key: value_type``
|
||||
item. As with the other keys, the same rules from
|
||||
`the typing spec <https://typing.readthedocs.io/en/latest/spec/typeddict.html#inheritance>`__
|
||||
and :pep:`PEP 705 <705#inheritance>` apply. We interpret the existing rules in the
|
||||
context of ``__extra__``.
|
||||
context of ``__extra_items__``.
|
||||
|
||||
We need to reinterpret the following rule to define how ``__extra__`` interacts
|
||||
with it:
|
||||
We need to reinterpret the following rule to define how ``__extra_items__``
|
||||
interacts with it:
|
||||
|
||||
* Changing a field type of a parent TypedDict class in a subclass is not allowed.
|
||||
|
||||
First, it is not allowed to change the value type of ``__extra__`` in a subclass
|
||||
First, it is not allowed to change the value type of ``__extra_items__`` in a subclass
|
||||
unless it is declared to be ``ReadOnly`` in the superclass::
|
||||
|
||||
class Parent(TypedDict):
|
||||
__extra__: int | None
|
||||
class Parent(TypedDict, closed=True):
|
||||
__extra_items__: int | None
|
||||
|
||||
class Child(Parent):
|
||||
__extra__: int # Not OK. Like any other TypedDict item, __extra__'s type cannot be changed
|
||||
class Child(Parent, closed=True):
|
||||
__extra_items__: int # Not OK. Like any other TypedDict item, __extra_items__'s type cannot be changed
|
||||
|
||||
Second, ``__extra__: T`` effectively defines the value type of any unnamed items
|
||||
accepted to the TypedDict and marks them as non-required. Thus, the above
|
||||
Second, ``__extra_items__: T`` effectively defines the value type of any unnamed
|
||||
items accepted to the TypedDict and marks them as non-required. Thus, the above
|
||||
restriction applies to any additional items defined in a subclass. For each item
|
||||
added in a subclass, all of the following conditions should apply:
|
||||
|
||||
- If ``__extra__`` is read-only
|
||||
- If ``__extra_items__`` is read-only
|
||||
|
||||
- The item can be either required or non-required
|
||||
|
||||
- The item's value type is consistent with ``T``
|
||||
|
||||
- If ``__extra__`` is not read-only
|
||||
- If ``__extra_items__`` is not read-only
|
||||
|
||||
- The item is non-required
|
||||
|
||||
|
@ -296,13 +362,13 @@ added in a subclass, all of the following conditions should apply:
|
|||
|
||||
- ``T`` is consistent with the item's value type
|
||||
|
||||
- If ``__extra__`` is not redeclared, the subclass inherits it as-is.
|
||||
- If ``__extra_items__`` is not redeclared, the subclass inherits it as-is.
|
||||
|
||||
For example::
|
||||
|
||||
class MovieBase(TypedDict):
|
||||
class MovieBase(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: int | None
|
||||
__extra_items__: int | None
|
||||
|
||||
class AdaptedMovie(MovieBase): # Not OK. 'bool' is not consistent with 'int | None'
|
||||
adapted_from_novel: bool
|
||||
|
@ -319,27 +385,34 @@ For example::
|
|||
Due to this nature, an important side effect allows us to define a TypedDict
|
||||
type that disallows additional items::
|
||||
|
||||
class MovieFinal(TypedDict):
|
||||
class MovieFinal(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: Never
|
||||
__extra_items__: Never
|
||||
|
||||
Here, annotating ``__extra__`` with :class:`typing.Never` specifies that
|
||||
Here, annotating ``__extra_items__`` with :class:`typing.Never` specifies that
|
||||
there can be no other keys in ``MovieFinal`` other than the known ones.
|
||||
Because of its potential common use, this is equivalent to::
|
||||
|
||||
class MovieFinal(TypedDict, closed=True):
|
||||
name: str
|
||||
|
||||
where we implicitly assume the ``__extra_items__: Never`` field by default
|
||||
if only ``closed=True`` is specified.
|
||||
|
||||
Type Consistency
|
||||
----------------
|
||||
|
||||
In addition to the set ``S`` of keys of the explicitly defined items, a
|
||||
TypedDict type that has the item ``__extra__: T`` is considered to have an
|
||||
TypedDict type that has the item ``__extra_items__: T`` is considered to have an
|
||||
infinite set of items that all satisfy the following conditions:
|
||||
|
||||
- If ``__extra__`` is read-only
|
||||
- If ``__extra_items__`` is read-only
|
||||
|
||||
- The key's value type is consistent with ``T``
|
||||
|
||||
- The key is not in ``S``.
|
||||
|
||||
- If ``__extra__`` is not read-only
|
||||
- If ``__extra_items__`` is not read-only
|
||||
|
||||
- The key is non-required
|
||||
|
||||
|
@ -349,7 +422,7 @@ infinite set of items that all satisfy the following conditions:
|
|||
|
||||
- The key is not in ``S``.
|
||||
|
||||
For type checking purposes, let ``__extra__`` be a non-required pseudo-item to
|
||||
For type checking purposes, let ``__extra_items__`` be a non-required pseudo-item to
|
||||
be included whenever "for each ... item/key" is stated in
|
||||
:pep:`the existing type consistency rules from PEP 705 <705#type-consistency>`,
|
||||
and we modify it as follows:
|
||||
|
@ -361,64 +434,67 @@ and we modify it as follows:
|
|||
* For each item in ``B``, ``A`` has the corresponding key, unless the item
|
||||
in ``B`` is read-only, not required, and of top value type
|
||||
(``ReadOnly[NotRequired[object]]``). **[Edit: Otherwise, if the
|
||||
corresponding key with the same name cannot be found in ``A``, "__extra__"
|
||||
is considered the corresponding key.]**
|
||||
corresponding key with the same name cannot be found in ``A``,
|
||||
"__extra_items__" is considered the corresponding key.]**
|
||||
|
||||
* For each item in ``B``, if ``A`` has the corresponding key **[Edit: or
|
||||
"__extra__"]**, the corresponding value type in ``A`` is consistent with the
|
||||
value type in ``B``.
|
||||
"__extra_items__"]**, the corresponding value type in ``A`` is consistent
|
||||
with the value type in ``B``.
|
||||
|
||||
* For each non-read-only item in ``B``, its value type is consistent with
|
||||
the corresponding value type in ``A``. **[Edit: if the corresponding key
|
||||
with the same name cannot be found in ``A``, "__extra__" is considered the
|
||||
corresponding key.]**
|
||||
with the same name cannot be found in ``A``, "__extra_items__" is
|
||||
considered the corresponding key.]**
|
||||
|
||||
* For each required key in ``B``, the corresponding key is required in ``A``.
|
||||
For each non-required key in ``B``, if the item is not read-only in ``B``,
|
||||
the corresponding key is not required in ``A``.
|
||||
**[Edit: if the corresponding key with the same name cannot be found in
|
||||
``A``, "__extra__" is considered to be non-required as the corresponding
|
||||
key.]**
|
||||
``A``, "__extra_items__" is considered to be non-required as the
|
||||
corresponding key.]**
|
||||
|
||||
The following examples illustrate these checks in action.
|
||||
|
||||
``__extra__`` puts various restrictions on additional items for type
|
||||
``__extra_items__`` puts various restrictions on additional items for type
|
||||
consistency checks::
|
||||
|
||||
class Movie(TypedDict):
|
||||
class Movie(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: int | None
|
||||
__extra_items__: int | None
|
||||
|
||||
class MovieDetails(TypedDict):
|
||||
class MovieDetails(TypedDict, closed=True):
|
||||
name: str
|
||||
year: NotRequired[int]
|
||||
__extra_items__: int | None
|
||||
|
||||
details: MovieDetails = {"name": "Kill Bill Vol. 1", "year": 2003}
|
||||
movie: Movie = details # Not OK. While 'int' is consistent with 'int | None',
|
||||
# 'int | None' is not consistent with 'int'
|
||||
|
||||
class MovieWithYear(TypedDict):
|
||||
class MovieWithYear(TypedDict, closed=True):
|
||||
name: str
|
||||
year: int | None
|
||||
__extra_items__: int | None
|
||||
|
||||
details: MovieWithYear = {"name": "Kill Bill Vol. 1", "year": 2003}
|
||||
movie: Movie = details # Not OK. 'year' is not required in 'Movie',
|
||||
# so it shouldn't be required in 'MovieWithYear' either
|
||||
|
||||
Because "year" is absent in ``Movie``, ``__extra__`` is considered the
|
||||
Because "year" is absent in ``Movie``, ``__extra_items__`` is considered the
|
||||
corresponding key. ``"year"`` being required violates the rule "For each
|
||||
required key in ``B``, the corresponding key is required in ``A``".
|
||||
|
||||
When ``__extra__`` is defined to be read-only in a TypedDict type, it is possible
|
||||
for an item to have a narrower type than ``__extra__``'s value type::
|
||||
When ``__extra_items__`` is defined to be read-only in a TypedDict type, it is possible
|
||||
for an item to have a narrower type than ``__extra_items__``'s value type::
|
||||
|
||||
class Movie(TypedDict):
|
||||
class Movie(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: ReadOnly[str | int]
|
||||
__extra_items__: ReadOnly[str | int]
|
||||
|
||||
class MovieDetails(TypedDict):
|
||||
class MovieDetails(TypedDict, closed=True):
|
||||
name: str
|
||||
year: NotRequired[int]
|
||||
__extra_items__: int
|
||||
|
||||
details: MovieDetails = {"name": "Kill Bill Vol. 2", "year": 2004}
|
||||
movie: Movie = details # OK. 'int' is consistent with 'str | int'.
|
||||
|
@ -426,22 +502,34 @@ for an item to have a narrower type than ``__extra__``'s value type::
|
|||
This behaves the same way as :pep:`705` specified if ``year: ReadOnly[str | int]``
|
||||
is an item defined in ``Movie``.
|
||||
|
||||
``__extra__`` as a pseudo-item follows the same rules that other items have, so
|
||||
when both TypedDicts contain ``__extra__``, this check is naturally enforced::
|
||||
``__extra_items__`` as a pseudo-item follows the same rules that other items have, so
|
||||
when both TypedDicts contain ``__extra_items__``, this check is naturally enforced::
|
||||
|
||||
class MovieExtraInt(TypedDict):
|
||||
class MovieExtraInt(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: int
|
||||
__extra_items__: int
|
||||
|
||||
class MovieExtraStr(TypedDict):
|
||||
class MovieExtraStr(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: str
|
||||
__extra_items__: str
|
||||
|
||||
extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
|
||||
extra_str: MovieExtraStr = {"name": "No Country for Old Men", "description": ""}
|
||||
extra_int = extra_str # Not OK. 'str' is inconsistent with 'int' for item '__extra__'
|
||||
extra_str = extra_int # Not OK. 'int' is inconsistent with 'str' for item '__extra__'
|
||||
extra_int = extra_str # Not OK. 'str' is inconsistent with 'int' for item '__extra_items__'
|
||||
extra_str = extra_int # Not OK. 'int' is inconsistent with 'str' for item '__extra_items__'
|
||||
|
||||
A non-closed TypedDict type implicitly allows non-required extra keys of value
|
||||
type ``ReadOnly[object]``. This allows to apply the type consistency rules
|
||||
between this type and a closed TypedDict type::
|
||||
|
||||
class MovieNotClosed(TypedDict):
|
||||
name: str
|
||||
|
||||
extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
|
||||
not_closed: MovieNotClosed = {"name": "No Country for Old Men"}
|
||||
extra_int = not_closed # Not OK. 'ReadOnly[object]' implicitly on 'MovieNotClosed' is not consistent with 'int' for item '__extra_items__'
|
||||
not_closed = extra_int # OK
|
||||
|
||||
Interaction with Mapping[KT, VT]
|
||||
--------------------------------
|
||||
|
||||
|
@ -458,9 +546,9 @@ spec:
|
|||
|
||||
For example::
|
||||
|
||||
class MovieExtraStr(TypedDict):
|
||||
class MovieExtraStr(TypedDict, closed=True):
|
||||
name: str
|
||||
__extra__: str
|
||||
__extra_items__: str
|
||||
|
||||
extra_str: MovieExtraStr = {"name": "Blade Runner", "summary": ""}
|
||||
str_mapping: Mapping[str, str] = extra_str # OK
|
||||
|
@ -478,10 +566,10 @@ Furthermore, type checkers should be able to infer the precise return types of
|
|||
Interaction with dict[KT, VT]
|
||||
-----------------------------
|
||||
|
||||
Note that because the presence of ``__extra__`` prohibits additional required
|
||||
keys in a TypedDict type's structural subtypes, we can determine if the
|
||||
TypedDict type and its structural subtypes will ever have any required key
|
||||
during static analysis.
|
||||
Note that because the presence of ``__extra_items__`` on a closed TypedDict type
|
||||
prohibits additional required keys in its structural subtypes, we can determine
|
||||
if the TypedDict type and its structural subtypes will ever have any required
|
||||
key during static analysis.
|
||||
|
||||
If there is no required key, the TypedDict type is consistent with ``dict[KT,
|
||||
VT]`` and vice versa if all items on the TypedDict type satisfy the following
|
||||
|
@ -493,8 +581,8 @@ conditions:
|
|||
|
||||
For example::
|
||||
|
||||
class IntDict(TypedDict):
|
||||
__extra__: int
|
||||
class IntDict(TypedDict, closed=True):
|
||||
__extra_items__: int
|
||||
|
||||
class IntDictWithNum(IntDict):
|
||||
num: NotRequired[int]
|
||||
|
@ -513,27 +601,31 @@ In this case, methods that are previously unavailable on a TypedDict are allowed
|
|||
|
||||
reveal_type(not_required_num.popitem()) # OK. Revealed type is tuple[str, int]
|
||||
|
||||
Open Issues
|
||||
===========
|
||||
How to Teach this
|
||||
=================
|
||||
|
||||
Alternatives to the ``__extra__`` Reserved Key
|
||||
----------------------------------------------
|
||||
The choice of ``"__extra_items__"`` and the requirement of ``closed=True``
|
||||
whenever it is used as a special key intended to make it more understandable to
|
||||
new users even without former knowledge of this feature.
|
||||
|
||||
As it was pointed out in the `PEP 705 review comment
|
||||
<https://discuss.python.org/t/pep-705-typeddict-read-only-and-other-keys/36457/6>`__,
|
||||
``__extra__`` as a reserved item has some disadvantages, including not allowing
|
||||
"__extra__" as a regular key, requiring special handling to disallow
|
||||
``Required`` and ``NotRequired``. There could be some better alternatives to
|
||||
this without the above-mentioned issues.
|
||||
Details of this should be documented in both the typing spec and the
|
||||
:mod:`typing` documentation.
|
||||
|
||||
Backwards Compatibility
|
||||
=======================
|
||||
|
||||
While dunder attributes like ``__extra__`` are reserved for stdlib, it is still
|
||||
a limitation that ``__extra__`` is no longer usable as a regular key. If the
|
||||
proposal is accepted, none of ``__required_keys__``, ``__optional_keys__``,
|
||||
``__readonly_keys__`` and ``__mutable_keys__`` should include ``__extra__`` in
|
||||
runtime.
|
||||
Because ``__extra_items__`` remains as a regular key if ``closed=True`` is not
|
||||
specified, no existing codebase will break due to this change.
|
||||
|
||||
If the proposal is accepted, none of ``__required_keys__``,
|
||||
``__optional_keys__``, ``__readonly_keys__`` and ``__mutable_keys__`` should
|
||||
include ``"__extra_items__"`` defined on the same TypedDict type when
|
||||
``closed=True`` is specified.
|
||||
|
||||
Note that ``closed`` as a keyword argument does not collide with the keyword
|
||||
arguments alternative to define keys with the functional syntax that allows
|
||||
things like ``TD = TypedDict("TD", foo=str, bar=int)``, because it is scheduled
|
||||
to be removed in Python 3.13.
|
||||
|
||||
Because this is a type-checking feature, it can be made available to older
|
||||
versions as long as the type checker supports it.
|
||||
|
@ -554,24 +646,40 @@ Because it did not offer a way to specify the type of the extra items, the type
|
|||
checkers will need to assume that the type of the extra items is ``Any``, which
|
||||
compromises type safety. Furthermore, the current behavior of TypedDict already
|
||||
allows untyped extra items to be present in runtime, due to structural
|
||||
subtyping.
|
||||
subtyping. ``closed=True`` plays a similar role in the current proposal.
|
||||
|
||||
Supporting ``TypedDict(extra=type)``
|
||||
------------------------------------
|
||||
|
||||
This adds more corner cases to determine whether a type should be treated as a
|
||||
type or a value. And it will require more work to support using special forms to
|
||||
type the extra items.
|
||||
While this design is potentially viable, there are several partially addressable
|
||||
concerns to consider. Adding everything up, it is slightly less favorable than
|
||||
the current proposal.
|
||||
|
||||
While this saves us from reserving an attribute for special use, it will require
|
||||
extra work to implement inheritance, and it is less natural to integrate with
|
||||
generic TypedDicts.
|
||||
- Usability of forward reference
|
||||
As in the functional syntax, using a quoted type or a type alias will be
|
||||
required when SomeType is a forward reference. This is already a requirement
|
||||
for the functional syntax, so implementations can potentially reuse that piece
|
||||
of logic, but this is still extra work that the ``closed=True`` proposal doesn't
|
||||
have.
|
||||
|
||||
- Concerns about using type as a value
|
||||
Whatever is not allowed as the value type in the functional syntax should not
|
||||
be allowed as the argument for extra either. While type checkers might be able
|
||||
to reuse this check, it still needs to be somehow special-cased for the
|
||||
class-based syntax.
|
||||
|
||||
- How to teach
|
||||
Notably, the ``extra=type`` often gets brought up due to it being an intuitive
|
||||
solution for the use case, so it is potentially simpler to learn than the less
|
||||
obvious solution. However, the more common used case only requires
|
||||
``closed=True``, and the other drawbacks mentioned earlier outweigh what is
|
||||
need to teach the usage of the special key.
|
||||
|
||||
Support Extra Items with Intersection
|
||||
-------------------------------------
|
||||
|
||||
Supporting intersections in Python's type system requires a lot of careful
|
||||
considerations, and it can take a long time for the community to reach a
|
||||
consideration, and it can take a long time for the community to reach a
|
||||
consensus on a reasonable design.
|
||||
|
||||
Ideally, extra items in TypedDict should not be blocked by work on
|
||||
|
@ -579,17 +687,17 @@ intersections, nor does it necessarily need to be supported through
|
|||
intersections.
|
||||
|
||||
Moreover, the intersection between ``Mapping[...]`` and ``TypedDict`` is not
|
||||
equivalent to a TypedDict type with the proposed ``__extra__`` special item, as
|
||||
the value type of all known items in ``TypedDict`` needs to satisfy the
|
||||
equivalent to a TypedDict type with the proposed ``__extra_items__`` special
|
||||
item, as the value type of all known items in ``TypedDict`` needs to satisfy the
|
||||
is-subtype-of relation with the value type of ``Mapping[...]``.
|
||||
|
||||
Requiring Type Compatibility of the Known Items with ``__extra__``
|
||||
------------------------------------------------------------------
|
||||
Requiring Type Compatibility of the Known Items with ``__extra_items__``
|
||||
------------------------------------------------------------------------
|
||||
|
||||
``__extra__`` restricts the value type for keys that are *unknown* to the
|
||||
``__extra_items__`` restricts the value type for keys that are *unknown* to the
|
||||
TypedDict type. So the value type of any *known* item is not necessarily
|
||||
consistent with ``__extra__``'s type, and ``__extra__``'s type is not
|
||||
necessarily consistent with the value types of all known items.
|
||||
consistent with ``__extra_items__``'s type, and ``__extra_items__``'s type is
|
||||
not necessarily consistent with the value types of all known items.
|
||||
|
||||
This differs from TypeScript's `Index Signatures
|
||||
<https://www.typescriptlang.org/docs/handbook/2/objects.html#index-signatures>`__
|
||||
|
@ -611,7 +719,7 @@ For example:
|
|||
This is a known limitation discussed in `TypeScript's issue tracker
|
||||
<https://github.com/microsoft/TypeScript/issues/17867>`__,
|
||||
where it is suggested that there should be a way to exclude the defined keys
|
||||
from the index signature, so that it is possible to define a type like
|
||||
from the index signature so that it is possible to define a type like
|
||||
``MovieWithExtraNumber``.
|
||||
|
||||
Reference Implementation
|
||||
|
|
Loading…
Reference in New Issue