iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 692: Introduce proposed enhancements and expand Motivation section (#2732) 

Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This commit is contained in:
Franek Magiera 2022-08-23 23:07:52 +02:00 committed by GitHub
parent b62ca71d52
commit 4f34145396
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 63 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

103
pep-0692.rst
View File

@ -2,13 +2,15 @@ PEP: 692
Title: Using TypedDict for more precise \*\*kwargs typing
Author: Franek Magiera <framagie@gmail.com>
Sponsor: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/U42MJE6QZYWPVIFHJIGIT7OE52ZGIQV3/
Discussions-To: https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwargs-typing/17314
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 29-May-2022
Python-Version: 3.12
Post-History: `29-May-2022 <https://mail.python.org/archives/list/typing-sig@python.org/thread/U42MJE6QZYWPVIFHJIGIT7OE52ZGIQV3/>`__,
`12-Jul-2022 <https://mail.python.org/archives/list/python-dev@python.org/thread/PLCNW2XR4OOKAKHEZQM7R2AYVYUXPZGW/>`__,
`12-Jul-2022 <https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwargs-typing/17314>`__,
Abstract
@ -20,7 +22,9 @@ be very limiting. Therefore, in this PEP we propose a new way to enable more
precise ``**kwargs`` typing. The new approach revolves around using
``TypedDict`` to type ``**kwargs`` that comprise keyword arguments of different
types. It also involves introducing a grammar change and a new dunder
``__unpack__``.
``__typing_unpack__``.
.. _pep-692-motivation:
Motivation
==========
@ -39,9 +43,31 @@ type annotating ``**kwargs`` is not possible. This is especially a problem for
already existing codebases where the need of refactoring the code in order to
introduce proper type annotations may be considered not worth the effort. This
in turn prevents the project from getting all of the benefits that type hinting
can provide. As a consequence, there has been a `lot of discussion <mypyIssue4441_>`__
around supporting more precise ``**kwargs`` typing and it became a
feature that would be valuable for a large part of the Python community.
can provide.
Moreover, ``**kwargs`` can be used to reduce the amount of code needed in
cases when there is a top-level function that is a part of a public API and it
calls a bunch of helper functions, all of which expect the same keyword
arguments. Unfortunately, if those helper functions were to use ``**kwargs``,
there is no way to properly type hint them if the keyword arguments they expect
are of different types. In addition, even if the keyword arguments are of the
same type, there is no way to check whether the function is being called with
keyword names that it actually expects.
As described in the :ref:`Intended Usage <pep-692-intended-usage>` section,
using ``**kwargs`` is not always the best tool for the job. Despite that, it is
still a widely used pattern. As a consequence, there has been a lot of
discussion around supporting more precise ``**kwargs`` typing and it became a
feature that would be valuable for a large part of the Python community. This
is best illustrated by the `mypy GitHub issue 4441 <mypyIssue4441_>`__ which
contains a lot of real world cases that could benefit from this propsal.
One more use case worth mentioning for which ``**kwargs`` are also convenient,
is when a function should accommodate optional keyword-only arguments that
don't have default values. A need for a pattern like that can arise when values
that are usually used as defaults to indicate no user input, such as ``None``,
can be passed in by a user and should result in a valid, non-default behavior.
For example, this issue `came up <httpxIssue1384_>`__ in the popular ``httpx`` library.
Rationale
=========
@ -123,8 +149,8 @@ Keyword collisions
A ``TypedDict`` that is used to type ``**kwargs`` could potentially contain
keys that are already defined in the function's signature. If the duplicate
name is a standard argument, an error should be reported by type checkers.
If the duplicate name is a positional only argument, no errors should be
name is a standard parameter, an error should be reported by type checkers.
If the duplicate name is a positional-only parameter, no errors should be
generated. For example::
def foo(name, **kwargs: **Movie) -> None: ... # WRONG! "name" will
@ -132,7 +158,7 @@ generated. For example::
# first parameter.
def foo(name, /, **kwargs: **Movie) -> None: ... # OK! "name" is a
# positional argument,
# positional parameter,
# so **kwargs can contain
# a "name" keyword.
@ -207,9 +233,9 @@ Continuing the previous example::
dest = src # OK!
It is worth pointing out that the destination function's arguments that are to
It is worth pointing out that the destination function's parameters that are to
be compatible with the keys and values from the ``TypedDict`` must be keyword
only arguments::
only::
def dest(animal: Dog, string: str, number: int = ...): ...
dest(animal_instance, "some string") # OK!
@ -336,35 +362,26 @@ would not cause errors at runtime during function invocation. Otherwise, the
type checker should generate an error.
In cases similar to the ``bar`` function above the problem could be worked
around by explicitly dereferencing desired fields and using them as parameters
around by explicitly dereferencing desired fields and using them as arguments
to perform the function call::
def bar(**kwargs: **Animal):
name = kwargs["name"]
takes_name(name)
.. _pep-692-intended-usage:
Intended Usage
--------------
This proposal will bring a large benefit to the codebases that already use
``**kwargs`` because of the flexibility that they provided in the initial
phases of the development, but now are mature enough to use a stricter
contract via type hints.
Adding type hints directly in the source code as opposed to the ``*.pyi``
stubs benefits anyone who reads the code as it is easier to understand. Given
that currently precise ``**kwargs`` type hinting is impossible in that case the
choices are to either not type hint ``**kwargs`` at all, which isn't ideal, or
to refactor the function to use explicit keyword arguments, which often exceeds
the scope of time and effort allocated to adding type hinting and, as any code
change, introduces risk for both project maintainers and users. In that case
hinting ``**kwargs`` using a ``TypedDict`` as described in this PEP will not
require refactoring and function body and function invocations could be
appropriately type checked.
Another useful pattern that justifies using and typing ``**kwargs`` as proposed
is when the function's API should allow for optional keyword arguments that
don't have default values.
The intended use cases for this proposal are described in the
:ref:`pep-692-motivation` section. In summary, more precise ``**kwargs`` typing
can bring benefits to already existing codebases that decided to use
``**kwargs`` initially, but now are mature enough to use a stricter contract
via type hints. Using ``**kwargs`` can also help in reducing code duplication
and the amount of copy-pasting needed when there is a bunch of functions that
require the same set of keyword arguments. Finally, ``**kwargs`` are useful for
cases when a function needs to facilitate optional keyword arguments that don't
have obvious default values.
However, it has to be pointed out that in some cases there are better tools
for the job than using ``TypedDict`` to type ``**kwargs`` as proposed in this
@ -377,9 +394,9 @@ than using ``**kwargs`` and a ``TypedDict``::
Similarly, when type hinting third party libraries via stubs it is again better
to state the function signature explicitly - this is the only way to type such
a function if it has default parameters. Another issue that may arise in this
a function if it has default arguments. Another issue that may arise in this
case when trying to type hint the function with a ``TypedDict`` is that some
standard function arguments may be treated as keyword only::
standard function parameters may be treated as keyword only::
def foo(name, year): ... # Function in a third party library.
@ -397,6 +414,9 @@ explicitly as::
def foo(name: str, year: int): ...
Also, for the benefit of IDEs and documentation pages, functions that are part
of the public API should prefer explicit keyword parameters whenever possible.
Grammar Changes
===============
@ -420,7 +440,7 @@ After:
| '**' param_no_default
param_no_default_double_star_annotation:
| param_double_star_annotation & ')'
| param_double_star_annotation ','? &')'
param_double_star_annotation: NAME double_star_annotation
@ -463,13 +483,15 @@ previous example::
>>> def foo(**kwargs: **Movie): ...
...
>>> foo.__annotations__
{'kwargs': **Movie}
{'kwargs': Unpack[Movie]}
The double asterisk syntax should call the ``__unpack__`` special method on
the object it was used on. This means that ``def foo(**kwargs: **T): ...`` is
equivalent to ``def foo(**kwargs: T.__unpack__()): ...``. In addition,
``**Movie`` in the example above is the ``repr`` of the object that
``__unpack__()`` returns.
To accomplish this, we propose a new dunder called ``__typing_unpack__``.
The double asterisk syntax should result in a call to the ``__typing_unpack__``
special method on an object it was used on. This means that at runtime,
``def foo(**kwargs: **T): ...`` is equivalent to
``def foo(**kwargs: type(T).__typing_unpack__(T)): ...``.
``TypedDict`` is the only type in the standard library that is expected to
implement ``__typing_unpack__``, which should return ``Unpack[self]``.
Backwards Compatibility
-----------------------
@ -558,6 +580,7 @@ overloaded::
References
==========
.. _httpxIssue1384: https://github.com/encode/httpx/issues/1384
.. _mypyIssue4441: https://github.com/python/mypy/issues/4441
.. _mypyPull10576: https://github.com/python/mypy/pull/10576
.. _mypyExtensionsPull22: https://github.com/python/mypy_extensions/pull/22/files