python-peps/peps/pep-0718.rst

233 lines
6.9 KiB
ReStructuredText

PEP: 718
Title: Subscriptable functions
Author: James Hilton-Balfe <gobot1234yt@gmail.com>
Sponsor: Guido van Rossum <guido@python.org>
Discussions-To: https://discuss.python.org/t/28457/
Status: Draft
Type: Standards Track
Topic: Typing
Content-Type: text/x-rst
Created: 23-Jun-2023
Python-Version: 3.13
Post-History: `24-Jun-2023 <https://discuss.python.org/t/28457/>`__
Abstract
--------
This PEP proposes making function objects subscriptable for typing purposes. Doing so
gives developers explicit control over the types produced by the type checker where
bi-directional inference (which allows for the types of parameters of anonymous
functions to be inferred) and other methods than specialisation are insufficient. It
also brings functions in line with regular classes in their ability to be
subscriptable.
Motivation
----------
Unknown Types
^^^^^^^^^^^^^
Currently, it is not possible to infer the type parameters to generic functions in
certain situations:
.. code-block:: python
def make_list[T](*args: T) -> list[T]: ...
reveal_type(make_list()) # type checker cannot infer a meaningful type for T
Making instances of ``FunctionType`` subscriptable would allow for this constructor to
be typed:
.. code-block:: python
reveal_type(make_list[int]()) # type is list[int]
Currently you have to use an assignment to provide a precise type:
.. code-block:: python
x: list[int] = make_list()
reveal_type(x) # type is list[int]
but this code is unnecessarily verbose taking up multiple lines for a simple function
call.
Similarly, ``T`` in this example cannot currently be meaningfully inferred, so ``x`` is
untyped without an extra assignment:
.. code-block:: python
def factory[T](func: Callable[[T], Any]) -> Foo[T]: ...
reveal_type(factory(lambda x: "Hello World" * x))
If function objects were subscriptable, however, a more specific type could be given:
.. code-block:: python
reveal_type(factory[int](lambda x: "Hello World" * x)) # type is Foo[int]
Undecidable Inference
^^^^^^^^^^^^^^^^^^^^^
There are even cases where subclass relations make type inference impossible. However,
if you can specialise the function type checkers can infer a meaningful type.
.. code-block:: python
def foo[T](x: Sequence[T] | T) -> list[T]: ...
reveal_type(foo[bytes](b"hello"))
Currently, type checkers do not consistently synthesise a type here.
Unsolvable Type Parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^
Currently, with unspecialised literals, it is not possible to determine a type for
situations similar to:
.. code-block:: python
def foo[T](x: list[T]) -> T: ...
reveal_type(foo([])) # type checker cannot infer T (yet again)
.. code-block:: python
reveal_type(foo[int]([])) # type is int
It is also useful to be able to specify in cases in which a certain type must be passed
to a function beforehand:
.. code-block:: python
words = ["hello", "world"]
foo[int](words) # Invalid: list[str] is incompatible with list[int]
Allowing subscription makes functions and methods consistent with generic classes where
they weren't already. Whilst all of the proposed changes can be implemented using
callable generic classes, syntactic sugar would be highly welcome.
Due to this, specialising the function and using it as a new factory is fine
.. code-block:: python
make_int_list = make_list[int]
reveal_type(make_int_list()) # type is list[int]
Monomorphisation and Reification
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This proposal also opens the door to
`monomorphisation <https://en.wikipedia.org/wiki/Monomorphization>`_ and
`reified types <https://en.wikipedia.org/wiki/Reification_(computer_science)>`_.
This would allow for a functionality which anecdotally has been requested many times.
*Please note this feature is not being proposed by the PEP, but may be implemented in
the future.*
The syntax for such a feature may look something like:
.. code-block:: python
def foo[T]():
return T.__value__
assert foo[int]() is int
Rationale
---------
Function objects in this PEP is used to refer to ``FunctionType``\ , ``MethodType``\ ,
``BuiltinFunctionType``\ , ``BuiltinMethodType`` and ``MethodWrapperType``\ .
For ``MethodType`` you should be able to write:
.. code-block:: python
class Foo:
def make_list[T](self, *args: T) -> list[T]: ...
Foo().make_list[int]()
and have it work similarly to a ``FunctionType``.
For ``BuiltinFunctionType``, so builtin generic functions (e.g. ``max`` and ``min``)
work like ones defined in Python. Built-in functions should behave as much like
functions implemented in Python as possible.
``BuiltinMethodType`` is the same type as ``BuiltinFunctionType``.
``MethodWrapperType`` (e.g. the type of ``object().__str__``) is useful for
generic magic methods.
Specification
-------------
Function objects should implement ``__getitem__`` to allow for subscription at runtime
and return an instance of ``types.GenericAlias`` with ``__origin__`` set as the
callable and ``__args__`` as the types passed.
Type checkers should support subscripting functions and understand that the parameters
passed to the function subscription should follow the same rules as a generic callable
class.
Setting ``__orig_class__``
^^^^^^^^^^^^^^^^^^^^^^^^^^
Currently, ``__orig_class__`` is an attribute set in ``GenericAlias.__call__`` to the
instance of the ``GenericAlias`` that created the called class e.g.
.. code-block:: python
class Foo[T]: ...
assert Foo[int]().__orig_class__ == Foo[int]
Currently, ``__orig_class__`` is unconditionally set; however, to avoid potential
erasure on any created instances, this attribute should not be set if ``__origin__`` is
an instance of any function object.
The following code snippet would fail at runtime without this change as
``__orig_class__`` would be ``bar[str]`` and not ``Foo[int]``.
.. code-block:: python
def bar[U]():
return Foo[int]()
assert bar[str]().__orig_class__ == Foo[int]
Interactions with ``@typing.overload``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Overloaded functions should work much the same as already, since they have no effect on
the runtime type. The only change is that more situations will be decidable and the
behaviour/overload can be specified by the developer rather than leaving it to ordering
of overloads/unions.
Backwards Compatibility
-----------------------
Currently these classes are not subclassable and so there are no backwards
compatibility concerns with regards to classes already implementing
``__getitem__``.
Reference Implementation
------------------------
The runtime changes proposed can be found here
https://github.com/Gobot1234/cpython/tree/function-subscript
Acknowledgements
----------------
Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido
for some motivating examples.
Copyright
---------
This document is placed in the public domain or under the CC0-1.0-Universal license,
whichever is more permissive.