PEP: 718 Title: Subscriptable functions Author: James Hilton-Balfe Sponsor: Guido van Rossum Discussions-To: https://discuss.python.org/t/28457/ Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 23-Jun-2023 Python-Version: 3.13 Post-History: `24-Jun-2023 `__ Abstract -------- This PEP proposes making function objects subscriptable for typing purposes. Doing so gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient. It also brings functions in line with regular classes in their ability to be subscriptable. Motivation ---------- Unknown Types ^^^^^^^^^^^^^ Currently, it is not possible to infer the type parameters to generic functions in certain situations: .. code-block:: python def make_list[T](*args: T) -> list[T]: ... reveal_type(make_list()) # type checker cannot infer a meaningful type for T Making instances of ``FunctionType`` subscriptable would allow for this constructor to be typed: .. code-block:: python reveal_type(make_list[int]()) # type is list[int] Currently you have to use an assignment to provide a precise type: .. code-block:: python x: list[int] = make_list() reveal_type(x) # type is list[int] but this code is unnecessarily verbose taking up multiple lines for a simple function call. Similarly, ``T`` in this example cannot currently be meaningfully inferred, so ``x`` is untyped without an extra assignment: .. code-block:: python def factory[T](func: Callable[[T], Any]) -> Foo[T]: ... reveal_type(factory(lambda x: "Hello World" * x)) If function objects were subscriptable, however, a more specific type could be given: .. code-block:: python reveal_type(factory[int](lambda x: "Hello World" * x)) # type is Foo[int] Undecidable Inference ^^^^^^^^^^^^^^^^^^^^^ There are even cases where subclass relations make type inference impossible. However, if you can specialise the function type checkers can infer a meaningful type. .. code-block:: python def foo[T](x: Sequence[T] | T) -> list[T]: ... reveal_type(foo[bytes](b"hello")) Currently, type checkers do not consistently synthesise a type here. Unsolvable Type Parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^ Currently, with unspecialised literals, it is not possible to determine a type for situations similar to: .. code-block:: python def foo[T](x: list[T]) -> T: ... reveal_type(foo([])) # type checker cannot infer T (yet again) .. code-block:: python reveal_type(foo[int]([])) # type is int It is also useful to be able to specify in cases in which a certain type must be passed to a function beforehand: .. code-block:: python words = ["hello", "world"] foo[int](words) # Invalid: list[str] is incompatible with list[int] Allowing subscription makes functions and methods consistent with generic classes where they weren't already. Whilst all of the proposed changes can be implemented using callable generic classes, syntactic sugar would be highly welcome. Due to this, specialising the function and using it as a new factory is fine .. code-block:: python make_int_list = make_list[int] reveal_type(make_int_list()) # type is list[int] Monomorphisation and Reification ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This proposal also opens the door to `monomorphisation `_ and `reified types `_. This would allow for a functionality which anecdotally has been requested many times. *Please note this feature is not being proposed by the PEP, but may be implemented in the future.* The syntax for such a feature may look something like: .. code-block:: python def foo[T](): return T.__value__ assert foo[int]() is int Rationale --------- Function objects in this PEP is used to refer to ``FunctionType``\ , ``MethodType``\ , ``BuiltinFunctionType``\ , ``BuiltinMethodType`` and ``MethodWrapperType``\ . For ``MethodType`` you should be able to write: .. code-block:: python class Foo: def make_list[T](self, *args: T) -> list[T]: ... Foo().make_list[int]() and have it work similarly to a ``FunctionType``. For ``BuiltinFunctionType``, so builtin generic functions (e.g. ``max`` and ``min``) work like ones defined in Python. Built-in functions should behave as much like functions implemented in Python as possible. ``BuiltinMethodType`` is the same type as ``BuiltinFunctionType``. ``MethodWrapperType`` (e.g. the type of ``object().__str__``) is useful for generic magic methods. Specification ------------- Function objects should implement ``__getitem__`` to allow for subscription at runtime and return an instance of ``types.GenericAlias`` with ``__origin__`` set as the callable and ``__args__`` as the types passed. Type checkers should support subscripting functions and understand that the parameters passed to the function subscription should follow the same rules as a generic callable class. Setting ``__orig_class__`` ^^^^^^^^^^^^^^^^^^^^^^^^^^ Currently, ``__orig_class__`` is an attribute set in ``GenericAlias.__call__`` to the instance of the ``GenericAlias`` that created the called class e.g. .. code-block:: python class Foo[T]: ... assert Foo[int]().__orig_class__ == Foo[int] Currently, ``__orig_class__`` is unconditionally set; however, to avoid potential erasure on any created instances, this attribute should not be set if ``__origin__`` is an instance of any function object. The following code snippet would fail at runtime without this change as ``__orig_class__`` would be ``bar[str]`` and not ``Foo[int]``. .. code-block:: python def bar[U](): return Foo[int]() assert bar[str]().__orig_class__ == Foo[int] Interactions with ``@typing.overload`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Overloaded functions should work much the same as already, since they have no effect on the runtime type. The only change is that more situations will be decidable and the behaviour/overload can be specified by the developer rather than leaving it to ordering of overloads/unions. Backwards Compatibility ----------------------- Currently these classes are not subclassable and so there are no backwards compatibility concerns with regards to classes already implementing ``__getitem__``. Reference Implementation ------------------------ The runtime changes proposed can be found here https://github.com/Gobot1234/cpython/tree/function-subscript Acknowledgements ---------------- Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido for some motivating examples. Copyright --------- This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.