python-peps/peps/pep-0713.rst

PEP: 713
Title: Callable Modules
Author: Amethyst Reese <amethyst at n7.gg>
Sponsor: Łukasz Langa <lukasz at python.org>
Discussions-To: https://discuss.python.org/t/pep-713-callable-modules/26127
Status: Rejected
Type: Standards Track
Content-Type: text/x-rst
Created: 20-Apr-2023
Python-Version: 3.12
Post-History: `23-Apr-2023 <https://discuss.python.org/t/pep-713-callable-modules/26127>`__
Resolution: https://discuss.python.org/t/26127/86


Rejection Notice
================

The Steering Council didn't feel that there was a compelling reason to
have this PEP, even though it clearly could be done from a consistency
point of view.
If this idea comes up again in the future, this is a useful prior
discussion to refer to.


Abstract
========

Modules are currently not directly callable. Classes can define a ``__call__``
method that makes instance objects callable, but defining a similarly named
function in the global module scope has no effect, and that function can
only be called by importing or referencing it directly as ``module.__call__``.
:pep:`562` added support for :meth:`~object.__getattr__` and :meth:`~object.__dir__` for modules, but
defining ``__getattr__`` to return a value for ``__call__`` still does not
make a module callable.

This PEP proposes support for making modules directly callable by defining
a ``__call__`` object in the module's global namespace, either as a standard
function, or an arbitrary callable object.


Motivation
==========

Many modules have only a single primary interface to their functionality.
In many cases, that interface is a single callable object, where being able
to import and use the module directly as a callable provides a more "Pythonic"
interface for users::

    # user.py

    import fancy

    @fancy
    def func(...):
        ...

Currently, providing this style of interface requires modifying the module
object at runtime to make it callable.

This is commonly done by replacing the module object in ``sys.modules`` with
a callable alternative (such as a function or class instance)::

    # fancy.py

    def fancy(...):
        ...

    sys.modules[__name__] = fancy

This has the effect of making the original module effectively unreachable
without further hooks from the author, even with ``from module import member``.
It also results in a "module" object that is missing all of the special module
attributes, including ``__doc__``, ``__package__``, ``__path__``, etc.

Alternatively, a module author can choose to override the module's ``__class__``
property with a custom type that provides a callable interface::

    # fancy.py

    def fancy(...):
        ...

    class FancyModule(types.ModuleType):
        def __call__(self, ...):
            return fancy(...)

    sys.modules[__name__].__class__ = FancyModule

The downside of either approach is that it not only results in extra
boilerplate, but also results in type checker failures because they don't
recognize that the module is callable at runtime:

.. code-block:: console

    $ mypy user.py
    user.py:3: error: Module not callable  [operator]
    Found 1 error in 1 file (checked 1 source file)


Specification
=============

When a module object is called, and a ``__call__`` object is found (either
as the result of a ``__getattr__`` or ``__dict__`` lookup), then that object
will be called with the given arguments.

If a ``__call__`` object is not found, then a ``TypeError`` will be raised,
matching the existing behavior.

All of these examples would be considered valid, callable modules:

.. code-block:: python

    # hello.py

    def __call__(...):
        pass

.. code-block:: python

    # hello.py

    class Hello:
        pass

    __call__ = Hello

.. code-block:: python

    # hello.py

    def hello():
        pass

    def __getattr__(name):
        if name == "__call__":
            return hello

The first two styles should generally be preferred, as it allows for easier
static analysis from tools like type checkers, though the third form would be
allowed in order to make the implementation more consistent.

The intent is to allow arbitrary callable object to be assigned to the module's
``__call__`` property or returned by the module's ``__getattr__`` method,
enabling module authors to pick the most suitable mechanism for making their
module callable by users.


Backwards Compatibility and Impact on Performance
=================================================

This PEP is not expected to cause any backwards incompatibility. Any modules
that already contain a ``__call__`` object will continue to function the same
as before, though with the additional ability to be called directly. It is
considered unlikely that modules with an existing ``__call__`` object would
depend on the existing behavior of raising ``TypeError`` when called.

Performance implications of this PEP are minimal, as it defines a new interface.
Calling a module would trigger a lookup for the name ``__call__`` on a module
object. Existing workarounds for creating callable modules already depend on
this behavior for generic objects, resulting in similar performance for these
callable modules.

Type checkers will likely need to be updated accordingly to treat modules with
a ``__call__`` object as callable. This should be possible to support in type
checkers when checking code targeted at older Python versions that do not
support callable modules, with the expectation that these modules would also
include one of the workarounds mentioned earlier to make the module callable.


How to Teach This
=================

The documentation for :external+python:ref:`callable types <types>` will
include modules in the list, with a link to :meth:`~object.__call__`.
The :external+python:ref:`callable-types` documentation will include a section
covering callable modules, with example code, similar to the section for
`customizing module attribute access`__.

__ https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access


Reference Implementation
========================

The proposed implementation of callable modules is available in
`CPython PR #103742 <https://github.com/python/cpython/pull/103742>`_.


Rejected Ideas
==============

Given the introduction of ``__getattr__`` and ``__dir__``, and the proposal
to enable use of ``__call__``, it was considered if it was worth allowing use
of *all* :external+python:ref:`specialnames` for modules, such as ``__or__``
and ``__iter__``. While this would not be completely undesired, it increases
the potential for backward compatibility concerns, and these other special
methods are likely to provide less utility to library authors in comparison
to ``__call__``.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.