212 lines
6.2 KiB
ReStructuredText
212 lines
6.2 KiB
ReStructuredText
PEP: 562
|
||
Title: Module __getattr__ and __dir__
|
||
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
|
||
Status: Accepted
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 09-Sep-2017
|
||
Python-Version: 3.7
|
||
Post-History: 09-Sep-2017
|
||
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151033.html
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
It is proposed to support ``__getattr__`` and ``__dir__`` function defined
|
||
on modules to provide basic customization of module attribute access.
|
||
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
It is sometimes convenient to customize or otherwise have control over
|
||
access to module attributes. A typical example is managing deprecation
|
||
warnings. Typical workarounds are assigning ``__class__`` of a module object
|
||
to a custom subclass of ``types.ModuleType`` or replacing the ``sys.modules``
|
||
item with a custom wrapper instance. It would be convenient to simplify this
|
||
procedure by recognizing ``__getattr__`` defined directly in a module that
|
||
would act like a normal ``__getattr__`` method, except that it will be defined
|
||
on module *instances*. For example::
|
||
|
||
# lib.py
|
||
|
||
from warnings import warn
|
||
|
||
deprecated_names = ["old_function", ...]
|
||
|
||
def _deprecated_old_function(arg, other):
|
||
...
|
||
|
||
def __getattr__(name):
|
||
if name in deprecated_names:
|
||
warn(f"{name} is deprecated", DeprecationWarning)
|
||
return globals()[f"_deprecated_{name}"]
|
||
raise AttributeError(f"module {__name__} has no attribute {name}")
|
||
|
||
# main.py
|
||
|
||
from lib import old_function # Works, but emits the warning
|
||
|
||
Another widespread use case for ``__getattr__`` would be lazy submodule
|
||
imports. Consider a simple example::
|
||
|
||
# lib/__init__.py
|
||
|
||
import importlib
|
||
|
||
__all__ = ['submod', ...]
|
||
|
||
def __getattr__(name):
|
||
if name in __all__:
|
||
return importlib.import_module("." + name, __name__)
|
||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||
|
||
# lib/submod.py
|
||
|
||
print("Submodule loaded")
|
||
class HeavyClass:
|
||
...
|
||
|
||
# main.py
|
||
|
||
import lib
|
||
lib.submodule.HeavyClass # prints "Submodule loaded"
|
||
|
||
There is a related proposal PEP 549 that proposes to support instance
|
||
properties for a similar functionality. The difference is this PEP proposes
|
||
a faster and simpler mechanism, but provides more basic customization.
|
||
An additional motivation for this proposal is that PEP 484 already defines
|
||
the use of module ``__getattr__`` for this purpose in Python stub files,
|
||
see [1]_.
|
||
|
||
In addition, to allow modifying result of a ``dir()`` call on a module
|
||
to show deprecated and other dynamically generated attributes, it is
|
||
proposed to support module level ``__dir__`` function. For example::
|
||
|
||
# lib.py
|
||
|
||
deprecated_names = ["old_function", ...]
|
||
__all__ = ["new_function_one", "new_function_two", ...]
|
||
|
||
def new_function_one(arg, other):
|
||
...
|
||
def new_function_two(arg, other):
|
||
...
|
||
|
||
def __dir__():
|
||
return sorted(__all__ + deprecated_names)
|
||
|
||
# main.py
|
||
|
||
import lib
|
||
|
||
dir(lib) # prints ["new_function_one", "new_function_two", "old_function", ...]
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
The ``__getattr__`` function at the module level should accept one argument
|
||
which is the name of an attribute and return the computed value or raise
|
||
an ``AttributeError``::
|
||
|
||
def __getattr__(name: str) -> Any: ...
|
||
|
||
If an attribute is not found on a module object through the normal lookup
|
||
(i.e. ``object.__getattribute__``), then ``__getattr__`` is searched in
|
||
the module ``__dict__`` before raising an ``AttributeError``. If found, it is
|
||
called with the attribute name and the result is returned. Looking up a name
|
||
as a module global will bypass module ``__getattr__``. This is intentional,
|
||
otherwise calling ``__getattr__`` for builtins will significantly harm
|
||
performance.
|
||
|
||
The ``__dir__`` function should accept no arguments, and return
|
||
a list of strings that represents the names accessible on module::
|
||
|
||
def __dir__() -> List[str]: ...
|
||
|
||
If present, this function overrides the standard ``dir()`` search on
|
||
a module.
|
||
|
||
The reference implementation for this PEP can be found in [2]_.
|
||
|
||
|
||
Backwards compatibility and impact on performance
|
||
=================================================
|
||
|
||
This PEP may break code that uses module level (global) names ``__getattr__``
|
||
and ``__dir__``. (But the language reference explicitly reserves *all*
|
||
undocumented dunder names, and allows "breakage without warning"; see [3]_.)
|
||
The performance implications of this PEP are minimal, since ``__getattr__``
|
||
is called only for missing attributes.
|
||
|
||
Some tools that perform module attributes discovery might not expect
|
||
``__getattr__``. This problem is not new however, since it is already possible
|
||
to replace a module with a module subclass with overridden ``__getattr__`` and
|
||
``__dir__``, but with this PEP such problems can occur more often.
|
||
|
||
|
||
Discussion
|
||
==========
|
||
|
||
Note that the use of module ``__getattr__`` requires care to keep the referred
|
||
objects pickleable. For example, the ``__name__`` attribute of a function
|
||
should correspond to the name with which it is accessible via
|
||
``__getattr__``::
|
||
|
||
def keep_pickleable(func):
|
||
func.__name__ = func.__name__.replace('_deprecated_', '')
|
||
func.__qualname__ = func.__qualname__.replace('_deprecated_', '')
|
||
return func
|
||
|
||
@keep_pickleable
|
||
def _deprecated_old_function(arg, other):
|
||
...
|
||
|
||
One should be also careful to avoid recursion as one would do with
|
||
a class level ``__getattr__``.
|
||
|
||
To use a module global with triggering ``__getattr__`` (for example if one
|
||
wants to use a lazy loaded submodule) one can access it as::
|
||
|
||
sys.modules[__name__].some_global
|
||
|
||
or as::
|
||
|
||
from . import some_global
|
||
|
||
Note that the latter sets the module attribute, thus ``__getattr__`` will be
|
||
called only once.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] PEP 484 section about ``__getattr__`` in stub files
|
||
(https://www.python.org/dev/peps/pep-0484/#stub-files)
|
||
|
||
.. [2] The reference implementation
|
||
(https://github.com/ilevkivskyi/cpython/pull/3/files)
|
||
|
||
.. [3] Reserved classes of identifiers
|
||
(https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers)
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|