PEP: 726 Title: Module ``__setattr__`` and ``__delattr__`` Author: Sergey B Kirpichev Sponsor: Adam Turner Discussions-To: https://discuss.python.org/t/32640/ Status: Rejected Type: Standards Track Content-Type: text/x-rst Created: 24-Aug-2023 Python-Version: 3.13 Post-History: `06-Apr-2023 `__, `31-Aug-2023 `__, Resolution: https://discuss.python.org/t/32640/32 Abstract ======== This PEP proposes supporting user-defined ``__setattr__`` and ``__delattr__`` methods on modules to extend customization of module attribute access beyond :pep:`562`. Motivation ========== There are several potential uses of a module ``__setattr__``: 1. To prevent setting an attribute at all (i.e. make it read-only) 2. To validate the value to be assigned 3. To intercept setting an attribute and update some other state Proper support for read-only attributes would also require adding the ``__delattr__`` function to prevent their deletion. It would be convenient to directly support such customization, by recognizing ``__setattr__`` and ``__delattr__`` methods defined in a module that would act like normal :py:meth:`python:object.__setattr__` and :py:meth:`python:object.__delattr__` methods, except that they will be defined on module *instances*. Together with existing ``__getattr__`` and ``__dir__`` methods this will streamline all variants of customizing module attribute access. For example .. code:: python # mplib.py CONSTANT = 3.14 prec = 53 dps = 15 def dps_to_prec(n): """Return the number of bits required to represent n decimals accurately.""" return max(1, int(round((int(n)+1)*3.3219280948873626))) def prec_to_dps(n): """Return the number of accurate decimals that can be represented with n bits.""" return max(1, int(round(int(n)/3.3219280948873626)-1)) def validate(n): n = int(n) if n <= 0: raise ValueError('Positive integer expected') return n def __setattr__(name, value): if name == 'CONSTANT': raise AttributeError('Read-only attribute!') if name == 'dps': value = validate(value) globals()['dps'] = value globals()['prec'] = dps_to_prec(value) return if name == 'prec': value = validate(value) globals()['prec'] = value globals()['dps'] = prec_to_dps(value) return globals()[name] = value def __delattr__(name): if name in ('CONSTANT', 'dps', 'prec'): raise AttributeError('Read-only attribute!') del globals()[name] .. code:: pycon >>> import mplib >>> mplib.foo = 'spam' >>> mplib.CONSTANT = 42 Traceback (most recent call last): ... AttributeError: Read-only attribute! >>> del mplib.foo >>> del mplib.CONSTANT Traceback (most recent call last): ... AttributeError: Read-only attribute! >>> mplib.prec 53 >>> mplib.dps 15 >>> mplib.dps = 5 >>> mplib.prec 20 >>> mplib.dps = 0 Traceback (most recent call last): ... ValueError: Positive integer expected Existing Options ================ The current workaround is assigning the ``__class__`` of a module object to a custom subclass of :py:class:`python:types.ModuleType` (see [1]_). For example, to prevent modification or deletion of an attribute we could use: .. code:: python # mod.py import sys from types import ModuleType CONSTANT = 3.14 class ImmutableModule(ModuleType): def __setattr__(name, value): raise AttributeError('Read-only attribute!') def __delattr__(name): raise AttributeError('Read-only attribute!') sys.modules[__name__].__class__ = ImmutableModule But this variant is slower (~2x) than the proposed solution. More importantly, it also brings a noticeable speed regression (~2-3x) for attribute *access*. Specification ============= The ``__setattr__`` function at the module level should accept two arguments, the name of an attribute and the value to be assigned, and return :py:obj:`None` or raise an :exc:`AttributeError`. .. code:: python def __setattr__(name: str, value: typing.Any, /) -> None: ... The ``__delattr__`` function should accept one argument, the name of an attribute, and return :py:obj:`None` or raise an :py:exc:`AttributeError`: .. code:: python def __delattr__(name: str, /) -> None: ... The ``__setattr__`` and ``__delattr__`` functions are looked up in the module ``__dict__``. If present, the appropriate function is called to customize setting the attribute or its deletion, else the normal mechanism (storing/deleting the value in the module dictionary) will work. Defining module ``__setattr__`` or ``__delattr__`` only affects lookups made using the attribute access syntax --- directly accessing the module globals (whether by ``globals()`` within the module, or via a reference to the module's globals dictionary) is unaffected. For example: .. code:: pycon >>> import mod >>> mod.__dict__['foo'] = 'spam' # bypasses __setattr__, defined in mod.py or .. code:: python # mod.py def __setattr__(name, value): ... foo = 'spam' # bypasses __setattr__ globals()['bar'] = 'spam' # here too def f(): global x x = 123 f() # and here To use a module global and trigger ``__setattr__`` (or ``__delattr__``), one can access it via ``sys.modules[__name__]`` within the module's code: .. code:: python # mod.py sys.modules[__name__].foo = 'spam' # bypasses __setattr__ def __setattr__(name, value): ... sys.modules[__name__].bar = 'spam' # triggers __setattr__ This limitation is intentional (just as for the :pep:`562`), because the interpreter highly optimizes access to module globals and disabling all that and going through special methods written in Python would slow down the code unacceptably. How to Teach This ================= The "Customizing module attribute access" [1]_ section of the documentation will be expanded to include new functions. Reference Implementation ======================== The reference implementation for this PEP can be found in `CPython PR #108261 `__. Backwards compatibility ======================= This PEP may break code that uses module level (global) names ``__setattr__`` and ``__delattr__``, but the language reference explicitly reserves *all* undocumented dunder names, and allows "breakage without warning" [2]_. The performance implications of this PEP are small, since additional dictionary lookup is much cheaper than storing/deleting the value in the dictionary. Also it is hard to imagine a module that expects the user to set (and/or delete) attributes enough times to be a performance concern. On another hand, proposed mechanism allows to override setting/deleting of attributes without affecting speed of attribute access, which is much more likely scenario to get a performance penalty. Discussion ========== As pointed out by Victor Stinner, the proposed API could be useful already in the stdlib, for example to ensure that :py:obj:`sys.modules` type is always a :py:class:`dict`: .. code:: pycon >>> import sys >>> sys.modules = 123 >>> import asyncio Traceback (most recent call last): File "", line 1, in File "", line 1260, in _find_and_load AttributeError: 'int' object has no attribute 'get' or to prevent deletion of critical :py:mod:`sys` attributes, which makes the code more complicated. For example, code using :py:obj:`sys.stderr` has to check if the attribute exists and if it's not :py:obj:`None`. Currently, it's possible to remove any :py:mod:`sys` attribute, including functions: .. code:: pycon >>> import sys >>> del sys.excepthook >>> 1+ # notice the next line sys.excepthook is missing File "", line 1 1+ ^ SyntaxError: invalid syntax See `related issue `__ for other details. Other stdlib modules also come with attributes which can be overridden (as a feature) and some input validation here could be helpful. Examples: :py:obj:`threading.excepthook`, :py:obj:`warnings.showwarning`, :py:obj:`io.DEFAULT_BUFFER_SIZE` or :py:obj:`os.SEEK_SET`. Also a typical use case for customizing module attribute access is managing deprecation warnings. But the :pep:`562` accomplishes this scenario only partially: e.g. it's impossible to issue a warning during an attempt to *change* a renamed attribute. Footnotes ========= .. [1] Customizing module attribute access (https://docs.python.org/3.11/reference/datamodel.html#customizing-module-attribute-access) .. [2] Reserved classes of identifiers (https://docs.python.org/3.11/reference/lexical_analysis.html#reserved-classes-of-identifiers) Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.