python-peps/peps/pep-0726.rst

PEP: 726
Title: Module ``__setattr__`` and ``__delattr__``
Author: Sergey B Kirpichev <skirpichev@gmail.com>
Sponsor: Adam Turner <python@quite.org.uk>
Discussions-To: https://discuss.python.org/t/32640/
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 24-Aug-2023
Python-Version: 3.13
Post-History: `06-Apr-2023 <https://discuss.python.org/t/25506/>`__,
              `31-Aug-2023 <https://discuss.python.org/t/32640/>`__,


Abstract
========

This PEP proposes supporting user-defined ``__setattr__``
and ``__delattr__`` methods on modules to extend customization
of module attribute access beyond :pep:`562`.

Motivation
==========

There are several potential uses of a module ``__setattr__``:

1. To prevent setting an attribute at all (i.e. make it read-only)
2. To validate the value to be assigned
3. To intercept setting an attribute and update some other state

Proper support for read-only attributes would also require adding the
``__delattr__`` function to prevent their deletion.

It would be convenient to directly support such customization, by recognizing
``__setattr__`` and ``__delattr__`` methods defined in a module that would act
like normal :py:meth:`python:object.__setattr__` and
:py:meth:`python:object.__delattr__` methods, except that they will be defined
on module *instances*.  Together with existing ``__getattr__`` and ``__dir__``
methods this will streamline all variants of customizing module attribute access.

For example

.. code:: python

   # mplib.py

   CONSTANT = 3.14
   prec = 53
   dps = 15

   def dps_to_prec(n):
       """Return the number of bits required to represent n decimals accurately."""
       return max(1, int(round((int(n)+1)*3.3219280948873626)))

   def prec_to_dps(n):
       """Return the number of accurate decimals that can be represented with n bits."""
       return max(1, int(round(int(n)/3.3219280948873626)-1))

   def validate(n):
       n = int(n)
       if n <= 0:
           raise ValueError('Positive integer expected')
       return n

   def __setattr__(name, value):
       if name == 'CONSTANT':
           raise AttributeError('Read-only attribute!')
       if name == 'dps':
           value = validate(value)
           globals()['dps'] = value
           globals()['prec'] = dps_to_prec(value)
           return
       if name == 'prec':
           value = validate(value)
           globals()['prec'] = value
           globals()['dps'] = prec_to_dps(value)
           return
       globals()[name] = value

   def __delattr__(name):
       if name in ('CONSTANT', 'dps', 'prec'):
           raise AttributeError('Read-only attribute!')
       del globals()[name]

.. code:: pycon

  >>> import mplib
  >>> mplib.foo = 'spam'
  >>> mplib.CONSTANT = 42
  Traceback (most recent call last):
    ...
  AttributeError: Read-only attribute!
  >>> del mplib.foo
  >>> del mplib.CONSTANT
  Traceback (most recent call last):
    ...
  AttributeError: Read-only attribute!
  >>> mplib.prec
  53
  >>> mplib.dps
  15
  >>> mplib.dps = 5
  >>> mplib.prec
  20
  >>> mplib.dps = 0
  Traceback (most recent call last):
    ...
  ValueError: Positive integer expected


Existing Options
================

The current workaround is assigning the ``__class__`` of a module object to a
custom subclass of :py:class:`python:types.ModuleType` (see [1]_).

For example, to prevent modification or deletion of an attribute we could use:

.. code:: python

   # mod.py

   import sys
   from types import ModuleType

   CONSTANT = 3.14

   class ImmutableModule(ModuleType):
       def __setattr__(name, value):
           raise AttributeError('Read-only attribute!')

       def __delattr__(name):
           raise AttributeError('Read-only attribute!')

   sys.modules[__name__].__class__ = ImmutableModule

But this variant is slower (~2x) than the proposed solution.  More
importantly, it also brings a noticeable speed regression (~2-3x) for
attribute *access*.

Specification
=============

The ``__setattr__`` function at the module level should accept two
arguments, the name of an attribute and the value to be assigned,
and return :py:obj:`None` or raise an :exc:`AttributeError`.

.. code:: python

   def __setattr__(name: str, value: typing.Any, /) -> None: ...

The ``__delattr__`` function should accept one argument,
the name of an attribute, and return :py:obj:`None` or raise an
:py:exc:`AttributeError`:

.. code:: python

   def __delattr__(name: str, /) -> None: ...

The ``__setattr__`` and ``__delattr__`` functions are looked up in the
module ``__dict__``.  If present, the appropriate function is called to
customize setting the attribute or its deletion, else the normal
mechanism (storing/deleting the value in the module dictionary) will work.

Defining module ``__setattr__`` or ``__delattr__`` only affects lookups made
using the attribute access syntax --- directly accessing the module globals
(whether by ``globals()`` within the module, or via a reference to the module's
globals dictionary) is unaffected.  For example:

.. code:: pycon

   >>> import mod
   >>> mod.__dict__['foo'] = 'spam'  # bypasses __setattr__, defined in mod.py

or

.. code:: python

   # mod.py

   def __setattr__(name, value):
      ...

   foo = 'spam'  # bypasses __setattr__
   globals()['bar'] = 'spam'  # here too

   def f():
       global x
       x = 123

   f()  # and here

To use a module global and trigger ``__setattr__`` (or ``__delattr__``),
one can access it via ``sys.modules[__name__]`` within the module's code:

.. code:: python

   # mod.py

   sys.modules[__name__].foo = 'spam'  # bypasses __setattr__

   def __setattr__(name, value):
       ...

   sys.modules[__name__].bar = 'spam'  # triggers __setattr__

This limitation is intentional (just as for the :pep:`562`), because the
interpreter highly optimizes access to module globals and disabling all that
and going through special methods written in Python would slow down the code
unacceptably.


How to Teach This
=================

The "Customizing module attribute access" [1]_ section of the documentation
will be expanded to include new functions.


Reference Implementation
========================

The reference implementation for this PEP can be found in `CPython PR #108261
<https://github.com/python/cpython/pull/108261>`__.


Backwards compatibility
=======================

This PEP may break code that uses module level (global) names
``__setattr__`` and ``__delattr__``, but the language reference
explicitly reserves *all* undocumented dunder names, and allows
"breakage without warning" [2]_.

The performance implications of this PEP are small, since additional
dictionary lookup is much cheaper than storing/deleting the value in
the dictionary.  Also it is hard to imagine a module that expects the
user to set (and/or delete) attributes enough times to be a
performance concern.  On another hand, proposed mechanism allows to
override setting/deleting of attributes without affecting speed of
attribute access, which is much more likely scenario to get a
performance penalty.


Discussion
==========

As pointed out by Victor Stinner, the proposed API could be useful already in
the stdlib, for example to ensure that :py:obj:`sys.modules` type is always a
:py:class:`dict`:

.. code:: pycon

   >>> import sys
   >>> sys.modules = 123
   >>> import asyncio
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "<frozen importlib._bootstrap>", line 1260, in _find_and_load
   AttributeError: 'int' object has no attribute 'get'

or to prevent deletion of critical :py:mod:`sys` attributes, which makes the
code more complicated.  For example, code using :py:obj:`sys.stderr` has to
check if the attribute exists and if it's not :py:obj:`None`.  Currently, it's
possible to remove any :py:mod:`sys` attribute, including functions:

.. code:: pycon

   >>> import sys
   >>> del sys.excepthook
   >>> 1+  # notice the next line
   sys.excepthook is missing
    File "<stdin>", line 1
      1+
       ^
   SyntaxError: invalid syntax

See `related issue
<https://github.com/python/cpython/issues/106016#issue-1771174774>`__ for
other details.

Other stdlib modules also come with attributes which can be overridden (as a
feature) and some input validation here could be helpful.  Examples:
:py:obj:`threading.excepthook`, :py:obj:`warnings.showwarning`,
:py:obj:`io.DEFAULT_BUFFER_SIZE` or :py:obj:`os.SEEK_SET`.

Also a typical use case for customizing module attribute access is managing
deprecation warnings.  But the :pep:`562` accomplishes this scenario only
partially: e.g. it's impossible to issue a warning during an attempt to
*change* a renamed attribute.


Footnotes
=========

.. [1] Customizing module attribute access
       (https://docs.python.org/3.11/reference/datamodel.html#customizing-module-attribute-access)

.. [2] Reserved classes of identifiers
       (https://docs.python.org/3.11/reference/lexical_analysis.html#reserved-classes-of-identifiers)


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.