python-peps/pep-0447.txt

PEP: 447
Title: Add __locallookup__ method to metaclass
Version: $Revision$
Last-Modified: $Date$
Author: Ronald Oussoren <ronaldoussoren@mac.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jun-2013
Post-History: 2-Jul-2013, 15-Jul-2013


Abstract
========

Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek
in the ``__dict__`` of classes on the MRO for a class when looking for
an attribute. This PEP adds an optional ``__locallookup__`` method to
a metaclass that can be used to override this behavior.

Rationale
=========

It is currently not possible to influence how the `super class`_ looks
up attributes (that is, ``super.__getattribute__`` unconditionally
peeks in the class ``__dict__``), and that can be problematic for
dynamic classes that can grow new methods on demand.

The ``__locallookup__`` method makes it possible to dynamicly add
attributes even when looking them up using the `super class`_.

The new method affects ``object.__getattribute__`` (and
`PyObject_GenericGetAttr`_) as well for consistency.

Background
----------

The current behavior of ``super.__getattribute__`` causes problems for
classes that are dynamic proxies for other (non-Python) classes or types,
an example of which is `PyObjC`_. PyObjC creates a Python class for every
class in the Objective-C runtime, and looks up methods in the Objective-C
runtime when they are used. This works fine for normal access, but doesn't
work for access with ``super`` objects. Because of this PyObjC currently
includes a custom ``super`` that must be used with its classes.

The API in this PEP makes it possible to remove the custom ``super`` and
simplifies the implementation because the custom lookup behavior can be
added in a central location.


The superclass attribute lookup hook
====================================

Both ``super.__getattribute__`` and ``object.__getattribute__`` (or
`PyObject_GenericGetAttr`_ in C code) walk an object's MRO and peek in the
class' ``__dict__`` to look up attributes. A way to affect this lookup is
using a method on the meta class for the type, that by default looks up
the name in the class ``__dict__``.

In Python code
--------------

A meta type can define a method ``__locallookup__`` that is called during
attribute resolution by both ``super.__getattribute__`` and ``object.__getattribute``::

    class MetaType(type):

        def __locallookup__(cls, name):
            try:
                return cls.__dict__[name]
            except KeyError:
                raise AttributeError(name) from None

The ``__locallookup__`` method has as its arguments a class and the name of the attribute
that is looked up. It should return the value of the attribute without invoking descriptors,
or raise `AttributeError`_ when the name cannot be found.

The `type`_ class does not provide an implementation for ``__locallookup__``, primarily
to enable some optimizations in the Python implementation.

Example usage
.............

The code below implements a silly metaclass that redirects attribute lookup to uppercase
versions of names::

    class UpperCaseAccess (type):

        def __locallookup__(cls, name):
	    return cls.__dict__[name.upper()]

    class SillyObject (metaclass=UpperCaseAccess):
        def m(self):
	    return 42

	def M(self):
	    return "fourtytwo"

    obj = SillyObject()
    assert obj.m() == "fortytwo"


In C code
---------

A new slot ``tp_locallookup`` is added to the ``PyTypeObject`` struct, this slot
corresponds to the ``__locallookup__`` method on `type`_.

The slot has the following prototype::

    PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name);

This method should lookup *name* in the namespace of *cls*, without looking at superclasses,
and should not invoke descriptors. The method returns ``NULL`` without setting an exception
when the *name* cannot be found, and returns a new reference otherwise (not a borrowed reference).


Usage of this hook
------------------

The new method is optional and will not be defined on `type`_. Both ``super.__getattribute__``
and ``object.__getattribute__``/`PyObject_GenericGetAttr`_ (through ``_PyType_Lookup``) will use the
the ``__locallookup__`` method when it is present in the meta type of a type on the MRO and will
continue to peek in the type's ``__dict__`` when the meta type does not have a ``__locallookup``
method.

Other changes to the implementation
...................................

The change for `PyObject_GenericGetAttr`_ will be done by changing the private function
``_PyType_Lookup``. This currently returns a borrowed reference, but must return a new
reference when the ``__locallookup__`` method is present. Because of this ``_PyType_Lookup``
will be renamed to ``_PyType_LookupName``, this will cause compile-time errors for all out-of-tree
users of this private API.

By making ``__locallookup_`` optional the implementation can continue to use the type attribute
lookup cache for types that don't have a metaclass with this new method, which should minimize the
performance impact of the change.

**TODO**: run pybench, an possibly the full speedtest,  with and without this change and insert
the results.


Alternative proposals
---------------------

``__getattribute_super__``
..........................

An earlier version of this PEP used the following static method on classes::

    def __getattribute_super__(cls, name, object, owner): pass

This method performed name lookup as well as invoking descriptors and was necessarily
limited to working only with ``super.__getattribute__``.


Reuse ``tp_getattro``
.....................

It would be nice to avoid adding a new slot, thus keeping the API simpler and
easier to understand.  A comment on `Issue 18181`_ asked about reusing the
``tp_getattro`` slot, that is super could call the ``tp_getattro`` slot of all
methods along the MRO.

That won't work because ``tp_getattro`` will look in the instance
``__dict__`` before it tries to resolve attributes using classes in the MRO.
This would mean that using ``tp_getattro`` instead of peeking the class
dictionaries changes the semantics of the `super class`_.


References
==========

* `Issue 18181`_ contains a prototype implementation


Copyright
=========

This document has been placed in the public domain.

.. _`Issue 18181`: http://bugs.python.org/issue18181

.. _`super class`: http://docs.python.org/3/library/functions.html#super

.. _`NotImplemented`: http://docs.python.org/3/library/constants.html#NotImplemented

.. _`PyObject_GenericGetAttr`: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr

.. _`type`: http://docs.python.org/3/library/functions.html#type

.. _`AttributeError`: http://docs.python.org/3/library/exceptions.html#AttributeError

.. _`PyObjC`: http://pyobjc.sourceforge.net/

.. _`classmethod`: http://docs.python.org/3/library/functions.html#classmethod