diff --git a/pep-0447.txt b/pep-0447.txt index e8b476d9e..3e92dc6a2 100644 --- a/pep-0447.txt +++ b/pep-0447.txt @@ -7,7 +7,7 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jun-2013 -Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013 +Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013, 22-Jul-2015 Abstract @@ -16,7 +16,8 @@ Abstract Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek in the ``__dict__`` of classes on the MRO for a class when looking for an attribute. This PEP adds an optional ``__getdescriptor__`` method to -a metaclass that can be used to override this behavior. +a metaclass that replaces this behavior and gives more control over attribute +lookup, especially when using a `super`_ object. That is, the MRO walking loop in ``_PyType_Lookup`` and ``super.__getattribute__`` gets changed from:: @@ -39,6 +40,15 @@ to:: return NotFound +The default implemention of ``__getdescriptor__`` looks in the class +dictionary:: + + class type: + def __getdescriptor__(cls, name): + try: + return cls.__dict__[name] + except KeyError: + raise AttributeError(name) from None Rationale ========= @@ -63,13 +73,23 @@ classes that are dynamic proxies for other (non-Python) classes or types, an example of which is `PyObjC`_. PyObjC creates a Python class for every class in the Objective-C runtime, and looks up methods in the Objective-C runtime when they are used. This works fine for normal access, but doesn't -work for access with ``super`` objects. Because of this PyObjC currently -includes a custom ``super`` that must be used with its classes. +work for access with `super`_ objects. Because of this PyObjC currently +includes a custom `super`_ that must be used with its classes, as well as +completely reimplementing `PyObject_GenericGetAttr`_ for normal attribute +access. -The API in this PEP makes it possible to remove the custom ``super`` and +The API in this PEP makes it possible to remove the custom `super`_ and simplifies the implementation because the custom lookup behavior can be added in a central location. +.. note:: + + `PyObjC`_ cannot precalculate the contents of the class ``__dict__`` + because Objective-C classes can grow new methods at runtime. Furthermore + Objective-C classes tend to contain a lot of methods while most Python + code will only use a small subset of them, this makes precalculating + unnecessarily expensive. + The superclass attribute lookup hook ==================================== @@ -88,12 +108,12 @@ unchanged unless a metatype actually defines the new special method. Aside: Attribute resolution algorithm in Python ----------------------------------------------- -The attribute resolution proces as implemented by ``object.__getattribute__`` (or -PyObject_GenericGetAttr`` in CPython's implementation) is fairly straightforward, -but not entirely so without reading C code. +The attribute resolution proces as implemented by ``object.__getattribute__`` +(or PyObject_GenericGetAttr`` in CPython's implementation) is fairly +straightforward, but not entirely so without reading C code. -The current CPython implementation of object.__getattribute__ is basicly equivalent -to the following (pseudo-) Python code (excluding some house keeping and speed tricks):: +The current CPython implementation of object.__getattribute__ is basicly +equivalent to the following (pseudo-) Python code (excluding some house keeping and speed tricks):: def _PyType_Lookup(tp, name): @@ -179,8 +199,8 @@ to the following (pseudo-) Python code (excluding some house keeping and speed t This PEP should change the dict lookup at the lines starting at "# PEP 447" with -a method call to perform the actual lookup, making is possible to affect that lookup -both for normal attribute access and access through the `super proxy`_. +a method call to perform the actual lookup, making is possible to affect that +lookup both for normal attribute access and access through the `super proxy`_. Note that specific classes can already completely override the default behaviour by implementing their own ``__getattribute__`` slot (with or without @@ -232,18 +252,19 @@ uppercase versions of names:: obj = SillyObject() assert obj.m() == "fortytwo" -As mentioned earlier in this PEP a more realistic use case of this functionallity -is a ``__getdescriptor__`` method that dynamicly populates the class ``__dict__`` -based on attribute access, primarily when it is not possible to reliably keep the -class dict in sync with its source, for example because the source used to populate -``__dict__`` is dynamic as well and does not have triggers that can be used to detect -changes to that source. +As mentioned earlier in this PEP a more realistic use case of this +functionallity is a ``__getdescriptor__`` method that dynamicly populates the +class ``__dict__`` based on attribute access, primarily when it is not +possible to reliably keep the class dict in sync with its source, for example +because the source used to populate ``__dict__`` is dynamic as well and does +not have triggers that can be used to detect changes to that source. -An example of that are the class bridges in PyObjC: the class bridge is a Python -object (class) that represents an Objective-C class and conceptually has a Python -method for every Objective-C method in the Objective-C class. As with Python it is -possible to add new methods to an Objective-C class, or replace existing ones, and -there are no callbacks that can be used to detect this. +An example of that are the class bridges in PyObjC: the class bridge is a +Python object (class) that represents an Objective-C class and conceptually +has a Python method for every Objective-C method in the Objective-C class. +As with Python it is possible to add new methods to an Objective-C class, or +replace existing ones, and there are no callbacks that can be used to detect +this. In C code --------- @@ -290,16 +311,20 @@ Use of the method introduced in this PEP can affect introspection of classes with a metaclass that uses a custom ``__getdescriptor__`` method. This section lists those changes. +The items listed below are only affected by custom ``__getdescriptor__`` +methods, the default implementation for ``object`` won't cause problems +because that still only uses the class ``__dict__`` and won't cause visible +changes to the visible behaviour of the ``object.__getattribute__``. + * ``dir`` might not show all attributes - As with a custom ``__getattribute__`` method ``dir()`` might not see all + As with a custom ``__getattribute__`` method `dir()`_ might not see all (instance) attributes when using the ``__getdescriptor__()`` method to dynamicly resolve attributes. The solution for that is quite simple: classes using ``__getdescriptor__`` - should also implement ``__dir__`` if they want full support for the builtin - ``dir`` function. - + should also implement `__dir__()`_ if they want full support for the builtin + `dir()`_ function. * ``inspect.getattr_static`` might not show all attributes @@ -316,13 +341,26 @@ lists those changes. **TODO**: I haven't fully worked out what the impact of this is, and if there are mitigations for those using either updates to these functions, or - additional methods that users should implement to be fully compatible with these - functions. + additional methods that users should implement to be fully compatible with + these functions. + + One possible mitigation is to have a custom ``__getattribute__`` for these + classes that fills ``__dict__`` before returning and and defers to the + default implementation for other attributes. + +* Direct introspection of the class ``__dict__`` + + Any code that directly access the class ``__dict__`` for introspection + can be affected by a custom ``__getdescriptor__`` method. Performance impact ------------------ +**WARNING**: The benchmark results in this section are old, and will be updated +when I've ported the patch to the current trunk. I don't expect significant +changes to the results in this section. + The pybench output below compares an implementation of this PEP with the regular source tree, both based on changeset a5681f50bae2, run on an idle machine an Core i7 processor running Centos 6.4. @@ -567,11 +605,22 @@ That won't work because ``tp_getattro`` will look in the instance This would mean that using ``tp_getattro`` instead of peeking the class dictionaries changes the semantics of the `super class`_. +Alternate placement of the new method +..................................... + +This PEP proposes to add ``__getdescriptor__`` as a method on the metaclass. +An alternative would be to add it as a class method on the class itself +(simular to how ``__new__`` is a `staticmethod`_ of the class and not a method +of the metaclass). + +The two are functionally equivalent, and there's something to be said about +not requiring the use of a meta class. + References ========== -* `Issue 18181`_ contains a prototype implementation +* `Issue 18181`_ contains an out of date prototype implementation Copyright ========= @@ -584,6 +633,14 @@ This document has been placed in the public domain. .. _`super proxy`: http://docs.python.org/3/library/functions.html#super +.. _`super`: http://docs.python.org/3/library/functions.html#super + +.. _`dir()`: http://docs.python.org/3/library/functions.html#dir + +.. _`staticmethod`: http://docs.python.org/3/library/functions.html#staticmethod + +.. _`__dir__()`: https://docs.python.org/3/reference/datamodel.html#object.__dir__ + .. _`NotImplemented`: http://docs.python.org/3/library/constants.html#NotImplemented .. _`PyObject_GenericGetAttr`: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr