595 lines
20 KiB
ReStructuredText
595 lines
20 KiB
ReStructuredText
PEP: 367
|
||
Title: New Super
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Calvin Spealman <ironfroggy@gmail.com>,
|
||
Tim Delaney <timothy.c.delaney@gmail.com>
|
||
Status: Superseded
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 28-Apr-2007
|
||
Python-Version: 2.6
|
||
Post-History: `28-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072807.html>`__,
|
||
`29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072835.html>`__,
|
||
`29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072858.html>`__,
|
||
`14-May-2007 <https://mail.python.org/pipermail/python-dev/2007-May/073127.html>`__
|
||
|
||
Numbering Note
|
||
==============
|
||
|
||
This PEP has been renumbered to :pep:`3135`. The text below is the last
|
||
version submitted under the old number.
|
||
|
||
Abstract
|
||
========
|
||
|
||
This PEP proposes syntactic sugar for use of the ``super`` type to automatically
|
||
construct instances of the super type binding to the class that a method was
|
||
defined in, and the instance (or class object for classmethods) that the method
|
||
is currently acting upon.
|
||
|
||
The premise of the new super usage suggested is as follows::
|
||
|
||
super.foo(1, 2)
|
||
|
||
to replace the old::
|
||
|
||
super(Foo, self).foo(1, 2)
|
||
|
||
and the current ``__builtin__.super`` be aliased to ``__builtin__.__super__``
|
||
(with ``__builtin__.super`` to be removed in Python 3.0).
|
||
|
||
It is further proposed that assignment to ``super`` become a ``SyntaxError``,
|
||
similar to the behaviour of ``None``.
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
The current usage of super requires an explicit passing of both the class and
|
||
instance it must operate from, requiring a breaking of the DRY (Don't Repeat
|
||
Yourself) rule. This hinders any change in class name, and is often considered
|
||
a wart by many.
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
Within the specification section, some special terminology will be used to
|
||
distinguish similar and closely related concepts. "super type" will refer to
|
||
the actual builtin type named "super". A "super instance" is simply an instance
|
||
of the super type, which is associated with a class and possibly with an
|
||
instance of that class.
|
||
|
||
Because the new ``super`` semantics are not backwards compatible with Python
|
||
2.5, the new semantics will require a ``__future__`` import::
|
||
|
||
from __future__ import new_super
|
||
|
||
The current ``__builtin__.super`` will be aliased to ``__builtin__.__super__``.
|
||
This will occur regardless of whether the new ``super`` semantics are active.
|
||
It is not possible to simply rename ``__builtin__.super``, as that would affect
|
||
modules that do not use the new ``super`` semantics. In Python 3.0 it is
|
||
proposed that the name ``__builtin__.super`` will be removed.
|
||
|
||
Replacing the old usage of super, calls to the next class in the MRO (method
|
||
resolution order) can be made without explicitly creating a ``super``
|
||
instance (although doing so will still be supported via ``__super__``). Every
|
||
function will have an implicit local named ``super``. This name behaves
|
||
identically to a normal local, including use by inner functions via a cell,
|
||
with the following exceptions:
|
||
|
||
1. Assigning to the name ``super`` will raise a ``SyntaxError`` at compile time;
|
||
|
||
2. Calling a static method or normal function that accesses the name ``super``
|
||
will raise a ``TypeError`` at runtime.
|
||
|
||
Every function that uses the name ``super``, or has an inner function that
|
||
uses the name ``super``, will include a preamble that performs the equivalent
|
||
of::
|
||
|
||
super = __builtin__.__super__(<class>, <instance>)
|
||
|
||
where ``<class>`` is the class that the method was defined in, and
|
||
``<instance>`` is the first parameter of the method (normally ``self`` for
|
||
instance methods, and ``cls`` for class methods). For static methods and normal
|
||
functions, ``<class>`` will be ``None``, resulting in a ``TypeError`` being
|
||
raised during the preamble.
|
||
|
||
Note: The relationship between ``super`` and ``__super__`` is similar to that
|
||
between ``import`` and ``__import__``.
|
||
|
||
Much of this was discussed in the thread of the python-dev list, "Fixing super
|
||
anyone?" [1]_.
|
||
|
||
|
||
Open Issues
|
||
-----------
|
||
|
||
|
||
Determining the class object to use
|
||
'''''''''''''''''''''''''''''''''''
|
||
|
||
The exact mechanism for associating the method with the defining class is not
|
||
specified in this PEP, and should be chosen for maximum performance. For
|
||
CPython, it is suggested that the class instance be held in a C-level variable
|
||
on the function object which is bound to one of ``NULL`` (not part of a class),
|
||
``Py_None`` (static method) or a class object (instance or class method).
|
||
|
||
|
||
Should ``super`` actually become a keyword?
|
||
'''''''''''''''''''''''''''''''''''''''''''
|
||
|
||
With this proposal, ``super`` would become a keyword to the same extent that
|
||
``None`` is a keyword. It is possible that further restricting the ``super``
|
||
name may simplify implementation, however some are against the actual
|
||
keyword-ization of super. The simplest solution is often the correct solution
|
||
and the simplest solution may well not be adding additional keywords to the
|
||
language when they are not needed. Still, it may solve other open issues.
|
||
|
||
|
||
Closed Issues
|
||
-------------
|
||
|
||
super used with __call__ attributes
|
||
'''''''''''''''''''''''''''''''''''
|
||
|
||
It was considered that it might be a problem that instantiating super instances
|
||
the classic way, because calling it would lookup the __call__ attribute and
|
||
thus try to perform an automatic super lookup to the next class in the MRO.
|
||
However, this was found to be false, because calling an object only looks up
|
||
the __call__ method directly on the object's type. The following example shows
|
||
this in action.
|
||
|
||
::
|
||
|
||
class A(object):
|
||
def __call__(self):
|
||
return '__call__'
|
||
def __getattribute__(self, attr):
|
||
if attr == '__call__':
|
||
return lambda: '__getattribute__'
|
||
a = A()
|
||
assert a() == '__call__'
|
||
assert a.__call__() == '__getattribute__'
|
||
|
||
In any case, with the renaming of ``__builtin__.super`` to
|
||
``__builtin__.__super__`` this issue goes away entirely.
|
||
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
It is impossible to implement the above specification entirely in Python. This
|
||
reference implementation has the following differences to the specification:
|
||
|
||
1. New ``super`` semantics are implemented using bytecode hacking.
|
||
|
||
2. Assignment to ``super`` is not a ``SyntaxError``. Also see point #4.
|
||
|
||
3. Classes must either use the metaclass ``autosuper_meta`` or inherit from
|
||
the base class ``autosuper`` to acquire the new ``super`` semantics.
|
||
|
||
4. ``super`` is not an implicit local variable. In particular, for inner
|
||
functions to be able to use the super instance, there must be an assignment
|
||
of the form ``super = super`` in the method.
|
||
|
||
The reference implementation assumes that it is being run on Python 2.5+.
|
||
|
||
::
|
||
|
||
#!/usr/bin/env python
|
||
#
|
||
# autosuper.py
|
||
|
||
from array import array
|
||
import dis
|
||
import new
|
||
import types
|
||
import __builtin__
|
||
__builtin__.__super__ = __builtin__.super
|
||
del __builtin__.super
|
||
|
||
# We need these for modifying bytecode
|
||
from opcode import opmap, HAVE_ARGUMENT, EXTENDED_ARG
|
||
|
||
LOAD_GLOBAL = opmap['LOAD_GLOBAL']
|
||
LOAD_NAME = opmap['LOAD_NAME']
|
||
LOAD_CONST = opmap['LOAD_CONST']
|
||
LOAD_FAST = opmap['LOAD_FAST']
|
||
LOAD_ATTR = opmap['LOAD_ATTR']
|
||
STORE_FAST = opmap['STORE_FAST']
|
||
LOAD_DEREF = opmap['LOAD_DEREF']
|
||
STORE_DEREF = opmap['STORE_DEREF']
|
||
CALL_FUNCTION = opmap['CALL_FUNCTION']
|
||
STORE_GLOBAL = opmap['STORE_GLOBAL']
|
||
DUP_TOP = opmap['DUP_TOP']
|
||
POP_TOP = opmap['POP_TOP']
|
||
NOP = opmap['NOP']
|
||
JUMP_FORWARD = opmap['JUMP_FORWARD']
|
||
ABSOLUTE_TARGET = dis.hasjabs
|
||
|
||
def _oparg(code, opcode_pos):
|
||
return code[opcode_pos+1] + (code[opcode_pos+2] << 8)
|
||
|
||
def _bind_autosuper(func, cls):
|
||
co = func.func_code
|
||
name = func.func_name
|
||
newcode = array('B', co.co_code)
|
||
codelen = len(newcode)
|
||
newconsts = list(co.co_consts)
|
||
newvarnames = list(co.co_varnames)
|
||
|
||
# Check if the global 'super' keyword is already present
|
||
try:
|
||
sn_pos = list(co.co_names).index('super')
|
||
except ValueError:
|
||
sn_pos = None
|
||
|
||
# Check if the varname 'super' keyword is already present
|
||
try:
|
||
sv_pos = newvarnames.index('super')
|
||
except ValueError:
|
||
sv_pos = None
|
||
|
||
# Check if the cellvar 'super' keyword is already present
|
||
try:
|
||
sc_pos = list(co.co_cellvars).index('super')
|
||
except ValueError:
|
||
sc_pos = None
|
||
|
||
# If 'super' isn't used anywhere in the function, we don't have anything to do
|
||
if sn_pos is None and sv_pos is None and sc_pos is None:
|
||
return func
|
||
|
||
c_pos = None
|
||
s_pos = None
|
||
n_pos = None
|
||
|
||
# Check if the 'cls_name' and 'super' objects are already in the constants
|
||
for pos, o in enumerate(newconsts):
|
||
if o is cls:
|
||
c_pos = pos
|
||
|
||
if o is __super__:
|
||
s_pos = pos
|
||
|
||
if o == name:
|
||
n_pos = pos
|
||
|
||
# Add in any missing objects to constants and varnames
|
||
if c_pos is None:
|
||
c_pos = len(newconsts)
|
||
newconsts.append(cls)
|
||
|
||
if n_pos is None:
|
||
n_pos = len(newconsts)
|
||
newconsts.append(name)
|
||
|
||
if s_pos is None:
|
||
s_pos = len(newconsts)
|
||
newconsts.append(__super__)
|
||
|
||
if sv_pos is None:
|
||
sv_pos = len(newvarnames)
|
||
newvarnames.append('super')
|
||
|
||
# This goes at the start of the function. It is:
|
||
#
|
||
# super = __super__(cls, self)
|
||
#
|
||
# If 'super' is a cell variable, we store to both the
|
||
# local and cell variables (i.e. STORE_FAST and STORE_DEREF).
|
||
#
|
||
preamble = [
|
||
LOAD_CONST, s_pos & 0xFF, s_pos >> 8,
|
||
LOAD_CONST, c_pos & 0xFF, c_pos >> 8,
|
||
LOAD_FAST, 0, 0,
|
||
CALL_FUNCTION, 2, 0,
|
||
]
|
||
|
||
if sc_pos is None:
|
||
# 'super' is not a cell variable - we can just use the local variable
|
||
preamble += [
|
||
STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
|
||
]
|
||
else:
|
||
# If 'super' is a cell variable, we need to handle LOAD_DEREF.
|
||
preamble += [
|
||
DUP_TOP,
|
||
STORE_FAST, sv_pos & 0xFF, sv_pos >> 8,
|
||
STORE_DEREF, sc_pos & 0xFF, sc_pos >> 8,
|
||
]
|
||
|
||
preamble = array('B', preamble)
|
||
|
||
# Bytecode for loading the local 'super' variable.
|
||
load_super = array('B', [
|
||
LOAD_FAST, sv_pos & 0xFF, sv_pos >> 8,
|
||
])
|
||
|
||
preamble_len = len(preamble)
|
||
need_preamble = False
|
||
i = 0
|
||
|
||
while i < codelen:
|
||
opcode = newcode[i]
|
||
need_load = False
|
||
remove_store = False
|
||
|
||
if opcode == EXTENDED_ARG:
|
||
raise TypeError("Cannot use 'super' in function with EXTENDED_ARG opcode")
|
||
|
||
# If the opcode is an absolute target it needs to be adjusted
|
||
# to take into account the preamble.
|
||
elif opcode in ABSOLUTE_TARGET:
|
||
oparg = _oparg(newcode, i) + preamble_len
|
||
newcode[i+1] = oparg & 0xFF
|
||
newcode[i+2] = oparg >> 8
|
||
|
||
# If LOAD_GLOBAL(super) or LOAD_NAME(super) then we want to change it into
|
||
# LOAD_FAST(super)
|
||
elif (opcode == LOAD_GLOBAL or opcode == LOAD_NAME) and _oparg(newcode, i) == sn_pos:
|
||
need_preamble = need_load = True
|
||
|
||
# If LOAD_FAST(super) then we just need to add the preamble
|
||
elif opcode == LOAD_FAST and _oparg(newcode, i) == sv_pos:
|
||
need_preamble = need_load = True
|
||
|
||
# If LOAD_DEREF(super) then we change it into LOAD_FAST(super) because
|
||
# it's slightly faster.
|
||
elif opcode == LOAD_DEREF and _oparg(newcode, i) == sc_pos:
|
||
need_preamble = need_load = True
|
||
|
||
if need_load:
|
||
newcode[i:i+3] = load_super
|
||
|
||
i += 1
|
||
|
||
if opcode >= HAVE_ARGUMENT:
|
||
i += 2
|
||
|
||
# No changes needed - get out.
|
||
if not need_preamble:
|
||
return func
|
||
|
||
# Our preamble will have 3 things on the stack
|
||
co_stacksize = max(3, co.co_stacksize)
|
||
|
||
# Conceptually, our preamble is on the `def` line.
|
||
co_lnotab = array('B', co.co_lnotab)
|
||
|
||
if co_lnotab:
|
||
co_lnotab[0] += preamble_len
|
||
|
||
co_lnotab = co_lnotab.tostring()
|
||
|
||
# Our code consists of the preamble and the modified code.
|
||
codestr = (preamble + newcode).tostring()
|
||
|
||
codeobj = new.code(co.co_argcount, len(newvarnames), co_stacksize,
|
||
co.co_flags, codestr, tuple(newconsts), co.co_names,
|
||
tuple(newvarnames), co.co_filename, co.co_name,
|
||
co.co_firstlineno, co_lnotab, co.co_freevars,
|
||
co.co_cellvars)
|
||
|
||
func.func_code = codeobj
|
||
func.func_class = cls
|
||
return func
|
||
|
||
class autosuper_meta(type):
|
||
def __init__(cls, name, bases, clsdict):
|
||
UnboundMethodType = types.UnboundMethodType
|
||
|
||
for v in vars(cls):
|
||
o = getattr(cls, v)
|
||
if isinstance(o, UnboundMethodType):
|
||
_bind_autosuper(o.im_func, cls)
|
||
|
||
class autosuper(object):
|
||
__metaclass__ = autosuper_meta
|
||
|
||
if __name__ == '__main__':
|
||
class A(autosuper):
|
||
def f(self):
|
||
return 'A'
|
||
|
||
class B(A):
|
||
def f(self):
|
||
return 'B' + super.f()
|
||
|
||
class C(A):
|
||
def f(self):
|
||
def inner():
|
||
return 'C' + super.f()
|
||
|
||
# Needed to put 'super' into a cell
|
||
super = super
|
||
return inner()
|
||
|
||
class D(B, C):
|
||
def f(self, arg=None):
|
||
var = None
|
||
return 'D' + super.f()
|
||
|
||
assert D().f() == 'DBCA'
|
||
|
||
Disassembly of B.f and C.f reveals the different preambles used when ``super``
|
||
is simply a local variable compared to when it is used by an inner function.
|
||
|
||
::
|
||
|
||
>>> dis.dis(B.f)
|
||
|
||
214 0 LOAD_CONST 4 (<type 'super'>)
|
||
3 LOAD_CONST 2 (<class '__main__.B'>)
|
||
6 LOAD_FAST 0 (self)
|
||
9 CALL_FUNCTION 2
|
||
12 STORE_FAST 1 (super)
|
||
|
||
215 15 LOAD_CONST 1 ('B')
|
||
18 LOAD_FAST 1 (super)
|
||
21 LOAD_ATTR 1 (f)
|
||
24 CALL_FUNCTION 0
|
||
27 BINARY_ADD
|
||
28 RETURN_VALUE
|
||
|
||
::
|
||
|
||
>>> dis.dis(C.f)
|
||
|
||
218 0 LOAD_CONST 4 (<type 'super'>)
|
||
3 LOAD_CONST 2 (<class '__main__.C'>)
|
||
6 LOAD_FAST 0 (self)
|
||
9 CALL_FUNCTION 2
|
||
12 DUP_TOP
|
||
13 STORE_FAST 1 (super)
|
||
16 STORE_DEREF 0 (super)
|
||
|
||
219 19 LOAD_CLOSURE 0 (super)
|
||
22 LOAD_CONST 1 (<code object inner at 00C160A0, file "autosuper.py", line 219>)
|
||
25 MAKE_CLOSURE 0
|
||
28 STORE_FAST 2 (inner)
|
||
|
||
223 31 LOAD_FAST 1 (super)
|
||
34 STORE_DEREF 0 (super)
|
||
|
||
224 37 LOAD_FAST 2 (inner)
|
||
40 CALL_FUNCTION 0
|
||
43 RETURN_VALUE
|
||
|
||
Note that in the final implementation, the preamble would not be part of the
|
||
bytecode of the method, but would occur immediately following unpacking of
|
||
parameters.
|
||
|
||
|
||
Alternative Proposals
|
||
=====================
|
||
|
||
No Changes
|
||
----------
|
||
|
||
Although its always attractive to just keep things how they are, people have
|
||
sought a change in the usage of super calling for some time, and for good
|
||
reason, all mentioned previously.
|
||
|
||
- Decoupling from the class name (which might not even be bound to the
|
||
right class anymore!)
|
||
- Simpler looking, cleaner super calls would be better
|
||
|
||
Dynamic attribute on super type
|
||
-------------------------------
|
||
|
||
The proposal adds a dynamic attribute lookup to the super type, which will
|
||
automatically determine the proper class and instance parameters. Each super
|
||
attribute lookup identifies these parameters and performs the super lookup on
|
||
the instance, as the current super implementation does with the explicit
|
||
invocation of a super instance upon a class and instance.
|
||
|
||
This proposal relies on sys._getframe(), which is not appropriate for anything
|
||
except a prototype implementation.
|
||
|
||
|
||
super(__this_class__, self)
|
||
---------------------------
|
||
|
||
This is nearly an anti-proposal, as it basically relies on the acceptance of
|
||
the __this_class__ PEP, which proposes a special name that would always be
|
||
bound to the class within which it is used. If that is accepted, __this_class__
|
||
could simply be used instead of the class' name explicitly, solving the name
|
||
binding issues [2]_.
|
||
|
||
self.__super__.foo(\*args)
|
||
--------------------------
|
||
|
||
The __super__ attribute is mentioned in this PEP in several places, and could
|
||
be a candidate for the complete solution, actually using it explicitly instead
|
||
of any super usage directly. However, double-underscore names are usually an
|
||
internal detail, and attempted to be kept out of everyday code.
|
||
|
||
super(self, \*args) or __super__(self, \*args)
|
||
----------------------------------------------
|
||
|
||
This solution only solves the problem of the type indication, does not handle
|
||
differently named super methods, and is explicit about the name of the
|
||
instance. It is less flexible without being able to enacted on other method
|
||
names, in cases where that is needed. One use case this fails is where a
|
||
base-class has a factory classmethod and a subclass has two factory
|
||
classmethods, both of which needing to properly make super calls to the one
|
||
in the base-class.
|
||
|
||
super.foo(self, \*args)
|
||
-----------------------
|
||
|
||
This variation actually eliminates the problems with locating the proper
|
||
instance, and if any of the alternatives were pushed into the spotlight, I
|
||
would want it to be this one.
|
||
|
||
super or super()
|
||
----------------
|
||
|
||
This proposal leaves no room for different names, signatures, or application
|
||
to other classes, or instances. A way to allow some similar use alongside the
|
||
normal proposal would be favorable, encouraging good design of multiple
|
||
inheritance trees and compatible methods.
|
||
|
||
super(\*p, \*\*kw)
|
||
------------------
|
||
|
||
There has been the proposal that directly calling ``super(*p, **kw)`` would
|
||
be equivalent to calling the method on the ``super`` object with the same name
|
||
as the method currently being executed i.e. the following two methods would be
|
||
equivalent:
|
||
|
||
::
|
||
|
||
def f(self, *p, **kw):
|
||
super.f(*p, **kw)
|
||
|
||
::
|
||
|
||
def f(self, *p, **kw):
|
||
super(*p, **kw)
|
||
|
||
There is strong sentiment for and against this, but implementation and style
|
||
concerns are obvious. Guido has suggested that this should be excluded from
|
||
this PEP on the principle of KISS (Keep It Simple Stupid).
|
||
|
||
|
||
|
||
History
|
||
=======
|
||
29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super"
|
||
- Updated much of the language and added a terminology section
|
||
for clarification in confusing places.
|
||
- Added reference implementation and history sections.
|
||
|
||
06-May-2007 - Updated by Tim Delaney to reflect discussions on the python-3000
|
||
and python-dev mailing lists.
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] Fixing super anyone?
|
||
(https://mail.python.org/pipermail/python-3000/2007-April/006667.html)
|
||
|
||
.. [2] PEP 3130: Access to Module/Class/Function Currently Being Defined (this)
|
||
(https://mail.python.org/pipermail/python-ideas/2007-April/000542.html)
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|