316 lines
12 KiB
Plaintext
316 lines
12 KiB
Plaintext
PEP: 203
|
||
Title: Augmented Assignments
|
||
Version: $Revision$
|
||
Owner: thomas@xs4all.net (Thomas Wouters)
|
||
Python-Version: 2.0
|
||
Status: Draft
|
||
Created: 13-Jul-2000
|
||
Type: Standard
|
||
Post-History: 14-Aug-2000
|
||
|
||
|
||
Introduction
|
||
|
||
This PEP describes the `augmented assignment' proposal for Python
|
||
2.0. This PEP tracks the status and ownership of this feature,
|
||
slated for introduction in Python 2.0. It contains a description
|
||
of the feature and outlines changes necessary to support the
|
||
feature. This PEP summarizes discussions held in mailing list
|
||
forums, and provides URLs for further information where
|
||
appropriate. The CVS revision history of this file contains the
|
||
definitive historical record.
|
||
|
||
|
||
Proposed semantics
|
||
|
||
The proposed patch that adds augmented assignment to Python
|
||
introduces the following new operators:
|
||
|
||
+= -= *= /= %= **= <<= >>= &= ^= |=
|
||
|
||
They implement the same operator as their normal binary form,
|
||
except that the operation is done `in-place' when the left-hand
|
||
side object supports it, and that the left-hand side is only
|
||
evaluated once.
|
||
|
||
They truly behave as augmented assignment, in that they perform
|
||
all of the normal load and store operations, in addition to the
|
||
binary operation they are intended to do. So, given the expression:
|
||
|
||
x += y
|
||
|
||
The object `x' is loaded, then `y' is added to it, and the
|
||
resulting object is stored back in the original place. The precise
|
||
action performed on the two arguments depends on the type of `x',
|
||
and possibly of `y'.
|
||
|
||
The idea behind augmented assignment in Python is that it isn't
|
||
just an easier way to write the common practice of storing the
|
||
result of a binary operation in its left-hand operand, but also a
|
||
way for the left-hand operand in question to know that it should
|
||
operate `on itself', rather than creating a modified copy of
|
||
itself.
|
||
|
||
To make this possible, a number of new `hooks' are added to Python
|
||
classes and C extension types, which are called when the object in
|
||
question is used as the left hand side of an augmented assignment
|
||
operation. If the class or type does not implement the `in-place'
|
||
hooks, the normal hooks for the particular binary operation are
|
||
used.
|
||
|
||
So, given an instance object `x', the expression
|
||
|
||
x += y
|
||
|
||
tries to call x.__add_ab__(y), which is the `in-place' variant of
|
||
__add__. If __add_ab__ is not present, x.__add__(y) is
|
||
attempted, and finally y.__radd__(x) if __add__ is missing too.
|
||
There is no `right-hand-side' variant of __add_ab__, because that
|
||
would require for `y' to know how to in-place modify `x', which is
|
||
unsafe to say the least. The __add_ab__ hook should behave similar
|
||
to __add__, returning the result of the operation (which could be
|
||
`self') which is to be stored in the variable `x'.
|
||
|
||
For C extension types, the `hooks' are members of the
|
||
PyNumberMethods and PySequenceMethods structures, and are called
|
||
in exactly the same manner as the existing non-inplace operations,
|
||
including argument coercion. C methods should also take care to
|
||
return a new reference to the result object, whether it's the same
|
||
object or a new one. So if the original object is returned, it
|
||
should be INCREF()'d appropriately.
|
||
|
||
|
||
Rationale
|
||
|
||
There are two main reasons for adding this feature to Python:
|
||
simplicity of expression, and support for in-place operations. The
|
||
end result is a tradeoff between simplicity of syntax and
|
||
simplicity of expression; like most new features, augmented
|
||
assignment doesn't add anything that was previously impossible. It
|
||
merely makes these things easier to do.
|
||
|
||
Adding augmented assignment will make Python's syntax more complex.
|
||
Instead of a single assignment operation, there are now twelve
|
||
assignment operations, eleven of which also perform an binary
|
||
operation. However, these eleven new forms of assignment are easy
|
||
to understand as the coupling between assignment and the binary
|
||
operation, and they require no large conceptual leap to
|
||
understand. Furthermore, languages that do have augmented
|
||
assignment have shown that they are a popular, much used feature.
|
||
Expressions of the form
|
||
|
||
<x> = <x> <operator> <y>
|
||
|
||
are common enough in those languages to make the extra syntax
|
||
worthwhile, and Python does not have significantly fewer of those
|
||
expressions. Quite the opposite, in fact, since in Python you can
|
||
also concatenate lists with a binary operator, something that is
|
||
done quite frequently. Writing the above expression as
|
||
|
||
<x> <operator>= <y>
|
||
|
||
is both more readable and less error prone, because it is
|
||
instantly obvious to the reader that it is <x> that is being
|
||
changed, and not <x> that is being replaced by something almost,
|
||
but not quite, entirely unlike <x>.
|
||
|
||
The new in-place operations are especially useful to matrix
|
||
calculation and other applications that require large objects. In
|
||
order to efficiently deal with the available program memory, such
|
||
packages cannot blindly use the current binary operations. Because
|
||
these operations always create a new object, adding a single item
|
||
to an existing (large) object would result in copying the entire
|
||
object (which may cause the application to run out of memory), add
|
||
the single item, and then possibly delete the original object,
|
||
depending on reference count.
|
||
|
||
To work around this problem, the packages currently have to use
|
||
methods or functions to modify an object in-place, which is
|
||
definitely less readable than an augmented assignment expression.
|
||
Augmented assignment won't solve all the problems for these
|
||
packages, since some operations cannot be expressed in the limited
|
||
set of binary operators to start with, but it is a start. A
|
||
different PEP[3] is looking at adding new operators.
|
||
|
||
|
||
New methods
|
||
|
||
The proposed implementation adds the following 11 possible `hooks'
|
||
which Python classes can implement to overload the augmented
|
||
assignment operations:
|
||
|
||
__add_ab__
|
||
__sub_ab__
|
||
__mul_ab__
|
||
__div_ab__
|
||
__mod_ab__
|
||
__pow_ab__
|
||
__lshift_ab__
|
||
__rshift_ab__
|
||
__and_ab__
|
||
__xor_ab__
|
||
__or_ab__
|
||
|
||
The `__add_ab__' name is one proposed by Guido[1], and stands for
|
||
`and becomes'. Other proposed names include `__iadd__',
|
||
`__add_in__' and `__inplace_add__'. A firm decision by the BDFL is
|
||
probably needed to finalize this issue.
|
||
|
||
For C extension types, the following struct members are added:
|
||
|
||
To PyNumberMethods:
|
||
binaryfunc nb_inplace_add;
|
||
binaryfunc nb_inplace_subtract;
|
||
binaryfunc nb_inplace_multiply;
|
||
binaryfunc nb_inplace_divide;
|
||
binaryfunc nb_inplace_remainder;
|
||
binaryfunc nb_inplace_power;
|
||
binaryfunc nb_inplace_lshift;
|
||
binaryfunc nb_inplace_rshift;
|
||
binaryfunc nb_inplace_and;
|
||
binaryfunc nb_inplace_xor;
|
||
binaryfunc nb_inplace_or;
|
||
|
||
To PySequenceMethods:
|
||
binaryfunc sq_inplace_concat;
|
||
intargfunc sq_inplace_repeat;
|
||
|
||
In order to keep binary compatibility, the tp_flags TypeObject
|
||
member is used to determine whether the TypeObject in question has
|
||
allocated room for these slots. Until a clean break in binary
|
||
compatibility is made (which may or may not happen before 2.0)
|
||
code that wants to use one of the new struct members must first
|
||
check that they are available with the `PyType_HasFeature()'
|
||
macro:
|
||
|
||
if (PyType_HasFeature(x->ob_type, Py_TPFLAGS_HAVE_INPLACE_OPS) &&
|
||
x->ob_type->tp_as_number && x->ob_type->tp_as_number->nb_inplace_add) {
|
||
/* ... */
|
||
|
||
This check must be made even before testing the method slots for
|
||
NULL values! The macro only tests whether the slots are available,
|
||
not whether they are filled with methods or not.
|
||
|
||
|
||
Implementation
|
||
|
||
The current implementation of augmented assignment[2] adds, in
|
||
addition to the methods and slots already covered, 13 new bytecodes
|
||
and 13 new API functions.
|
||
|
||
The API functions are simply in-place versions of the current
|
||
binary-operation API functions:
|
||
|
||
PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceMultiply(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceDivide(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceRemainder(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlacePower(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceLshift(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceRshift(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceAnd(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceXor(PyObject *o1, PyObject *o2);
|
||
PyNumber_InPlaceOr(PyObject *o1, PyObject *o2);
|
||
PySequence_InPlaceConcat(PyObject *o1, PyObject *o2);
|
||
PySequence_InPlaceRepeat(PyObject *o, int count);
|
||
|
||
They call either the Python class hooks (if either of the objects
|
||
is a Python class instance) or the C type's number or sequence
|
||
methods.
|
||
|
||
The new bytecodes are:
|
||
INPLACE_ADD
|
||
INPLACE_SUBTRACT
|
||
INPLACE_MULTIPLY
|
||
INPLACE_DIVIDE
|
||
INPLACE_REMAINDER
|
||
INPLACE_POWER
|
||
INPLACE_LEFTSHIFT
|
||
INPLACE_RIGHTSHIFT
|
||
INPLACE_AND
|
||
INPLACE_XOR
|
||
INPLACE_OR
|
||
ROT_FOUR
|
||
DUP_TOPX
|
||
|
||
The INPLACE_* bytecodes mirror the BINARY_* bytecodes, except that
|
||
they are implemented as calls to the `InPlace' API functions. The
|
||
other two bytecodes are `utility' bytecodes: ROT_FOUR behaves like
|
||
ROT_THREE except that the four topmost stack items are rotated.
|
||
|
||
DUP_TOPX is a bytecode that takes a single argument, which should
|
||
be an integer between 1 and 5 (inclusive) which is the number of
|
||
items to duplicate in one block. Given a stack like this (where
|
||
the right side of the list is the `top' of the stack):
|
||
|
||
[1, 2, 3, 4, 5]
|
||
|
||
"DUP_TOPX 3" would duplicate the top 3 items, resulting in this
|
||
stack:
|
||
|
||
[1, 2, 3, 4, 5, 3, 4, 5]
|
||
|
||
DUP_TOPX with an argument of 1 is the same as DUP_TOP. The limit
|
||
of 5 is purely an implementation limit. The implementation of
|
||
augmented assignment requires only DUP_TOPX with an argument of 2
|
||
and 3, and could do without this new opcode at the cost of a fair
|
||
number of DUP_TOP and ROT_*.
|
||
|
||
|
||
Open Issues
|
||
|
||
The PyNumber_InPlacePower() function only takes two arguments, not
|
||
one like PyNumber_Power(). This is because there is no way to do
|
||
an inplace three-argument-power trough the augmented assignment
|
||
syntax or the power() function.
|
||
|
||
|
||
Possibly a more obvious name for the Python hooks can be found.
|
||
`_ab_' is what Guido proposed[1] as a working name, and comes from
|
||
an old Algol-68 naming convention.
|
||
|
||
|
||
Documentation needs to be written. The reference manual, the `dis'
|
||
section of the library manual, and possibly the tutorial.
|
||
|
||
|
||
The DUP_TOPX bytecode is a conveniency bytecode, and is not
|
||
actually necessary. It should be considered whether this bytecode
|
||
is worth having. There seems to be no other possible use for this
|
||
bytecode at this time.
|
||
|
||
|
||
The standard library should be adjusted to provide augmented
|
||
assignment hooks, where sensible.
|
||
|
||
|
||
It is not possible to do an inplace operation in the variant of
|
||
|
||
<builtin type> += <instance object>
|
||
|
||
Instead, the instance objects' __radd__ hook is called, with the
|
||
builtin type as argument. The same goes for the other operations.
|
||
It might necessary to add a right-hand version of __add_ab__ after
|
||
all, to support something like that.
|
||
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
References
|
||
|
||
[1] http://www.python.org/pipermail/python-list/2000-June/059556.html
|
||
[2]
|
||
http://sourceforge.net/patch?func=detailpatch&patch_id=100699&group_id=5470
|
||
[3] PEP 211, Adding New Linear Algebra Operators,
|
||
http://python.sourceforge.net/peps/pep-0211.html
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
End:
|