reSTify PEP203 and PEP238 (#268)
This commit is contained in:
parent
12cecb0548
commit
e15c2c15eb
379
pep-0203.txt
379
pep-0203.txt
|
@ -5,194 +5,181 @@ Last-Modified: $Date$
|
|||
Author: thomas@python.org (Thomas Wouters)
|
||||
Status: Final
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 13-Jul-2000
|
||||
Python-Version: 2.0
|
||||
Post-History: 14-Aug-2000
|
||||
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
This PEP describes the `augmented assignment' proposal for Python
|
||||
2.0. This PEP tracks the status and ownership of this feature,
|
||||
slated for introduction in Python 2.0. It contains a description
|
||||
of the feature and outlines changes necessary to support the
|
||||
feature. This PEP summarizes discussions held in mailing list
|
||||
forums, and provides URLs for further information where
|
||||
appropriate. The CVS revision history of this file contains the
|
||||
definitive historical record.
|
||||
This PEP describes the *augmented assignment* proposal for Python 2.0. This
|
||||
PEP tracks the status and ownership of this feature, slated for introduction
|
||||
in Python 2.0. It contains a description of the feature and outlines changes
|
||||
necessary to support the feature. This PEP summarizes discussions held in
|
||||
mailing list forums [1]_, and provides URLs for further information where
|
||||
appropriate. The CVS revision history of this file contains the definitive
|
||||
historical record.
|
||||
|
||||
|
||||
Proposed semantics
|
||||
Proposed Semantics
|
||||
==================
|
||||
|
||||
The proposed patch that adds augmented assignment to Python
|
||||
introduces the following new operators:
|
||||
The proposed patch that adds augmented assignment to Python introduces the
|
||||
following new operators::
|
||||
|
||||
+= -= *= /= %= **= <<= >>= &= ^= |=
|
||||
+= -= *= /= %= **= <<= >>= &= ^= |=
|
||||
|
||||
They implement the same operator as their normal binary form,
|
||||
except that the operation is done `in-place' when the left-hand
|
||||
side object supports it, and that the left-hand side is only
|
||||
evaluated once.
|
||||
They implement the same operator as their normal binary form, except that the
|
||||
operation is done *in-place* when the left-hand side object supports it, and
|
||||
that the left-hand side is only evaluated once.
|
||||
|
||||
They truly behave as augmented assignment, in that they perform
|
||||
all of the normal load and store operations, in addition to the
|
||||
binary operation they are intended to do. So, given the expression:
|
||||
They truly behave as augmented assignment, in that they perform all of the
|
||||
normal load and store operations, in addition to the binary operation they are
|
||||
intended to do. So, given the expression::
|
||||
|
||||
x += y
|
||||
|
||||
The object `x' is loaded, then `y' is added to it, and the
|
||||
resulting object is stored back in the original place. The precise
|
||||
action performed on the two arguments depends on the type of `x',
|
||||
and possibly of `y'.
|
||||
The object ``x`` is loaded, then ``y`` is added to it, and the resulting
|
||||
object is stored back in the original place. The precise action performed on
|
||||
the two arguments depends on the type of ``x``, and possibly of ``y``.
|
||||
|
||||
The idea behind augmented assignment in Python is that it isn't
|
||||
just an easier way to write the common practice of storing the
|
||||
result of a binary operation in its left-hand operand, but also a
|
||||
way for the left-hand operand in question to know that it should
|
||||
operate `on itself', rather than creating a modified copy of
|
||||
itself.
|
||||
The idea behind augmented assignment in Python is that it isn't just an easier
|
||||
way to write the common practice of storing the result of a binary operation
|
||||
in its left-hand operand, but also a way for the left-hand operand in question
|
||||
to know that it should operate *on itself*, rather than creating a modified
|
||||
copy of itself.
|
||||
|
||||
To make this possible, a number of new `hooks' are added to Python
|
||||
classes and C extension types, which are called when the object in
|
||||
question is used as the left hand side of an augmented assignment
|
||||
operation. If the class or type does not implement the `in-place'
|
||||
hooks, the normal hooks for the particular binary operation are
|
||||
used.
|
||||
To make this possible, a number of new *hooks* are added to Python classes and
|
||||
C extension types, which are called when the object in question is used as the
|
||||
left hand side of an augmented assignment operation. If the class or type
|
||||
does not implement the *in-place* hooks, the normal hooks for the particular
|
||||
binary operation are used.
|
||||
|
||||
So, given an instance object `x', the expression
|
||||
So, given an instance object ``x``, the expression::
|
||||
|
||||
x += y
|
||||
|
||||
tries to call x.__iadd__(y), which is the `in-place' variant of
|
||||
__add__. If __iadd__ is not present, x.__add__(y) is attempted,
|
||||
and finally y.__radd__(x) if __add__ is missing too. There is no
|
||||
`right-hand-side' variant of __iadd__, because that would require
|
||||
for `y' to know how to in-place modify `x', which is unsafe to say
|
||||
the least. The __iadd__ hook should behave similar to __add__,
|
||||
returning the result of the operation (which could be `self')
|
||||
which is to be assigned to the variable `x'.
|
||||
tries to call ``x.__iadd__(y)``, which is the *in-place* variant of
|
||||
``__add__`` . If ``__iadd__`` is not present, ``x.__add__(y)`` is attempted,
|
||||
and finally ``y.__radd__(x)`` if ``__add__`` is missing too. There is no
|
||||
*right-hand-side* variant of ``__iadd__``, because that would require for
|
||||
``y`` to know how to in-place modify ``x``, which is unsafe to say the least.
|
||||
The ``__iadd__`` hook should behave similar to ``__add__``, returning the
|
||||
result of the operation (which could be ``self``) which is to be assigned to
|
||||
the variable ``x``.
|
||||
|
||||
For C extension types, the `hooks' are members of the
|
||||
PyNumberMethods and PySequenceMethods structures. Some special
|
||||
semantics apply to make the use of these methods, and the mixing
|
||||
of Python instance objects and C types, as unsurprising as
|
||||
possible.
|
||||
For C extension types, the *hooks* are members of the ``PyNumberMethods`` and
|
||||
``PySequenceMethods`` structures. Some special semantics apply to make the
|
||||
use of these methods, and the mixing of Python instance objects and C types,
|
||||
as unsurprising as possible.
|
||||
|
||||
In the generic case of `x <augop> y' (or a similar case using the
|
||||
PyNumber_InPlace API functions) the principal object being
|
||||
operated on is `x'. This differs from normal binary operations,
|
||||
where `x' and `y' could be considered `co-operating', because
|
||||
unlike in binary operations, the operands in an in-place operation
|
||||
cannot be swapped. However, in-place operations do fall back to
|
||||
normal binary operations when in-place modification is not
|
||||
supported, resuling in the following rules:
|
||||
In the generic case of ``x <augop> y`` (or a similar case using the
|
||||
``PyNumber_InPlace`` API functions) the principal object being operated on is
|
||||
``x``. This differs from normal binary operations, where ``x`` and ``y``
|
||||
could be considered *co-operating*, because unlike in binary operations, the
|
||||
operands in an in-place operation cannot be swapped. However, in-place
|
||||
operations do fall back to normal binary operations when in-place modification
|
||||
is not supported, resulting in the following rules:
|
||||
|
||||
- If the left-hand object (`x') is an instance object, and it
|
||||
has a `__coerce__' method, call that function with `y' as the
|
||||
argument. If coercion succeeds, and the resulting left-hand
|
||||
object is a different object than `x', stop processing it as
|
||||
in-place and call the appropriate function for the normal binary
|
||||
operation, with the coerced `x' and `y' as arguments. The result
|
||||
of the operation is whatever that function returns.
|
||||
- If the left-hand object (``x``) is an instance object, and it has a
|
||||
``__coerce__`` method, call that function with ``y`` as the argument. If
|
||||
coercion succeeds, and the resulting left-hand object is a different object
|
||||
than ``x``, stop processing it as in-place and call the appropriate function
|
||||
for the normal binary operation, with the coerced ``x`` and ``y`` as
|
||||
arguments. The result of the operation is whatever that function returns.
|
||||
|
||||
If coercion does not yield a different object for `x', or `x'
|
||||
does not define a `__coerce__' method, and `x' has the
|
||||
appropriate `__ihook__' for this operation, call that method
|
||||
with `y' as the argument, and the result of the operation is
|
||||
whatever that method returns.
|
||||
If coercion does not yield a different object for ``x``, or ``x`` does not
|
||||
define a ``__coerce__`` method, and ``x`` has the appropriate ``__ihook__``
|
||||
for this operation, call that method with ``y`` as the argument, and the
|
||||
result of the operation is whatever that method returns.
|
||||
|
||||
- Otherwise, if the left-hand object is not an instance object,
|
||||
but its type does define the in-place function for this
|
||||
operation, call that function with `x' and `y' as the arguments,
|
||||
and the result of the operation is whatever that function
|
||||
returns.
|
||||
- Otherwise, if the left-hand object is not an instance object, but its type
|
||||
does define the in-place function for this operation, call that function
|
||||
with ``x`` and ``y`` as the arguments, and the result of the operation is
|
||||
whatever that function returns.
|
||||
|
||||
Note that no coercion on either `x' or `y' is done in this case,
|
||||
and it's perfectly valid for a C type to receive an instance
|
||||
object as the second argument; that is something that cannot
|
||||
happen with normal binary operations.
|
||||
Note that no coercion on either ``x`` or ``y`` is done in this case, and
|
||||
it's perfectly valid for a C type to receive an instance object as the
|
||||
second argument; that is something that cannot happen with normal binary
|
||||
operations.
|
||||
|
||||
- Otherwise, process it exactly as a normal binary operation (not
|
||||
in-place), including argument coercion. In short, if either
|
||||
argument is an instance object, resolve the operation through
|
||||
`__coerce__', `__hook__' and `__rhook__'. Otherwise, both
|
||||
objects are C types, and they are coerced and passed to the
|
||||
appropriate function.
|
||||
- Otherwise, process it exactly as a normal binary operation (not in-place),
|
||||
including argument coercion. In short, if either argument is an instance
|
||||
object, resolve the operation through ``__coerce__``, ``__hook__`` and
|
||||
``__rhook__``. Otherwise, both objects are C types, and they are coerced
|
||||
and passed to the appropriate function.
|
||||
|
||||
- If no way to process the operation can be found, raise a
|
||||
TypeError with an error message specific to the operation.
|
||||
- If no way to process the operation can be found, raise a ``TypeError`` with
|
||||
an error message specific to the operation.
|
||||
|
||||
- Some special casing exists to account for the case of `+' and
|
||||
`*', which have a special meaning for sequences: for `+',
|
||||
sequence concatenation, no coercion what so ever is done if a C
|
||||
type defines sq_concat or sq_inplace_concat. For `*', sequence
|
||||
repeating, `y' is converted to a C integer before calling either
|
||||
sq_inplace_repeat and sq_repeat. This is done even if `y' is an
|
||||
instance, though not if `x' is an instance.
|
||||
- Some special casing exists to account for the case of ``+`` and ``*``,
|
||||
which have a special meaning for sequences: for ``+``, sequence
|
||||
concatenation, no coercion what so ever is done if a C type defines
|
||||
``sq_concat`` or ``sq_inplace_concat``. For ``*``, sequence repeating,
|
||||
``y`` is converted to a C integer before calling either
|
||||
``sq_inplace_repeat`` and ``sq_repeat``. This is done even if ``y`` is an
|
||||
instance, though not if ``x`` is an instance.
|
||||
|
||||
The in-place function should always return a new reference, either
|
||||
to the old `x' object if the operation was indeed performed
|
||||
in-place, or to a new object.
|
||||
The in-place function should always return a new reference, either to the
|
||||
old ``x`` object if the operation was indeed performed in-place, or to a new
|
||||
object.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
There are two main reasons for adding this feature to Python:
|
||||
simplicity of expression, and support for in-place operations. The
|
||||
end result is a tradeoff between simplicity of syntax and
|
||||
simplicity of expression; like most new features, augmented
|
||||
assignment doesn't add anything that was previously impossible. It
|
||||
merely makes these things easier to do.
|
||||
There are two main reasons for adding this feature to Python: simplicity of
|
||||
expression, and support for in-place operations. The end result is a tradeoff
|
||||
between simplicity of syntax and simplicity of expression; like most new
|
||||
features, augmented assignment doesn't add anything that was previously
|
||||
impossible. It merely makes these things easier to do.
|
||||
|
||||
Adding augmented assignment will make Python's syntax more complex.
|
||||
Instead of a single assignment operation, there are now twelve
|
||||
assignment operations, eleven of which also perform a binary
|
||||
operation. However, these eleven new forms of assignment are easy
|
||||
to understand as the coupling between assignment and the binary
|
||||
operation, and they require no large conceptual leap to
|
||||
understand. Furthermore, languages that do have augmented
|
||||
assignment have shown that they are a popular, much used feature.
|
||||
Expressions of the form
|
||||
Adding augmented assignment will make Python's syntax more complex. Instead
|
||||
of a single assignment operation, there are now twelve assignment operations,
|
||||
eleven of which also perform a binary operation. However, these eleven new
|
||||
forms of assignment are easy to understand as the coupling between assignment
|
||||
and the binary operation, and they require no large conceptual leap to
|
||||
understand. Furthermore, languages that do have augmented assignment have
|
||||
shown that they are a popular, much used feature. Expressions of the form::
|
||||
|
||||
<x> = <x> <operator> <y>
|
||||
|
||||
are common enough in those languages to make the extra syntax
|
||||
worthwhile, and Python does not have significantly fewer of those
|
||||
expressions. Quite the opposite, in fact, since in Python you can
|
||||
also concatenate lists with a binary operator, something that is
|
||||
done quite frequently. Writing the above expression as
|
||||
are common enough in those languages to make the extra syntax worthwhile, and
|
||||
Python does not have significantly fewer of those expressions. Quite the
|
||||
opposite, in fact, since in Python you can also concatenate lists with a
|
||||
binary operator, something that is done quite frequently. Writing the above
|
||||
expression as::
|
||||
|
||||
<x> <operator>= <y>
|
||||
|
||||
is both more readable and less error prone, because it is
|
||||
instantly obvious to the reader that it is <x> that is being
|
||||
changed, and not <x> that is being replaced by something almost,
|
||||
but not quite, entirely unlike <x>.
|
||||
is both more readable and less error prone, because it is instantly obvious to
|
||||
the reader that it is ``<x>`` that is being changed, and not ``<x>`` that is
|
||||
being replaced by something almost, but not quite, entirely unlike ``<x>``.
|
||||
|
||||
The new in-place operations are especially useful to matrix
|
||||
calculation and other applications that require large objects. In
|
||||
order to efficiently deal with the available program memory, such
|
||||
packages cannot blindly use the current binary operations. Because
|
||||
these operations always create a new object, adding a single item
|
||||
to an existing (large) object would result in copying the entire
|
||||
object (which may cause the application to run out of memory), add
|
||||
the single item, and then possibly delete the original object,
|
||||
depending on reference count.
|
||||
The new in-place operations are especially useful to matrix calculation and
|
||||
other applications that require large objects. In order to efficiently deal
|
||||
with the available program memory, such packages cannot blindly use the
|
||||
current binary operations. Because these operations always create a new
|
||||
object, adding a single item to an existing (large) object would result in
|
||||
copying the entire object (which may cause the application to run out of
|
||||
memory), add the single item, and then possibly delete the original object,
|
||||
depending on reference count.
|
||||
|
||||
To work around this problem, the packages currently have to use
|
||||
methods or functions to modify an object in-place, which is
|
||||
definitely less readable than an augmented assignment expression.
|
||||
Augmented assignment won't solve all the problems for these
|
||||
packages, since some operations cannot be expressed in the limited
|
||||
set of binary operators to start with, but it is a start. A
|
||||
different PEP[2] is looking at adding new operators.
|
||||
To work around this problem, the packages currently have to use methods or
|
||||
functions to modify an object in-place, which is definitely less readable than
|
||||
an augmented assignment expression. Augmented assignment won't solve all the
|
||||
problems for these packages, since some operations cannot be expressed in the
|
||||
limited set of binary operators to start with, but it is a start. A
|
||||
different PEP [3]_ is looking at adding new operators.
|
||||
|
||||
|
||||
New methods
|
||||
===========
|
||||
|
||||
The proposed implementation adds the following 11 possible `hooks'
|
||||
which Python classes can implement to overload the augmented
|
||||
assignment operations:
|
||||
The proposed implementation adds the following 11 possible *hooks* which
|
||||
Python classes can implement to overload the augmented assignment operations::
|
||||
|
||||
__iadd__
|
||||
__isub__
|
||||
|
@ -206,11 +193,12 @@ New methods
|
|||
__ixor__
|
||||
__ior__
|
||||
|
||||
The `i' in `__iadd__' stands for `in-place'.
|
||||
The *i* in ``__iadd__`` stands for *in-place*.
|
||||
|
||||
For C extension types, the following struct members are added:
|
||||
For C extension types, the following struct members are added.
|
||||
|
||||
To ``PyNumberMethods``::
|
||||
|
||||
To PyNumberMethods:
|
||||
binaryfunc nb_inplace_add;
|
||||
binaryfunc nb_inplace_subtract;
|
||||
binaryfunc nb_inplace_multiply;
|
||||
|
@ -223,35 +211,36 @@ New methods
|
|||
binaryfunc nb_inplace_xor;
|
||||
binaryfunc nb_inplace_or;
|
||||
|
||||
To PySequenceMethods:
|
||||
To ``PySequenceMethods``::
|
||||
|
||||
binaryfunc sq_inplace_concat;
|
||||
intargfunc sq_inplace_repeat;
|
||||
|
||||
In order to keep binary compatibility, the tp_flags TypeObject
|
||||
member is used to determine whether the TypeObject in question has
|
||||
allocated room for these slots. Until a clean break in binary
|
||||
compatibility is made (which may or may not happen before 2.0)
|
||||
code that wants to use one of the new struct members must first
|
||||
check that they are available with the `PyType_HasFeature()'
|
||||
macro:
|
||||
In order to keep binary compatibility, the ``tp_flags`` TypeObject member is
|
||||
used to determine whether the TypeObject in question has allocated room for
|
||||
these slots. Until a clean break in binary compatibility is made (which may
|
||||
or may not happen before 2.0) code that wants to use one of the new struct
|
||||
members must first check that they are available with the
|
||||
``PyType_HasFeature()`` macro::
|
||||
|
||||
if (PyType_HasFeature(x->ob_type, Py_TPFLAGS_HAVE_INPLACE_OPS) &&
|
||||
x->ob_type->tp_as_number && x->ob_type->tp_as_number->nb_inplace_add) {
|
||||
/* ... */
|
||||
|
||||
This check must be made even before testing the method slots for
|
||||
NULL values! The macro only tests whether the slots are available,
|
||||
not whether they are filled with methods or not.
|
||||
This check must be made even before testing the method slots for ``NULL``
|
||||
values! The macro only tests whether the slots are available, not whether
|
||||
they are filled with methods or not.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
The current implementation of augmented assignment[1] adds, in
|
||||
addition to the methods and slots already covered, 13 new bytecodes
|
||||
and 13 new API functions.
|
||||
The current implementation of augmented assignment [2]_ adds, in addition to
|
||||
the methods and slots already covered, 13 new bytecodes and 13 new API
|
||||
functions.
|
||||
|
||||
The API functions are simply in-place versions of the current
|
||||
binary-operation API functions:
|
||||
The API functions are simply in-place versions of the current binary-operation
|
||||
API functions::
|
||||
|
||||
PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2);
|
||||
PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2);
|
||||
|
@ -267,11 +256,11 @@ Implementation
|
|||
PySequence_InPlaceConcat(PyObject *o1, PyObject *o2);
|
||||
PySequence_InPlaceRepeat(PyObject *o, int count);
|
||||
|
||||
They call either the Python class hooks (if either of the objects
|
||||
is a Python class instance) or the C type's number or sequence
|
||||
methods.
|
||||
They call either the Python class hooks (if either of the objects is a Python
|
||||
class instance) or the C type's number or sequence methods.
|
||||
|
||||
The new bytecodes are::
|
||||
|
||||
The new bytecodes are:
|
||||
INPLACE_ADD
|
||||
INPLACE_SUBTRACT
|
||||
INPLACE_MULTIPLY
|
||||
|
@ -286,61 +275,61 @@ Implementation
|
|||
ROT_FOUR
|
||||
DUP_TOPX
|
||||
|
||||
The INPLACE_* bytecodes mirror the BINARY_* bytecodes, except that
|
||||
they are implemented as calls to the `InPlace' API functions. The
|
||||
other two bytecodes are `utility' bytecodes: ROT_FOUR behaves like
|
||||
ROT_THREE except that the four topmost stack items are rotated.
|
||||
The ``INPLACE_*`` bytecodes mirror the ``BINARY_*`` bytecodes, except that
|
||||
they are implemented as calls to the ``InPlace`` API functions. The other two
|
||||
bytecodes are *utility* bytecodes: ``ROT_FOUR`` behaves like ``ROT_THREE``
|
||||
except that the four topmost stack items are rotated.
|
||||
|
||||
DUP_TOPX is a bytecode that takes a single argument, which should
|
||||
be an integer between 1 and 5 (inclusive) which is the number of
|
||||
items to duplicate in one block. Given a stack like this (where
|
||||
the right side of the list is the `top' of the stack):
|
||||
``DUP_TOPX`` is a bytecode that takes a single argument, which should be an
|
||||
integer between 1 and 5 (inclusive) which is the number of items to duplicate
|
||||
in one block. Given a stack like this (where the right side of the list is
|
||||
the *top* of the stack)::
|
||||
|
||||
[1, 2, 3, 4, 5]
|
||||
|
||||
"DUP_TOPX 3" would duplicate the top 3 items, resulting in this
|
||||
stack:
|
||||
``DUP_TOPX 3`` would duplicate the top 3 items, resulting in this stack::
|
||||
|
||||
[1, 2, 3, 4, 5, 3, 4, 5]
|
||||
|
||||
DUP_TOPX with an argument of 1 is the same as DUP_TOP. The limit
|
||||
of 5 is purely an implementation limit. The implementation of
|
||||
augmented assignment requires only DUP_TOPX with an argument of 2
|
||||
and 3, and could do without this new opcode at the cost of a fair
|
||||
number of DUP_TOP and ROT_*.
|
||||
``DUP_TOPX`` with an argument of 1 is the same as ``DUP_TOP``. The limit of 5
|
||||
is purely an implementation limit . The implementation of augmented
|
||||
assignment requires only ``DUP_TOPX`` with an argument of 2 and 3, and could
|
||||
do without this new opcode at the cost of a fair number of ``DUP_TOP`` and
|
||||
``ROT_*``.
|
||||
|
||||
|
||||
Open Issues
|
||||
===========
|
||||
|
||||
The PyNumber_InPlace API is only a subset of the normal PyNumber
|
||||
API: only those functions that are required to support the
|
||||
augmented assignment syntax are included. If other in-place API
|
||||
functions are needed, they can be added later.
|
||||
The ``PyNumber_InPlace`` API is only a subset of the normal ``PyNumber`` API:
|
||||
only those functions that are required to support the augmented assignment
|
||||
syntax are included. If other in-place API functions are needed, they can be
|
||||
added later.
|
||||
|
||||
|
||||
The DUP_TOPX bytecode is a conveniency bytecode, and is not
|
||||
actually necessary. It should be considered whether this bytecode
|
||||
is worth having. There seems to be no other possible use for this
|
||||
bytecode at this time.
|
||||
The ``DUP_TOPX`` bytecode is a conveniency bytecode, and is not actually
|
||||
necessary. It should be considered whether this bytecode is worth having.
|
||||
There seems to be no other possible use for this bytecode at this time.
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
This document has been placed in the public domain.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
[1] http://www.python.org/pipermail/python-list/2000-June/059556.html
|
||||
.. [1] http://www.python.org/pipermail/python-list/2000-June/059556.html
|
||||
|
||||
[2] http://sourceforge.net/patch?func=detailpatch&patch_id=100699&group_id=5470
|
||||
.. [2] http://sourceforge.net/patch?func=detailpatch&patch_id=100699&group_id=5470
|
||||
|
||||
[3] PEP 211, Adding A New Outer Product Operator, Wilson
|
||||
.. [3] PEP 211, Adding A New Outer Product Operator, Wilson
|
||||
http://www.python.org/dev/peps/pep-0211/
|
||||
|
||||
|
||||
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
End:
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
End:
|
||||
|
|
696
pep-0238.txt
696
pep-0238.txt
|
@ -2,222 +2,209 @@ PEP: 238
|
|||
Title: Changing the Division Operator
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: moshez@zadka.site.co.il (Moshe Zadka), guido@python.org (Guido van Rossum)
|
||||
Author: moshez@zadka.site.co.il (Moshe Zadka),
|
||||
guido@python.org (Guido van Rossum)
|
||||
Status: Final
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 11-Mar-2001
|
||||
Python-Version: 2.2
|
||||
Post-History: 16-Mar-2001, 26-Jul-2001, 27-Jul-2001
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
The current division (/) operator has an ambiguous meaning for
|
||||
numerical arguments: it returns the floor of the mathematical
|
||||
result of division if the arguments are ints or longs, but it
|
||||
returns a reasonable approximation of the division result if the
|
||||
arguments are floats or complex. This makes expressions expecting
|
||||
float or complex results error-prone when integers are not
|
||||
expected but possible as inputs.
|
||||
The current division (``/``) operator has an ambiguous meaning for numerical
|
||||
arguments: it returns the floor of the mathematical result of division if the
|
||||
arguments are ints or longs, but it returns a reasonable approximation of the
|
||||
division result if the arguments are floats or complex. This makes
|
||||
expressions expecting float or complex results error-prone when integers are
|
||||
not expected but possible as inputs.
|
||||
|
||||
We propose to fix this by introducing different operators for
|
||||
different operations: x/y to return a reasonable approximation of
|
||||
the mathematical result of the division ("true division"), x//y to
|
||||
return the floor ("floor division"). We call the current, mixed
|
||||
meaning of x/y "classic division".
|
||||
We propose to fix this by introducing different operators for different
|
||||
operations: ``x/y`` to return a reasonable approximation of the mathematical
|
||||
result of the division ("true division"), ``x//y`` to return the floor
|
||||
("floor division"). We call the current, mixed meaning of x/y
|
||||
"classic division".
|
||||
|
||||
Because of severe backwards compatibility issues, not to mention a
|
||||
major flamewar on c.l.py, we propose the following transitional
|
||||
measures (starting with Python 2.2):
|
||||
Because of severe backwards compatibility issues, not to mention a major
|
||||
flamewar on c.l.py, we propose the following transitional measures (starting
|
||||
with Python 2.2):
|
||||
|
||||
- Classic division will remain the default in the Python 2.x
|
||||
series; true division will be standard in Python 3.0.
|
||||
- Classic division will remain the default in the Python 2.x series; true
|
||||
division will be standard in Python 3.0.
|
||||
|
||||
- The // operator will be available to request floor division
|
||||
- The ``//`` operator will be available to request floor division
|
||||
unambiguously.
|
||||
|
||||
- The future division statement, spelled "from __future__ import
|
||||
division", will change the / operator to mean true division
|
||||
throughout the module.
|
||||
- The future division statement, spelled ``from __future__ import division``,
|
||||
will change the ``/`` operator to mean true division throughout the module.
|
||||
|
||||
- A command line option will enable run-time warnings for classic
|
||||
division applied to int or long arguments; another command line
|
||||
option will make true division the default.
|
||||
- A command line option will enable run-time warnings for classic division
|
||||
applied to int or long arguments; another command line option will make true
|
||||
division the default.
|
||||
|
||||
- The standard library will use the future division statement and
|
||||
the // operator when appropriate, so as to completely avoid
|
||||
classic division.
|
||||
- The standard library will use the future division statement and the ``//``
|
||||
operator when appropriate, so as to completely avoid classic division.
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
||||
The classic division operator makes it hard to write numerical
|
||||
expressions that are supposed to give correct results from
|
||||
arbitrary numerical inputs. For all other operators, one can
|
||||
write down a formula such as x*y**2 + z, and the calculated result
|
||||
will be close to the mathematical result (within the limits of
|
||||
numerical accuracy, of course) for any numerical input type (int,
|
||||
long, float, or complex). But division poses a problem: if the
|
||||
expressions for both arguments happen to have an integral type, it
|
||||
implements floor division rather than true division.
|
||||
The classic division operator makes it hard to write numerical expressions
|
||||
that are supposed to give correct results from arbitrary numerical inputs.
|
||||
For all other operators, one can write down a formula such as ``x*y**2 + z``,
|
||||
and the calculated result will be close to the mathematical result (within the
|
||||
limits of numerical accuracy, of course) for any numerical input type (int,
|
||||
long, float, or complex). But division poses a problem: if the expressions
|
||||
for both arguments happen to have an integral type, it implements floor
|
||||
division rather than true division.
|
||||
|
||||
The problem is unique to dynamically typed languages: in a
|
||||
statically typed language like C, the inputs, typically function
|
||||
arguments, would be declared as double or float, and when a call
|
||||
passes an integer argument, it is converted to double or float at
|
||||
the time of the call. Python doesn't have argument type
|
||||
declarations, so integer arguments can easily find their way into
|
||||
an expression.
|
||||
The problem is unique to dynamically typed languages: in a statically typed
|
||||
language like C, the inputs, typically function arguments, would be declared
|
||||
as double or float, and when a call passes an integer argument, it is
|
||||
converted to double or float at the time of the call. Python doesn't have
|
||||
argument type declarations, so integer arguments can easily find their way
|
||||
into an expression.
|
||||
|
||||
The problem is particularly pernicious since ints are perfect
|
||||
substitutes for floats in all other circumstances: math.sqrt(2)
|
||||
returns the same value as math.sqrt(2.0), 3.14*100 and 3.14*100.0
|
||||
return the same value, and so on. Thus, the author of a numerical
|
||||
routine may only use floating point numbers to test his code, and
|
||||
believe that it works correctly, and a user may accidentally pass
|
||||
in an integer input value and get incorrect results.
|
||||
The problem is particularly pernicious since ints are perfect substitutes for
|
||||
floats in all other circumstances: ``math.sqrt(2)`` returns the same value as
|
||||
``math.sqrt(2.0)``, ``3.14*100`` and ``3.14*100.0`` return the same value, and
|
||||
so on. Thus, the author of a numerical routine may only use floating point
|
||||
numbers to test his code, and believe that it works correctly, and a user may
|
||||
accidentally pass in an integer input value and get incorrect results.
|
||||
|
||||
Another way to look at this is that classic division makes it
|
||||
difficult to write polymorphic functions that work well with
|
||||
either float or int arguments; all other operators already do the
|
||||
right thing. No algorithm that works for both ints and floats has
|
||||
a need for truncating division in one case and true division in
|
||||
the other.
|
||||
Another way to look at this is that classic division makes it difficult to
|
||||
write polymorphic functions that work well with either float or int arguments;
|
||||
all other operators already do the right thing. No algorithm that works for
|
||||
both ints and floats has a need for truncating division in one case and true
|
||||
division in the other.
|
||||
|
||||
The correct work-around is subtle: casting an argument to float()
|
||||
is wrong if it could be a complex number; adding 0.0 to an
|
||||
argument doesn't preserve the sign of the argument if it was minus
|
||||
zero. The only solution without either downside is multiplying an
|
||||
argument (typically the first) by 1.0. This leaves the value and
|
||||
sign unchanged for float and complex, and turns int and long into
|
||||
a float with the corresponding value.
|
||||
The correct work-around is subtle: casting an argument to float() is wrong if
|
||||
it could be a complex number; adding 0.0 to an argument doesn't preserve the
|
||||
sign of the argument if it was minus zero. The only solution without either
|
||||
downside is multiplying an argument (typically the first) by 1.0. This leaves
|
||||
the value and sign unchanged for float and complex, and turns int and long
|
||||
into a float with the corresponding value.
|
||||
|
||||
It is the opinion of the authors that this is a real design bug in
|
||||
Python, and that it should be fixed sooner rather than later.
|
||||
Assuming Python usage will continue to grow, the cost of leaving
|
||||
this bug in the language will eventually outweigh the cost of
|
||||
fixing old code -- there is an upper bound to the amount of code
|
||||
to be fixed, but the amount of code that might be affected by the
|
||||
bug in the future is unbounded.
|
||||
It is the opinion of the authors that this is a real design bug in Python, and
|
||||
that it should be fixed sooner rather than later. Assuming Python usage will
|
||||
continue to grow, the cost of leaving this bug in the language will eventually
|
||||
outweigh the cost of fixing old code -- there is an upper bound to the amount
|
||||
of code to be fixed, but the amount of code that might be affected by the bug
|
||||
in the future is unbounded.
|
||||
|
||||
Another reason for this change is the desire to ultimately unify
|
||||
Python's numeric model. This is the subject of PEP 228[0] (which
|
||||
is currently incomplete). A unified numeric model removes most of
|
||||
the user's need to be aware of different numerical types. This is
|
||||
good for beginners, but also takes away concerns about different
|
||||
numeric behavior for advanced programmers. (Of course, it won't
|
||||
remove concerns about numerical stability and accuracy.)
|
||||
Another reason for this change is the desire to ultimately unify Python's
|
||||
numeric model. This is the subject of PEP 228 [0]_ (which is currently
|
||||
incomplete). A unified numeric model removes most of the user's need to be
|
||||
aware of different numerical types. This is good for beginners, but also
|
||||
takes away concerns about different numeric behavior for advanced programmers.
|
||||
(Of course, it won't remove concerns about numerical stability and accuracy.)
|
||||
|
||||
In a unified numeric model, the different types (int, long, float,
|
||||
complex, and possibly others, such as a new rational type) serve
|
||||
mostly as storage optimizations, and to some extent to indicate
|
||||
orthogonal properties such as inexactness or complexity. In a
|
||||
unified model, the integer 1 should be indistinguishable from the
|
||||
floating point number 1.0 (except for its inexactness), and both
|
||||
should behave the same in all numeric contexts. Clearly, in a
|
||||
unified numeric model, if a==b and c==d, a/c should equal b/d
|
||||
(taking some liberties due to rounding for inexact numbers), and
|
||||
since everybody agrees that 1.0/2.0 equals 0.5, 1/2 should also
|
||||
equal 0.5. Likewise, since 1//2 equals zero, 1.0//2.0 should also
|
||||
equal zero.
|
||||
In a unified numeric model, the different types (int, long, float, complex,
|
||||
and possibly others, such as a new rational type) serve mostly as storage
|
||||
optimizations, and to some extent to indicate orthogonal properties such as
|
||||
inexactness or complexity. In a unified model, the integer 1 should be
|
||||
indistinguishable from the floating point number 1.0 (except for its
|
||||
inexactness), and both should behave the same in all numeric contexts.
|
||||
Clearly, in a unified numeric model, if ``a==b`` and ``c==d``, ``a/c`` should
|
||||
equal ``b/d`` (taking some liberties due to rounding for inexact numbers), and
|
||||
since everybody agrees that ``1.0/2.0`` equals 0.5, ``1/2`` should also equal
|
||||
0.5. Likewise, since ``1//2`` equals zero, ``1.0//2.0`` should also equal
|
||||
zero.
|
||||
|
||||
|
||||
Variations
|
||||
==========
|
||||
|
||||
Aesthetically, x//y doesn't please everyone, and hence several
|
||||
variations have been proposed. They are addressed here:
|
||||
Aesthetically, ``x//y`` doesn't please everyone, and hence several variations
|
||||
have been proposed. They are addressed here:
|
||||
|
||||
- x div y. This would introduce a new keyword. Since div is a
|
||||
popular identifier, this would break a fair amount of existing
|
||||
code, unless the new keyword was only recognized under a future
|
||||
division statement. Since it is expected that the majority of
|
||||
code that needs to be converted is dividing integers, this would
|
||||
greatly increase the need for the future division statement.
|
||||
Even with a future statement, the general sentiment against
|
||||
adding new keywords unless absolutely necessary argues against
|
||||
this.
|
||||
- ``x div y``. This would introduce a new keyword. Since ``div`` is a
|
||||
popular identifier, this would break a fair amount of existing code, unless
|
||||
the new keyword was only recognized under a future division statement.
|
||||
Since it is expected that the majority of code that needs to be converted is
|
||||
dividing integers, this would greatly increase the need for the future
|
||||
division statement. Even with a future statement, the general sentiment
|
||||
against adding new keywords unless absolutely necessary argues against this.
|
||||
|
||||
- div(x, y). This makes the conversion of old code much harder.
|
||||
Replacing x/y with x//y or x div y can be done with a simple
|
||||
query replace; in most cases the programmer can easily verify
|
||||
that a particular module only works with integers so all
|
||||
occurrences of x/y can be replaced. (The query replace is still
|
||||
needed to weed out slashes occurring in comments or string
|
||||
literals.) Replacing x/y with div(x, y) would require a much
|
||||
more intelligent tool, since the extent of the expressions to
|
||||
the left and right of the / must be analyzed before the
|
||||
placement of the "div(" and ")" part can be decided.
|
||||
- ``div(x, y)``. This makes the conversion of old code much harder.
|
||||
Replacing ``x/y`` with ``x//y`` or ``x div y`` can be done with a simple
|
||||
query replace; in most cases the programmer can easily verify that a
|
||||
particular module only works with integers so all occurrences of ``x/y`` can
|
||||
be replaced. (The query replace is still needed to weed out slashes
|
||||
occurring in comments or string literals.) Replacing ``x/y`` with
|
||||
``div(x, y)`` would require a much more intelligent tool, since the extent
|
||||
of the expressions to the left and right of the ``/`` must be analyzed
|
||||
before the placement of the ``div(`` and ``)`` part can be decided.
|
||||
|
||||
- x \ y. The backslash is already a token, meaning line
|
||||
continuation, and in general it suggests an "escape" to Unix
|
||||
eyes. In addition (this due to Terry Reedy) this would make
|
||||
things like eval("x\y") harder to get right.
|
||||
- ``x \ y``. The backslash is already a token, meaning line continuation, and
|
||||
in general it suggests an *escape* to Unix eyes. In addition (this due to
|
||||
Terry Reedy) this would make things like ``eval("x\y")`` harder to get
|
||||
right.
|
||||
|
||||
|
||||
Alternatives
|
||||
============
|
||||
|
||||
In order to reduce the amount of old code that needs to be
|
||||
converted, several alternative proposals have been put forth.
|
||||
Here is a brief discussion of each proposal (or category of
|
||||
proposals). If you know of an alternative that was discussed on
|
||||
c.l.py that isn't mentioned here, please mail the second author.
|
||||
In order to reduce the amount of old code that needs to be converted, several
|
||||
alternative proposals have been put forth. Here is a brief discussion of each
|
||||
proposal (or category of proposals). If you know of an alternative that was
|
||||
discussed on c.l.py that isn't mentioned here, please mail the second author.
|
||||
|
||||
- Let / keep its classic semantics; introduce // for true
|
||||
division. This still leaves a broken operator in the language,
|
||||
and invites to use the broken behavior. It also shuts off the
|
||||
road to a unified numeric model a la PEP 228[0].
|
||||
- Let ``/`` keep its classic semantics; introduce ``//`` for true division.
|
||||
This still leaves a broken operator in the language, and invites to use the
|
||||
broken behavior. It also shuts off the road to a unified numeric model a la
|
||||
PEP 228 [0]_.
|
||||
|
||||
- Let int division return a special "portmanteau" type that
|
||||
behaves as an integer in integer context, but like a float in a
|
||||
float context. The problem with this is that after a few
|
||||
operations, the int and the float value could be miles apart,
|
||||
it's unclear which value should be used in comparisons, and of
|
||||
course many contexts (like conversion to string) don't have a
|
||||
clear integer or float preference.
|
||||
- Let int division return a special "portmanteau" type that behaves as an
|
||||
integer in integer context, but like a float in a float context. The
|
||||
problem with this is that after a few operations, the int and the float
|
||||
value could be miles apart, it's unclear which value should be used in
|
||||
comparisons, and of course many contexts (like conversion to string) don't
|
||||
have a clear integer or float preference.
|
||||
|
||||
- Use a directive to use specific division semantics in a module,
|
||||
rather than a future statement. This retains classic division
|
||||
as a permanent wart in the language, requiring future
|
||||
generations of Python programmers to be aware of the problem and
|
||||
the remedies.
|
||||
- Use a directive to use specific division semantics in a module, rather than
|
||||
a future statement. This retains classic division as a permanent wart in
|
||||
the language, requiring future generations of Python programmers to be
|
||||
aware of the problem and the remedies.
|
||||
|
||||
- Use "from __past__ import division" to use classic division
|
||||
semantics in a module. This also retains the classic division
|
||||
as a permanent wart, or at least for a long time (eventually the
|
||||
past division statement could raise an ImportError).
|
||||
- Use ``from __past__ import division`` to use classic division semantics in a
|
||||
module. This also retains the classic division as a permanent wart, or at
|
||||
least for a long time (eventually the past division statement could raise an
|
||||
``ImportError``).
|
||||
|
||||
- Use a directive (or some other way) to specify the Python
|
||||
version for which a specific piece of code was developed. This
|
||||
requires future Python interpreters to be able to emulate
|
||||
*exactly* several previous versions of Python, and moreover to
|
||||
do so for multiple versions within the same interpreter. This
|
||||
is way too much work. A much simpler solution is to keep
|
||||
multiple interpreters installed. Another argument against this
|
||||
is that the version directive is almost always overspecified:
|
||||
most code written for Python X.Y, works for Python X.(Y-1) and
|
||||
X.(Y+1) as well, so specifying X.Y as a version is more
|
||||
constraining than it needs to be. At the same time, there's no
|
||||
way to know at which future or past version the code will break.
|
||||
- Use a directive (or some other way) to specify the Python version for which
|
||||
a specific piece of code was developed. This requires future Python
|
||||
interpreters to be able to emulate *exactly* several previous versions of
|
||||
Python, and moreover to do so for multiple versions within the same
|
||||
interpreter. This is way too much work. A much simpler solution is to keep
|
||||
multiple interpreters installed. Another argument against this is that the
|
||||
version directive is almost always overspecified: most code written for
|
||||
Python X.Y, works for Python X.(Y-1) and X.(Y+1) as well, so specifying X.Y
|
||||
as a version is more constraining than it needs to be. At the same time,
|
||||
there's no way to know at which future or past version the code will break.
|
||||
|
||||
|
||||
API Changes
|
||||
===========
|
||||
|
||||
During the transitional phase, we have to support *three* division
|
||||
operators within the same program: classic division (for / in
|
||||
modules without a future division statement), true division (for /
|
||||
in modules with a future division statement), and floor division
|
||||
(for //). Each operator comes in two flavors: regular, and as an
|
||||
augmented assignment operator (/= or //=).
|
||||
During the transitional phase, we have to support *three* division operators
|
||||
within the same program: classic division (for ``/`` in modules without a
|
||||
future division statement), true division (for ``/`` in modules with a future
|
||||
division statement), and floor division (for ``//``). Each operator comes in
|
||||
two flavors: regular, and as an augmented assignment operator (``/=`` or
|
||||
``//=``).
|
||||
|
||||
The names associated with these variations are:
|
||||
The names associated with these variations are:
|
||||
|
||||
- Overloaded operator methods:
|
||||
- Overloaded operator methods::
|
||||
|
||||
__div__(), __floordiv__(), __truediv__();
|
||||
|
||||
__idiv__(), __ifloordiv__(), __itruediv__().
|
||||
|
||||
- Abstract API C functions:
|
||||
- Abstract API C functions::
|
||||
|
||||
PyNumber_Divide(), PyNumber_FloorDivide(),
|
||||
PyNumber_TrueDivide();
|
||||
|
@ -225,304 +212,301 @@ API Changes
|
|||
PyNumber_InPlaceDivide(), PyNumber_InPlaceFloorDivide(),
|
||||
PyNumber_InPlaceTrueDivide().
|
||||
|
||||
- Byte code opcodes:
|
||||
- Byte code opcodes::
|
||||
|
||||
BINARY_DIVIDE, BINARY_FLOOR_DIVIDE, BINARY_TRUE_DIVIDE;
|
||||
|
||||
INPLACE_DIVIDE, INPLACE_FLOOR_DIVIDE, INPLACE_TRUE_DIVIDE.
|
||||
|
||||
- PyNumberMethod slots:
|
||||
- PyNumberMethod slots::
|
||||
|
||||
nb_divide, nb_floor_divide, nb_true_divide,
|
||||
|
||||
nb_inplace_divide, nb_inplace_floor_divide,
|
||||
nb_inplace_true_divide.
|
||||
|
||||
The added PyNumberMethod slots require an additional flag in
|
||||
tp_flags; this flag will be named Py_TPFLAGS_HAVE_NEWDIVIDE and
|
||||
will be included in Py_TPFLAGS_DEFAULT.
|
||||
The added ``PyNumberMethod`` slots require an additional flag in ``tp_flags``;
|
||||
this flag will be named ``Py_TPFLAGS_HAVE_NEWDIVIDE`` and will be included in
|
||||
``Py_TPFLAGS_DEFAULT``.
|
||||
|
||||
The true and floor division APIs will look for the corresponding
|
||||
slots and call that; when that slot is NULL, they will raise an
|
||||
exception. There is no fallback to the classic divide slot.
|
||||
The true and floor division APIs will look for the corresponding slots and
|
||||
call that; when that slot is ``NULL``, they will raise an exception. There is
|
||||
no fallback to the classic divide slot.
|
||||
|
||||
In Python 3.0, the classic division semantics will be removed; the
|
||||
classic division APIs will become synonymous with true division.
|
||||
In Python 3.0, the classic division semantics will be removed; the classic
|
||||
division APIs will become synonymous with true division.
|
||||
|
||||
|
||||
Command Line Option
|
||||
===================
|
||||
|
||||
The -Q command line option takes a string argument that can take
|
||||
four values: "old", "warn", "warnall", or "new". The default is
|
||||
"old" in Python 2.2 but will change to "warn" in later 2.x
|
||||
versions. The "old" value means the classic division operator
|
||||
acts as described. The "warn" value means the classic division
|
||||
operator issues a warning (a DeprecationWarning using the standard
|
||||
warning framework) when applied to ints or longs. The "warnall"
|
||||
value also issues warnings for classic division when applied to
|
||||
floats or complex; this is for use by the fixdiv.py conversion
|
||||
script mentioned below. The "new" value changes the default
|
||||
globally so that the / operator is always interpreted as true
|
||||
division. The "new" option is only intended for use in certain
|
||||
educational environments, where true division is required, but
|
||||
asking the students to include the future division statement in
|
||||
all their code would be a problem.
|
||||
The ``-Q`` command line option takes a string argument that can take four
|
||||
values: ``old``, ``warn``, ``warnall``, or ``new``. The default is ``old``
|
||||
in Python 2.2 but will change to ``warn`` in later 2.x versions. The ``old``
|
||||
value means the classic division operator acts as described. The ``warn``
|
||||
value means the classic division operator issues a warning (a
|
||||
``DeprecationWarning`` using the standard warning framework) when applied
|
||||
to ints or longs. The ``warnall`` value also issues warnings for classic
|
||||
division when applied to floats or complex; this is for use by the
|
||||
``fixdiv.py`` conversion script mentioned below. The ``new`` value changes
|
||||
the default globally so that the ``/`` operator is always interpreted as
|
||||
true division. The ``new`` option is only intended for use in certain
|
||||
educational environments, where true division is required, but asking the
|
||||
students to include the future division statement in all their code would be a
|
||||
problem.
|
||||
|
||||
This option will not be supported in Python 3.0; Python 3.0 will
|
||||
always interpret / as true division.
|
||||
This option will not be supported in Python 3.0; Python 3.0 will always
|
||||
interpret ``/`` as true division.
|
||||
|
||||
(This option was originally proposed as -D, but that turned out to
|
||||
be an existing option for Jython, hence the Q -- mnemonic for
|
||||
Quotient. Other names have been proposed, like -Qclassic,
|
||||
-Qclassic-warn, -Qtrue, or -Qold_division etc.; these seem more
|
||||
verbose to me without much advantage. After all the term classic
|
||||
division is not used in the language at all (only in the PEP), and
|
||||
the term true division is rarely used in the language -- only in
|
||||
__truediv__.)
|
||||
(This option was originally proposed as ``-D``, but that turned out to be an
|
||||
existing option for Jython, hence the Q -- mnemonic for Quotient. Other names
|
||||
have been proposed, like ``-Qclassic``, ``-Qclassic-warn``, ``-Qtrue``, or
|
||||
``-Qold_division`` etc.; these seem more verbose to me without much advantage.
|
||||
After all the term classic division is not used in the language at all (only
|
||||
in the PEP), and the term true division is rarely used in the language -- only
|
||||
in ``__truediv__``.)
|
||||
|
||||
|
||||
Semantics of Floor Division
|
||||
===========================
|
||||
|
||||
Floor division will be implemented in all the Python numeric
|
||||
types, and will have the semantics of
|
||||
Floor division will be implemented in all the Python numeric types, and will
|
||||
have the semantics of::
|
||||
|
||||
a // b == floor(a/b)
|
||||
|
||||
except that the result type will be the common type into which a
|
||||
and b are coerced before the operation.
|
||||
except that the result type will be the common type into which *a* and *b* are
|
||||
coerced before the operation.
|
||||
|
||||
Specifically, if a and b are of the same type, a//b will be of
|
||||
that type too. If the inputs are of different types, they are
|
||||
first coerced to a common type using the same rules used for all
|
||||
other arithmetic operators.
|
||||
Specifically, if *a* and *b* are of the same type, ``a//b`` will be of that
|
||||
type too. If the inputs are of different types, they are first coerced to a
|
||||
common type using the same rules used for all other arithmetic operators.
|
||||
|
||||
In particular, if a and b are both ints or longs, the result has
|
||||
the same type and value as for classic division on these types
|
||||
(including the case of mixed input types; int//long and long//int
|
||||
will both return a long).
|
||||
In particular, if *a* and *b* are both ints or longs, the result has the same
|
||||
type and value as for classic division on these types (including the case of
|
||||
mixed input types; ``int//long`` and ``long//int`` will both return a long).
|
||||
|
||||
For floating point inputs, the result is a float. For example:
|
||||
For floating point inputs, the result is a float. For example::
|
||||
|
||||
3.5//2.0 == 1.0
|
||||
|
||||
For complex numbers, // raises an exception, since floor() of a
|
||||
complex number is not allowed.
|
||||
For complex numbers, ``//`` raises an exception, since ``floor()`` of a
|
||||
complex number is not allowed.
|
||||
|
||||
For user-defined classes and extension types, all semantics are up
|
||||
to the implementation of the class or type.
|
||||
For user-defined classes and extension types, all semantics are up to the
|
||||
implementation of the class or type.
|
||||
|
||||
|
||||
Semantics of True Division
|
||||
==========================
|
||||
|
||||
True division for ints and longs will convert the arguments to
|
||||
float and then apply a float division. That is, even 2/1 will
|
||||
return a float (2.0), not an int. For floats and complex, it will
|
||||
be the same as classic division.
|
||||
True division for ints and longs will convert the arguments to float and then
|
||||
apply a float division. That is, even ``2/1`` will return a ``float (2.0)``,
|
||||
not an int. For floats and complex, it will be the same as classic division.
|
||||
|
||||
The 2.2 implementation of true division acts as if the float type
|
||||
had unbounded range, so that overflow doesn't occur unless the
|
||||
magnitude of the mathematical *result* is too large to represent
|
||||
as a float. For example, after "x = 1L << 40000", float(x) raises
|
||||
OverflowError (note that this is also new in 2.2: previously the
|
||||
outcome was platform-dependent, most commonly a float infinity). But
|
||||
x/x returns 1.0 without exception, while x/1 raises OverflowError.
|
||||
The 2.2 implementation of true division acts as if the float type had
|
||||
unbounded range, so that overflow doesn't occur unless the magnitude of the
|
||||
mathematical *result* is too large to represent as a float. For example,
|
||||
after ``x = 1L << 40000``, ``float(x)`` raises ``OverflowError`` (note that
|
||||
this is also new in 2.2: previously the outcome was platform-dependent, most
|
||||
commonly a float infinity). But ``x/x`` returns 1.0 without exception,
|
||||
while ``x/1`` raises ``OverflowError``.
|
||||
|
||||
Note that for int and long arguments, true division may lose
|
||||
information; this is in the nature of true division (as long as
|
||||
rationals are not in the language). Algorithms that consciously
|
||||
use longs should consider using //, as true division of longs
|
||||
retains no more than 53 bits of precision (on most platforms).
|
||||
Note that for int and long arguments, true division may lose information; this
|
||||
is in the nature of true division (as long as rationals are not in the
|
||||
language). Algorithms that consciously use longs should consider using
|
||||
``//``, as true division of longs retains no more than 53 bits of precision
|
||||
(on most platforms).
|
||||
|
||||
If and when a rational type is added to Python (see PEP 239[2]),
|
||||
true division for ints and longs should probably return a
|
||||
rational. This avoids the problem with true division of ints and
|
||||
longs losing information. But until then, for consistency, float is
|
||||
the only choice for true division.
|
||||
If and when a rational type is added to Python (see PEP 239 [2]_), true
|
||||
division for ints and longs should probably return a rational. This avoids
|
||||
the problem with true division of ints and longs losing information. But
|
||||
until then, for consistency, float is the only choice for true division.
|
||||
|
||||
|
||||
The Future Division Statement
|
||||
=============================
|
||||
|
||||
If "from __future__ import division" is present in a module, or if
|
||||
-Qnew is used, the / and /= operators are translated to true
|
||||
division opcodes; otherwise they are translated to classic
|
||||
division (until Python 3.0 comes along, where they are always
|
||||
translated to true division).
|
||||
If ``from __future__ import division`` is present in a module, or if
|
||||
``-Qnew`` is used, the ``/`` and ``/=`` operators are translated to true
|
||||
division opcodes; otherwise they are translated to classic division (until
|
||||
Python 3.0 comes along, where they are always translated to true division).
|
||||
|
||||
The future division statement has no effect on the recognition or
|
||||
translation of // and //=.
|
||||
The future division statement has no effect on the recognition or translation
|
||||
of ``//`` and ``//=``.
|
||||
|
||||
See PEP 236[4] for the general rules for future statements.
|
||||
See PEP 236 [4]_ for the general rules for future statements.
|
||||
|
||||
(It has been proposed to use a longer phrase, like "true_division"
|
||||
or "modern_division". These don't seem to add much information.)
|
||||
(It has been proposed to use a longer phrase, like *true_division* or
|
||||
*modern_division*. These don't seem to add much information.)
|
||||
|
||||
|
||||
Open Issues
|
||||
===========
|
||||
|
||||
We expect that these issues will be resolved over time, as more
|
||||
feedback is received or we gather more experience with the initial
|
||||
implementation.
|
||||
We expect that these issues will be resolved over time, as more feedback is
|
||||
received or we gather more experience with the initial implementation.
|
||||
|
||||
- It has been proposed to call // the quotient operator, and the /
|
||||
operator the ratio operator. I'm not sure about this -- for
|
||||
some people quotient is just a synonym for division, and ratio
|
||||
suggests rational numbers, which is wrong. I prefer the
|
||||
terminology to be slightly awkward if that avoids unambiguity.
|
||||
Also, for some folks "quotient" suggests truncation towards
|
||||
zero, not towards infinity as "floor division" says explicitly.
|
||||
- It has been proposed to call ``//`` the quotient operator, and the ``/``
|
||||
operator the ratio operator. I'm not sure about this -- for some people
|
||||
quotient is just a synonym for division, and ratio suggests rational
|
||||
numbers, which is wrong. I prefer the terminology to be slightly awkward
|
||||
if that avoids unambiguity. Also, for some folks *quotient* suggests
|
||||
truncation towards zero, not towards infinity as *floor division*
|
||||
says explicitly.
|
||||
|
||||
- It has been argued that a command line option to change the
|
||||
default is evil. It can certainly be dangerous in the wrong
|
||||
hands: for example, it would be impossible to combine a 3rd
|
||||
party library package that requires -Qnew with another one that
|
||||
requires -Qold. But I believe that the VPython folks need a way
|
||||
to enable true division by default, and other educators might
|
||||
need the same. These usually have enough control over the
|
||||
- It has been argued that a command line option to change the default is
|
||||
evil. It can certainly be dangerous in the wrong hands: for example, it
|
||||
would be impossible to combine a 3rd party library package that requires
|
||||
``-Qnew`` with another one that requires ``-Qold``. But I believe that the
|
||||
VPython folks need a way to enable true division by default, and other
|
||||
educators might need the same. These usually have enough control over the
|
||||
library packages available in their environment.
|
||||
|
||||
- For classes to have to support all three of __div__(),
|
||||
__floordiv__() and __truediv__() seems painful; and what to do
|
||||
in 3.0? Maybe we only need __div__() and __floordiv__(), or
|
||||
maybe at least true division should try __truediv__() first and
|
||||
__div__() second.
|
||||
- For classes to have to support all three of ``__div__()``,
|
||||
``__floordiv__()`` and ``__truediv__()`` seems painful; and what to do in
|
||||
3.0? Maybe we only need ``__div__()`` and ``__floordiv__()``, or maybe at
|
||||
least true division should try ``__truediv__()`` first and ``__div__()``
|
||||
second.
|
||||
|
||||
|
||||
Resolved Issues
|
||||
===============
|
||||
|
||||
- Issue: For very large long integers, the definition of true
|
||||
division as returning a float causes problems, since the range of
|
||||
Python longs is much larger than that of Python floats. This
|
||||
problem will disappear if and when rational numbers are supported.
|
||||
- Issue: For very large long integers, the definition of true division as
|
||||
returning a float causes problems, since the range of Python longs is much
|
||||
larger than that of Python floats. This problem will disappear if and when
|
||||
rational numbers are supported.
|
||||
|
||||
Resolution: For long true division, Python uses an internal
|
||||
float type with native double precision but unbounded range, so
|
||||
that OverflowError doesn't occur unless the quotient is too large
|
||||
to represent as a native double.
|
||||
Resolution: For long true division, Python uses an internal float type with
|
||||
native double precision but unbounded range, so that OverflowError doesn't
|
||||
occur unless the quotient is too large to represent as a native double.
|
||||
|
||||
- Issue: In the interim, maybe the long-to-float conversion could be
|
||||
made to raise OverflowError if the long is out of range.
|
||||
- Issue: In the interim, maybe the long-to-float conversion could be made to
|
||||
raise ``OverflowError`` if the long is out of range.
|
||||
|
||||
Resolution: This has been implemented, but, as above, the
|
||||
magnitude of the inputs to long true division doesn't matter; only
|
||||
the magnitude of the quotient matters.
|
||||
Resolution: This has been implemented, but, as above, the magnitude of the
|
||||
inputs to long true division doesn't matter; only the magnitude of the
|
||||
quotient matters.
|
||||
|
||||
- Issue: Tim Peters will make sure that whenever an in-range float
|
||||
is returned, decent precision is guaranteed.
|
||||
- Issue: Tim Peters will make sure that whenever an in-range float is
|
||||
returned, decent precision is guaranteed.
|
||||
|
||||
Resolution: Provided the quotient of long true division is
|
||||
representable as a float, it suffers no more than 3 rounding
|
||||
errors: one each for converting the inputs to an internal float
|
||||
type with native double precision but unbounded range, and
|
||||
one more for the division. However, note that if the magnitude
|
||||
of the quotient is too *small* to represent as a native double,
|
||||
Resolution: Provided the quotient of long true division is representable as
|
||||
a float, it suffers no more than 3 rounding errors: one each for converting
|
||||
the inputs to an internal float type with native double precision but
|
||||
unbounded range, and one more for the division. However, note that if the
|
||||
magnitude of the quotient is too *small* to represent as a native double,
|
||||
0.0 is returned without exception ("silent underflow").
|
||||
|
||||
|
||||
FAQ
|
||||
===
|
||||
|
||||
Q. When will Python 3.0 be released?
|
||||
When will Python 3.0 be released?
|
||||
---------------------------------
|
||||
|
||||
A. We don't plan that long ahead, so we can't say for sure. We
|
||||
want to allow at least two years for the transition. If Python
|
||||
3.0 comes out sooner, we'll keep the 2.x line alive for
|
||||
backwards compatibility until at least two years from the
|
||||
release of Python 2.2. In practice, you will be able to
|
||||
continue to use the Python 2.x line for several years after
|
||||
Python 3.0 is released, so you can take your time with the
|
||||
transition. Sites are expected to have both Python 2.x and
|
||||
Python 3.x installed simultaneously.
|
||||
We don't plan that long ahead, so we can't say for sure. We want to allow
|
||||
at least two years for the transition. If Python 3.0 comes out sooner,
|
||||
we'll keep the 2.x line alive for backwards compatibility until at least
|
||||
two years from the release of Python 2.2. In practice, you will be able
|
||||
to continue to use the Python 2.x line for several years after Python 3.0
|
||||
is released, so you can take your time with the transition. Sites are
|
||||
expected to have both Python 2.x and Python 3.x installed simultaneously.
|
||||
|
||||
Q. Why isn't true division called float division?
|
||||
Why isn't true division called float division?
|
||||
----------------------------------------------
|
||||
|
||||
A. Because I want to keep the door open to *possibly* introducing
|
||||
rationals and making 1/2 return a rational rather than a
|
||||
float. See PEP 239[2].
|
||||
Because I want to keep the door open to *possibly* introducing rationals
|
||||
and making 1/2 return a rational rather than a float. See PEP 239 [2]_.
|
||||
|
||||
Q. Why is there a need for __truediv__ and __itruediv__?
|
||||
Why is there a need for ``__truediv__`` and ``__itruediv__``?
|
||||
-------------------------------------------------------------
|
||||
|
||||
A. We don't want to make user-defined classes second-class
|
||||
citizens. Certainly not with the type/class unification going
|
||||
on.
|
||||
We don't want to make user-defined classes second-class citizens.
|
||||
Certainly not with the type/class unification going on.
|
||||
|
||||
Q. How do I write code that works under the classic rules as well
|
||||
as under the new rules without using // or a future division
|
||||
statement?
|
||||
How do I write code that works under the classic rules as well as under the new rules without using ``//`` or a future division statement?
|
||||
------------------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
A. Use x*1.0/y for true division, divmod(x, y)[0] for int
|
||||
division. Especially the latter is best hidden inside a
|
||||
function. You may also write float(x)/y for true division if
|
||||
you are sure that you don't expect complex numbers. If you
|
||||
know your integers are never negative, you can use int(x/y) --
|
||||
while the documentation of int() says that int() can round or
|
||||
truncate depending on the C implementation, we know of no C
|
||||
implementation that doesn't truncate, and we're going to change
|
||||
the spec for int() to promise truncation. Note that classic
|
||||
division (and floor division) round towards negative infinity,
|
||||
while int() rounds towards zero, giving different answers for
|
||||
negative numbers.
|
||||
Use ``x*1.0/y`` for true division, ``divmod(x, y)`` [0]_ for int
|
||||
division. Especially the latter is best hidden inside a function. You
|
||||
may also write ``float(x)/y`` for true division if you are sure that you
|
||||
don't expect complex numbers. If you know your integers are never
|
||||
negative, you can use ``int(x/y)`` -- while the documentation of ``int()``
|
||||
says that ``int()`` can round or truncate depending on the C
|
||||
implementation, we know of no C implementation that doesn't truncate, and
|
||||
we're going to change the spec for ``int()`` to promise truncation. Note
|
||||
that classic division (and floor division) round towards negative
|
||||
infinity, while ``int()`` rounds towards zero, giving different answers
|
||||
for negative numbers.
|
||||
|
||||
Q. How do I specify the division semantics for input(), compile(),
|
||||
execfile(), eval() and exec?
|
||||
How do I specify the division semantics for ``input()``, ``compile()``, ``execfile()``, ``eval()`` and ``exec``?
|
||||
----------------------------------------------------------------------------------------------------------------
|
||||
|
||||
A. They inherit the choice from the invoking module. PEP 236[4]
|
||||
now lists this as a resolved problem, referring to PEP 264[5].
|
||||
They inherit the choice from the invoking module. PEP 236 [4]_ now lists
|
||||
this as a resolved problem, referring to PEP 264 [5]_.
|
||||
|
||||
Q. What about code compiled by the codeop module?
|
||||
What about code compiled by the codeop module?
|
||||
----------------------------------------------
|
||||
|
||||
A. This is dealt with properly; see PEP 264[5].
|
||||
This is dealt with properly; see PEP 264 [5]_.
|
||||
|
||||
Q. Will there be conversion tools or aids?
|
||||
Will there be conversion tools or aids?
|
||||
---------------------------------------
|
||||
|
||||
A. Certainly. While these are outside the scope of the PEP, I
|
||||
should point out two simple tools that will be released with
|
||||
Python 2.2a3: Tools/scripts/finddiv.py finds division operators
|
||||
(slightly smarter than "grep /") and Tools/scripts/fixdiv.py
|
||||
can produce patches based on run-time analysis.
|
||||
Certainly. While these are outside the scope of the PEP, I should point
|
||||
out two simple tools that will be released with Python 2.2a3:
|
||||
``Tools/scripts/finddiv.py`` finds division operators (slightly smarter
|
||||
than ``grep /``) and ``Tools/scripts/fixdiv.py`` can produce patches based
|
||||
on run-time analysis.
|
||||
|
||||
Q. Why is my question not answered here?
|
||||
Why is my question not answered here?
|
||||
-------------------------------------
|
||||
|
||||
A. Because we weren't aware of it. If it's been discussed on
|
||||
c.l.py and you believe the answer is of general interest,
|
||||
please notify the second author. (We don't have the time or
|
||||
inclination to answer every question sent in private email,
|
||||
hence the requirement that it be discussed on c.l.py first.)
|
||||
Because we weren't aware of it. If it's been discussed on c.l.py and you
|
||||
believe the answer is of general interest, please notify the second
|
||||
author. (We don't have the time or inclination to answer every question
|
||||
sent in private email, hence the requirement that it be discussed on
|
||||
c.l.py first.)
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Essentially everything mentioned here is implemented in CVS and
|
||||
will be released with Python 2.2a3; most of it was already
|
||||
released with Python 2.2a2.
|
||||
Essentially everything mentioned here is implemented in CVS and will be
|
||||
released with Python 2.2a3; most of it was already released with Python 2.2a2.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
[0] PEP 228, Reworking Python's Numeric Model
|
||||
.. [0] PEP 228, Reworking Python's Numeric Model
|
||||
http://www.python.org/dev/peps/pep-0228/
|
||||
|
||||
[1] PEP 237, Unifying Long Integers and Integers, Zadka,
|
||||
.. [1] PEP 237, Unifying Long Integers and Integers, Zadka,
|
||||
http://www.python.org/dev/peps/pep-0237/
|
||||
|
||||
[2] PEP 239, Adding a Rational Type to Python, Zadka,
|
||||
.. [2] PEP 239, Adding a Rational Type to Python, Zadka,
|
||||
http://www.python.org/dev/peps/pep-0239/
|
||||
|
||||
[3] PEP 240, Adding a Rational Literal to Python, Zadka,
|
||||
.. [3] PEP 240, Adding a Rational Literal to Python, Zadka,
|
||||
http://www.python.org/dev/peps/pep-0240/
|
||||
|
||||
[4] PEP 236, Back to the __future__, Peters,
|
||||
.. [4] PEP 236, Back to the __future__, Peters,
|
||||
http://www.python.org/dev/peps/pep-0236/
|
||||
|
||||
[5] PEP 264, Future statements in simulated shells
|
||||
.. [5] PEP 264, Future statements in simulated shells
|
||||
http://www.python.org/dev/peps/pep-0236/
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
This document has been placed in the public domain.
|
||||
|
||||
|
||||
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
End:
|
||||
|
||||
..
|
||||
Local Variables:
|
||||
mode: indented-text
|
||||
indent-tabs-mode: nil
|
||||
End:
|
||||
|
|
Loading…
Reference in New Issue