247 lines
9.6 KiB
Plaintext
247 lines
9.6 KiB
Plaintext
PEP: 208
|
|
Title: Reworking the Coercion Model
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: nas@arctrix.com (Neil Schemenauer), mal@lemburg.com (Marc-André Lemburg)
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 04-Dec-2000
|
|
Python-Version: 2.1
|
|
Post-History:
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
Many Python types implement numeric operations. When the arguments of
|
|
a numeric operation are of different types, the interpreter tries to
|
|
coerce the arguments into a common type. The numeric operation is
|
|
then performed using this common type. This PEP proposes a new type
|
|
flag to indicate that arguments to a type's numeric operations should
|
|
not be coerced. Operations that do not support the supplied types
|
|
indicate it by returning a new singleton object. Types which do not
|
|
set the type flag are handled in a backwards compatible manner.
|
|
Allowing operations handle different types is often simpler, more
|
|
flexible, and faster than having the interpreter do coercion.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
When implementing numeric or other related operations, it is often
|
|
desirable to provide not only operations between operands of one type
|
|
only, e.g. integer + integer, but to generalize the idea behind the
|
|
operation to other type combinations as well, e.g. integer + float.
|
|
|
|
A common approach to this mixed type situation is to provide a method
|
|
of "lifting" the operands to a common type (coercion) and then use
|
|
that type's operand method as execution mechanism. Yet, this strategy
|
|
has a few drawbacks:
|
|
|
|
* the "lifting" process creates at least one new (temporary)
|
|
operand object,
|
|
|
|
* since the coercion method is not being told about the operation
|
|
that is to follow, it is not possible to implement operation
|
|
specific coercion of types,
|
|
|
|
* there is no elegant way to solve situations were a common type
|
|
is not at hand, and
|
|
|
|
* the coercion method will always have to be called prior to the
|
|
operation's method itself.
|
|
|
|
A fix for this situation is obviously needed, since these drawbacks
|
|
make implementations of types needing these features very cumbersome,
|
|
if not impossible. As an example, have a look at the ``DateTime`` and
|
|
``DateTimeDelta`` [1]_ types, the first being absolute, the second
|
|
relative. You can always add a relative value to an absolute one,
|
|
giving a new absolute value. Yet, there is no common type which the
|
|
existing coercion mechanism could use to implement that operation.
|
|
|
|
Currently, ``PyInstance`` types are treated specially by the interpreter
|
|
in that their numeric methods are passed arguments of different types.
|
|
Removing this special case simplifies the interpreter and allows other
|
|
types to implement numeric methods that behave like instance types.
|
|
This is especially useful for extension types like ExtensionClass.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
Instead of using a central coercion method, the process of handling
|
|
different operand types is simply left to the operation. If the
|
|
operation finds that it cannot handle the given operand type
|
|
combination, it may return a special singleton as indicator.
|
|
|
|
Note that "numbers" (anything that implements the number protocol, or
|
|
part of it) written in Python already use the first part of this
|
|
strategy - it is the C level API that we focus on here.
|
|
|
|
To maintain nearly 100% backward compatibility we have to be very
|
|
careful to make numbers that don't know anything about the new
|
|
strategy (old style numbers) work just as well as those that expect
|
|
the new scheme (new style numbers). Furthermore, binary compatibility
|
|
is a must, meaning that the interpreter may only access and use new
|
|
style operations if the number indicates the availability of these.
|
|
|
|
A new style number is considered by the interpreter as such if and
|
|
only if it sets the type flag ``Py_TPFLAGS_CHECKTYPES``. The main
|
|
difference between an old style number and a new style one is that the
|
|
numeric slot functions can no longer assume to be passed arguments of
|
|
identical type. New style slots must check all arguments for proper
|
|
type and implement the necessary conversions themselves. This may seem
|
|
to cause more work on the behalf of the type implementor, but is in
|
|
fact no more difficult than writing the same kind of routines for an
|
|
old style coercion slot.
|
|
|
|
If a new style slot finds that it cannot handle the passed argument
|
|
type combination, it may return a new reference of the special
|
|
singleton ``Py_NotImplemented`` to the caller. This will cause the caller
|
|
to try the other operands operation slots until it finds a slot that
|
|
does implement the operation for the specific type combination. If
|
|
none of the possible slots succeed, it raises a ``TypeError``.
|
|
|
|
To make the implementation easy to understand (the whole topic is
|
|
esoteric enough), a new layer in the handling of numeric operations is
|
|
introduced. This layer takes care of all the different cases that need
|
|
to be taken into account when dealing with all the possible
|
|
combinations of old and new style numbers. It is implemented by the
|
|
two static functions ``binary_op()`` and ``ternary_op()``, which are both
|
|
internal functions that only the functions in Objects/abstract.c
|
|
have access to. The numeric API (``PyNumber_*``) is easy to adapt to
|
|
this new layer.
|
|
|
|
As a side-effect all numeric slots can be NULL-checked (this has to be
|
|
done anyway, so the added feature comes at no extra cost).
|
|
|
|
|
|
The scheme used by the layer to execute a binary operation is as
|
|
follows:
|
|
|
|
=== === =================================
|
|
v w Action taken
|
|
=== === =================================
|
|
new new v.op(v,w), w.op(v,w)
|
|
new old v.op(v,w), coerce(v,w), v.op(v,w)
|
|
old new w.op(v,w), coerce(v,w), v.op(v,w)
|
|
old old coerce(v,w), v.op(v,w)
|
|
=== === =================================
|
|
|
|
The indicated action sequence is executed from left to right until
|
|
either the operation succeeds and a valid result (!=
|
|
``Py_NotImplemented``) is returned or an exception is raised. Exceptions
|
|
are returned to the calling function as-is. If a slot returns
|
|
``Py_NotImplemented``, the next item in the sequence is executed.
|
|
|
|
Note that coerce(v,w) will use the old style ``nb_coerce`` slot methods
|
|
via a call to ``PyNumber_Coerce()``.
|
|
|
|
Ternary operations have a few more cases to handle:
|
|
|
|
=== === === ====================================================
|
|
v w z Action taken
|
|
=== === === ====================================================
|
|
new new new v.op(v,w,z), w.op(v,w,z), z.op(v,w,z)
|
|
new old new v.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
old new new w.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
old old new z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
new new old v.op(v,w,z), w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
new old old v.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
old new old w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
|
|
old old old coerce(v,w,z), v.op(v,w,z)
|
|
=== === === ====================================================
|
|
|
|
The same notes as above, except that coerce(v,w,z) actually does::
|
|
|
|
if z != Py_None:
|
|
coerce(v,w), coerce(v,z), coerce(w,z)
|
|
else:
|
|
# treat z as absent variable
|
|
coerce(v,w)
|
|
|
|
|
|
The current implementation uses this scheme already (there's only one
|
|
ternary slot: ``nb_pow(a,b,c))``.
|
|
|
|
Note that the numeric protocol is also used for some other related
|
|
tasks, e.g. sequence concatenation. These can also benefit from the
|
|
new mechanism by implementing right-hand operations for type
|
|
combinations that would otherwise fail to work. As an example, take
|
|
string concatenation: currently you can only do string + string. With
|
|
the new mechanism, a new string-like type could implement new_type +
|
|
string and string + new_type, even though strings don't know anything
|
|
about new_type.
|
|
|
|
Since comparisons also rely on coercion (every time you compare an
|
|
integer to a float, the integer is first converted to float and then
|
|
compared...), a new slot to handle numeric comparisons is needed::
|
|
|
|
PyObject *nb_cmp(PyObject *v, PyObject *w)
|
|
|
|
This slot should compare the two objects and return an integer object
|
|
stating the result. Currently, this result integer may only be -1, 0, 1.
|
|
If the slot cannot handle the type combination, it may return a
|
|
reference to ``Py_NotImplemented``. [XXX Note that this slot is still
|
|
in flux since it should take into account rich comparisons
|
|
(i.e. PEP 207).]
|
|
|
|
Numeric comparisons are handled by a new numeric protocol API::
|
|
|
|
PyObject *PyNumber_Compare(PyObject *v, PyObject *w)
|
|
|
|
This function compare the two objects as "numbers" and return an
|
|
integer object stating the result. Currently, this result integer may
|
|
only be -1, 0, 1. In case the operation cannot be handled by the given
|
|
objects, a ``TypeError`` is raised.
|
|
|
|
The ``PyObject_Compare()`` API needs to adjusted accordingly to make use
|
|
of this new API.
|
|
|
|
Other changes include adapting some of the built-in functions (e.g.
|
|
``cmp()``) to use this API as well. Also, ``PyNumber_CoerceEx()`` will need to
|
|
check for new style numbers before calling the ``nb_coerce`` slot. New
|
|
style numbers don't provide a coercion slot and thus cannot be
|
|
explicitly coerced.
|
|
|
|
|
|
Reference Implementation
|
|
========================
|
|
|
|
A preliminary patch for the CVS version of Python is available through
|
|
the Source Forge patch manager [2]_.
|
|
|
|
|
|
Credits
|
|
=======
|
|
|
|
This PEP and the patch are heavily based on work done by Marc-André
|
|
Lemburg [3]_.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] http://www.lemburg.com/files/python/mxDateTime.html
|
|
.. [2] http://sourceforge.net/patch/?func=detailpatch&patch_id=102652&group_id=5470
|
|
.. [3] http://www.lemburg.com/files/python/CoercionProposal.html
|
|
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|
|
End:
|