python-peps/pep-0211.txt

329 lines
12 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

PEP: 211
Title: Adding New Linear Algebra Operators to Python
Version: $Revision$
Author: gvwilson@nevex.com (Greg Wilson)
Status: Draft
Type: Standards Track
Python-Version: 2.1
Created: 15-Jul-2000
Post-History:
Introduction
This PEP describes a conservative proposal to add linear algebra
operators to Python 2.0. It discusses why such operators are
desirable, and why a minimalist approach should be adopted at this
point. This PEP summarizes discussions held in mailing list
forums, and provides URLs for further information, where
appropriate. The CVS revision history of this file contains the
definitive historical record.
Summary
Add a single new infix binary operator '@' ("across"), and
corresponding special methods "__across__()", "__racross__()", and
"__iacross__()". This operator will perform mathematical matrix
multiplication on NumPy arrays, and generate cross-products when
applied to built-in sequence types. No existing operator
definitions will be changed.
Background
The first high-level programming language, Fortran, was invented
to do arithmetic. While this is now just a small part of
computing, there are still many programmers who need to express
complex mathematical operations in code.
The most influential of Fortran's successors was APL [1]. Its
author, Kenneth Iverson, designed the language as a notation for
expressing matrix algebra, and received the 1980 Turing Award for
his work.
APL's operators supported both familiar algebraic operations, such
as vector dot product and matrix multiplication, and a wide range
of structural operations, such as stitching vectors together to
create arrays. Even by programming's standards, APL is
exceptionally cryptic: many of its symbols did not exist on
standard keyboards, and expressions have to be read right to left.
Most subsequent work numerical languages, such as Fortran-90,
MATLAB, and Mathematica, have tried to provide the power of APL
without the obscurity. Python's NumPy [2] has most of the
features that users of such languages expect, but these are
provided through named functions and methods, rather than
overloaded operators. This makes NumPy clumsier than most
alternatives.
The author of this PEP therefore consulted the developers of GNU
Octave [3], an open source clone of MATLAB. When asked how
important it was to have infix operators for matrix solution,
Prof. James Rawlings replied [4]:
I DON'T think it's a must have, and I do a lot of matrix
inversion. I cannot remember if its A\b or b\A so I always
write inv(A)*b instead. I recommend dropping \.
Rawlings' feedback on other operators was similar. It is worth
noting in this context that notations such as "/" and "\" for
matrix solution were invented by programmers, not mathematicians,
and have not been adopted by the latter.
Based on this discussion, and feedback from classes at the US
national laboratories and elsewhere, we recommend only adding a
matrix multiplication operator to Python at this time. If there
is significant user demand for syntactic support for other
operations, these can be added in a later release.
Requirements
The most important requirement is minimal impact on existing
Python programs and users: the proposal must not break existing
code (except possibly NumPy).
The second most important requirement is the ability to handle all
common cases cleanly and clearly. There are nine such cases:
|5 6| * 9 = |45 54| MS: matrix-scalar multiplication
|7 8| |63 72|
9 * |5 6| = |45 54| SM: scalar-matrix multiplication
|7 8| |63 72|
|2 3| * |4 5| = |8 15| VE: vector elementwise multiplication
|2 3| * |4| = 23 VD: vector dot product
|5|
|2| * |4 5| = | 8 10| VO: vector outer product
|3| |12 15|
|1 2| * |5 6| = | 5 12| ME: matrix elementwise multiplication
|3 4| |7 8| |21 32|
|1 2| * |5 6| = |19 22| MM: mathematical matrix multiplication
|3 4| |7 8| |43 50|
|1 2| * |5 6| = |19 22| VM: vector-matrix multiplication
|7 8|
|5 6| * |1| = |17| MV: matrix-vector multiplication
|7 8| |2| |23|
Note that 1-dimensional vectors are treated as rows in VM, as
columns in MV, and as both in VD and VO. Both are special cases
of 2-dimensional matrices (Nx1 and 1xN respectively). We will
therefore define the new operator only for 2-dimensional arrays,
and provide an easy (and efficient) way for users to treat
1-dimensional structures as 2-dimensional.
Third, we must avoid confusion between Python's notation and those
of MATLAB and Fortran-90. In particular, mathematical matrix
multiplication (case MM) should not be represented as '.*', since:
(a) MATLAB uses prefix-'.' forms to mean 'elementwise', and raw
forms to mean "mathematical"; and
(b) even if the Python parser can be taught how to handle dotted
forms, '1.*A' will still be visually ambiguous.
Proposal
The meanings of all existing operators will be unchanged. In
particular, 'A*B' will continue to be interpreted elementwise.
This takes care of the cases MS, SM, VE, and ME, and ensures
minimal impact on existing programs.
A new operator '@' (pronounced "across") will be added to Python,
along with special methods "__across__()", "__racross__()", and
"__iacross__()", with the usual semantics. (We recommend using
"@", rather than the times-like "><", because of the ease with
which the latter could be mis-typed as inequality "<>".)
No new operators will be defined to mean "solve a set of linear
equations", or "invert a matrix".
(Optional) When applied to sequences, the "@" operator will return
a tuple of tuples containing the cross-product of their elements
in left-to-right order:
>>> [1, 2] @ (3, 4)
((1, 3), (1, 4), (2, 3), (2, 4))
>>> [1, 2] @ (3, 4) @ (5, 6)
((1, 3, 5), (1, 3, 6),
(1, 4, 5), (1, 4, 6),
(2, 3, 5), (2, 3, 6),
(2, 4, 5), (2, 4, 6))
This will require the same kind of special support from the parser
as chained comparisons (such as "a<b<c<=d"). However, it will
permit:
>>> for (i, j) in [1, 2] @ [3, 4]:
>>> print i, j
1 3
1 4
2 3
2 4
as a short-hand for the common nested loop idiom:
>>> for i in [1, 2]:
>>> for j in [3, 4]:
>>> print i, j
Response to the 'lockstep loop' questionnaire [5] indicated that
newcomers would be comfortable with this (so comfortable, in fact,
that most of them interpreted most multi-loop 'zip' syntaxes [6]
as implementing single-stage nesting).
Alternatives
01. Don't add new operators.
Python is not primarily a numerical language; it may not be worth
complexifying it for this special case. NumPy's success is proof
that users can and will use functions and methods for linear
algebra. However, support for real matrix multiplication is
frequently requested, as:
* functional forms are cumbersome for lengthy formulas, and do not
respect the operator precedence rules of conventional mathematics;
and
* method forms are asymmetric in their operands.
What's more, the proposed semantics for "@" for built-in sequence
types would simplify expression of a very common idiom (nested
loops). User testing during discussion of 'lockstep loops'
indicated that both new and experienced users would understand
this immediately.
02. Introduce prefixed forms of all existing operators, such as
"~*" and "~+", as proposed in PEP 0225 [7].
This proposal would duplicate all built-in mathematical operators
with matrix equivalents, as in numerical languages such as
MATLAB. Our objections to this are:
* Python is not primarily a numerical programming language. While
the (self-selected) participants in the discussions that led to
PEP 0225 may want all of these new operators, the majority of
Python users would be indifferent. The extra complexity they
would introduce into the language therefore does not seem
merited. (See also Rawlings' comments, quoted in the Background
section, about these operators not being essential.)
* The proposed syntax is difficult to read (i.e. passes the "low
toner" readability test).
03. Retain the existing meaning of all operators, but create a
behavioral accessor for arrays, such that:
A * B
is elementwise multiplication (ME), but:
A.m() * B.m()
is mathematical multiplication (MM). The method "A.m()" would
return an object that aliased A's memory (for efficiency), but
which had a different implementation of __mul__().
This proposal was made by Moshe Zadka, and is also considered by
PEP 0225 [7]. Its advantage is that it has no effect on the
existing implementation of Python: changes are localized in the
Numeric module. The disadvantages are
* The semantics of "A.m() * B", "A + B.m()", and so on would have
to be defined, and there is no "obvious" choice for them.
* Aliasing objects to trigger different operator behavior feels
less Pythonic than either calling methods (as in the existing
Numeric module) or using a different operator. This PEP is
primarily about look and feel, and about making Python more
attractive to people who are not already using it.
Related Proposals
0207 : Rich Comparisons
It may become possible to overload comparison operators
such as '<' so that an expression such as 'A < B' returns
an array, rather than a scalar value.
0209 : Adding Multidimensional Arrays
Multidimensional arrays are currently an extension to
Python, rather than a built-in type.
0225 : Elementwise/Objectwise Operators
A larger proposal that addresses the same subject, but
which proposes many more additions to the language.
Acknowledgments
I am grateful to Huaiyu Zhu [8] for initiating this discussion,
and for some of the ideas and terminology included below.
References
[1] http://www.acm.org/sigapl/whyapl.htm
[2] http://numpy.sourceforge.net
[3] http://bevo.che.wisc.edu/octave/
[4] http://www.egroups.com/message/python-numeric/4
[5] http://www.python.org/pipermail/python-dev/2000-July/013139.html
[6] PEP-0201.txt "Lockstep Iteration"
[7] http://www.python.org/pipermail/python-list/2000-August/112529.html
Appendix: NumPy
NumPy will overload "@" to perform mathematical multiplication of
arrays where shapes permit, and to throw an exception otherwise.
Its implementation of "@" will treat built-in sequence types as if
they were column vectors. This takes care of the cases MM and MV.
An attribute "T" will be added to the NumPy array type, such that
"m.T" is:
(a) the transpose of "m" for a 2-dimensional array
(b) the 1xN matrix transpose of "m" if "m" is a 1-dimensional
array; or
(c) a runtime error for an array with rank >= 3.
This attribute will alias the memory of the base object. NumPy's
"transpose()" function will be extended to turn built-in sequence
types into row vectors. This takes care of the VM, VD, and VO
cases. We propose an attribute because:
(a) the resulting notation is similar to the 'superscript T' (at
least, as similar as ASCII allows), and
(b) it signals that the transposition aliases the original object.
NumPy will define a value "inv", which will be recognized by the
exponentiation operator, such that "A ** inv" is the inverse of
"A". This is similar in spirit to NumPy's existing "newaxis"
value.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: