705 lines
28 KiB
Plaintext
705 lines
28 KiB
Plaintext
PEP: 209
|
|
Title: Multi-dimensional Arrays
|
|
Author: Paul Barrett <barrett@stsci.edu>, Travis Oliphant <oliphant@ee.byu.edu>
|
|
Status: Withdrawn
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 03-Jan-2001
|
|
Python-Version: 2.2
|
|
Post-History:
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP proposes a redesign and re-implementation of the
|
|
multi-dimensional array module, Numeric, to make it easier to add
|
|
new features and functionality to the module. Aspects of Numeric 2
|
|
that will receive special attention are efficient access to arrays
|
|
exceeding a gigabyte in size and composed of inhomogeneous data
|
|
structures or records. The proposed design uses four Python
|
|
classes: ArrayType, UFunc, Array, and ArrayView; and a low-level
|
|
C-extension module, _ufunc, to handle the array operations
|
|
efficiently. In addition, each array type has its own C-extension
|
|
module which defines the coercion rules, operations, and methods
|
|
for that type. This design enables new types, features, and
|
|
functionality to be added in a modular fashion. The new version
|
|
will introduce some incompatibilities with the current Numeric.
|
|
|
|
|
|
Motivation
|
|
==========
|
|
|
|
Multi-dimensional arrays are commonly used to store and manipulate
|
|
data in science, engineering, and computing. Python currently has
|
|
an extension module, named Numeric (henceforth called Numeric 1),
|
|
which provides a satisfactory set of functionality for users
|
|
manipulating homogeneous arrays of data of moderate size (of order
|
|
10 MB). For access to larger arrays (of order 100 MB or more) of
|
|
possibly inhomogeneous data, the implementation of Numeric 1 is
|
|
inefficient and cumbersome. In the future, requests by the
|
|
Numerical Python community for additional functionality is also
|
|
likely as PEPs 211: Adding New Linear Operators to Python, and
|
|
225: Elementwise/Objectwise Operators illustrate.
|
|
|
|
|
|
Proposal
|
|
========
|
|
|
|
This proposal recommends a re-design and re-implementation of
|
|
Numeric 1, henceforth called Numeric 2, which will enable new
|
|
types, features, and functionality to be added in an easy and
|
|
modular manner. The initial design of Numeric 2 should focus on
|
|
providing a generic framework for manipulating arrays of various
|
|
types and should enable a straightforward mechanism for adding new
|
|
array types and UFuncs. Functional methods that are more specific
|
|
to various disciplines can then be layered on top of this core.
|
|
This new module will still be called Numeric and most of the
|
|
behavior found in Numeric 1 will be preserved.
|
|
|
|
The proposed design uses four Python classes: ArrayType, UFunc,
|
|
Array, and ArrayView; and a low-level C-extension module to handle
|
|
the array operations efficiently. In addition, each array type
|
|
has its own C-extension module which defines the coercion rules,
|
|
operations, and methods for that type. At a later date, when core
|
|
functionality is stable, some Python classes can be converted to
|
|
C-extension types.
|
|
|
|
Some planned features are:
|
|
|
|
1. Improved memory usage
|
|
|
|
This feature is particularly important when handling large arrays
|
|
and can produce significant improvements in performance as well as
|
|
memory usage. We have identified several areas where memory usage
|
|
can be improved:
|
|
|
|
a. Use a local coercion model
|
|
|
|
Instead of using Python's global coercion model which creates
|
|
temporary arrays, Numeric 2, like Numeric 1, will implement a
|
|
local coercion model as described in :pep:`208` which defers the
|
|
responsibility of coercion to the operator. By using internal
|
|
buffers, a coercion operation can be done for each array
|
|
(including output arrays), if necessary, at the time of the
|
|
operation. Benchmarks [1]_ have shown that performance is at
|
|
most degraded only slightly and is improved in cases where the
|
|
internal buffers are less than the L2 cache size and the
|
|
processor is under load. To avoid array coercion altogether,
|
|
C functions having arguments of mixed type are allowed in
|
|
Numeric 2.
|
|
|
|
b. Avoid creation of temporary arrays
|
|
|
|
In complex array expressions (i.e. having more than one
|
|
operation), each operation will create a temporary array which
|
|
will be used and then deleted by the succeeding operation. A
|
|
better approach would be to identify these temporary arrays
|
|
and reuse their data buffers when possible, namely when the
|
|
array shape and type are the same as the temporary array being
|
|
created. This can be done by checking the temporary array's
|
|
reference count. If it is 1, then it will be deleted once the
|
|
operation is done and is a candidate for reuse.
|
|
|
|
c. Optional use of memory-mapped files
|
|
|
|
Numeric users sometimes need to access data from very large
|
|
files or to handle data that is greater than the available
|
|
memory. Memory-mapped arrays provide a mechanism to do this
|
|
by storing the data on disk while making it appear to be in
|
|
memory. Memory- mapped arrays should improve access to all
|
|
files by eliminating one of two copy steps during a file
|
|
access. Numeric should be able to access in-memory and
|
|
memory-mapped arrays transparently.
|
|
|
|
d. Record access
|
|
|
|
In some fields of science, data is stored in files as binary
|
|
records. For example, in astronomy, photon data is stored as a
|
|
1 dimensional list of photons in order of arrival time. These
|
|
records or C-like structures contain information about the
|
|
detected photon, such as its arrival time, its position on the
|
|
detector, and its energy. Each field may be of a different
|
|
type, such as char, int, or float. Such arrays introduce new
|
|
issues that must be dealt with, in particular byte alignment
|
|
or byte swapping may need to be performed for the numeric
|
|
values to be properly accessed (though byte swapping is also
|
|
an issue for memory mapped data). Numeric 2 is designed to
|
|
automatically handle alignment and representational issues
|
|
when data is accessed or operated on. There are two
|
|
approaches to implementing records; as either a derived array
|
|
class or a special array type, depending on your point-of-view.
|
|
We defer this discussion to the Open Issues section.
|
|
|
|
|
|
2. Additional array types
|
|
|
|
Numeric 1 has 11 defined types: char, ubyte, sbyte, short, int,
|
|
long, float, double, cfloat, cdouble, and object. There are no
|
|
ushort, uint, or ulong types, nor are there more complex types
|
|
such as a bit type which is of use to some fields of science and
|
|
possibly for implementing masked-arrays. The design of Numeric 1
|
|
makes the addition of these and other types a difficult and
|
|
error-prone process. To enable the easy addition (and deletion)
|
|
of new array types such as a bit type described below, a re-design
|
|
of Numeric is necessary.
|
|
|
|
a. Bit type
|
|
|
|
The result of a rich comparison between arrays is an array of
|
|
boolean values. The result can be stored in an array of type
|
|
char, but this is an unnecessary waste of memory. A better
|
|
implementation would use a bit or boolean type, compressing
|
|
the array size by a factor of eight. This is currently being
|
|
implemented for Numeric 1 (by Travis Oliphant) and should be
|
|
included in Numeric 2.
|
|
|
|
3. Enhanced array indexing syntax
|
|
|
|
The extended slicing syntax was added to Python to provide greater
|
|
flexibility when manipulating Numeric arrays by allowing
|
|
step-sizes greater than 1. This syntax works well as a shorthand
|
|
for a list of regularly spaced indices. For those situations
|
|
where a list of irregularly spaced indices are needed, an enhanced
|
|
array indexing syntax would allow 1-D arrays to be arguments.
|
|
|
|
4. Rich comparisons
|
|
|
|
The implementation of :pep:`207`: Rich Comparisons in Python 2.1
|
|
provides additional flexibility when manipulating arrays. We
|
|
intend to implement this feature in Numeric 2.
|
|
|
|
5. Array broadcasting rules
|
|
|
|
When an operation between a scalar and an array is done, the
|
|
implied behavior is to create a new array having the same shape as
|
|
the array operand containing the scalar value. This is called
|
|
array broadcasting. It also works with arrays of lesser rank,
|
|
such as vectors. This implicit behavior is implemented in Numeric
|
|
1 and will also be implemented in Numeric 2.
|
|
|
|
|
|
Design and Implementation
|
|
=========================
|
|
|
|
The design of Numeric 2 has four primary classes:
|
|
|
|
1. ArrayType:
|
|
|
|
This is a simple class that describes the fundamental properties
|
|
of an array-type, e.g. its name, its size in bytes, its coercion
|
|
relations with respect to other types, etc., e.g.
|
|
|
|
::
|
|
|
|
Int32 = ArrayType('Int32', 4, 'doc-string')
|
|
|
|
Its relation to the other types is defined when the C-extension
|
|
module for that type is imported. The corresponding Python code
|
|
is::
|
|
|
|
Int32.astype[Real64] = Real64
|
|
|
|
This says that the Real64 array-type has higher priority than the
|
|
Int32 array-type.
|
|
|
|
The following attributes and methods are proposed for the core
|
|
implementation. Additional attributes can be added on an
|
|
individual basis, e.g. .bitsize or .bitstrides for the bit type.
|
|
|
|
Attributes::
|
|
|
|
.name: e.g. "Int32", "Float64", etc.
|
|
.typecode: e.g. 'i', 'f', etc.
|
|
(for backward compatibility)
|
|
.size (in bytes): e.g. 4, 8, etc.
|
|
.array_rules (mapping): rules between array types
|
|
.pyobj_rules (mapping): rules between array and python types
|
|
.doc: documentation string
|
|
|
|
Methods::
|
|
|
|
__init__(): initialization
|
|
__del__(): destruction
|
|
__repr__(): representation
|
|
|
|
C-API: This still needs to be fleshed-out.
|
|
|
|
|
|
2. UFunc:
|
|
|
|
This class is the heart of Numeric 2. Its design is similar to
|
|
that of ArrayType in that the UFunc creates a singleton callable
|
|
object whose attributes are name, total and input number of
|
|
arguments, a document string, and an empty CFunc dictionary; e.g.
|
|
|
|
::
|
|
|
|
add = UFunc('add', 3, 2, 'doc-string')
|
|
|
|
When defined the add instance has no C functions associated with
|
|
it and therefore can do no work. The CFunc dictionary is
|
|
populated or registered later when the C-extension module for an
|
|
array-type is imported. The arguments of the register method are:
|
|
function name, function descriptor, and the CUFunc object. The
|
|
corresponding Python code is
|
|
|
|
::
|
|
|
|
add.register('add', (Int32, Int32, Int32), cfunc-add)
|
|
|
|
In the initialization function of an array type module, e.g.
|
|
Int32, there are two C API functions: one to initialize the
|
|
coercion rules and the other to register the CFunc objects.
|
|
|
|
When an operation is applied to some arrays, the ``__call__`` method
|
|
is invoked. It gets the type of each array (if the output array
|
|
is not given, it is created from the coercion rules) and checks
|
|
the CFunc dictionary for a key that matches the argument types.
|
|
If it exists the operation is performed immediately, otherwise the
|
|
coercion rules are used to search for a related operation and set
|
|
of conversion functions. The ``__call__`` method then invokes a
|
|
compute method written in C to iterate over slices of each array,
|
|
namely::
|
|
|
|
_ufunc.compute(slice, data, func, swap, conv)
|
|
|
|
The 'func' argument is a CFuncObject, while the 'swap' and 'conv'
|
|
arguments are lists of CFuncObjects for those arrays needing pre- or
|
|
post-processing, otherwise None is used. The data argument is
|
|
a list of buffer objects, and the slice argument gives the number
|
|
of iterations for each dimension along with the buffer offset and
|
|
step size for each array and each dimension.
|
|
|
|
We have predefined several UFuncs for use by the ``__call__`` method:
|
|
cast, swap, getobj, and setobj. The cast and swap functions do
|
|
coercion and byte-swapping, respectively and the getobj and setobj
|
|
functions do coercion between Numeric arrays and Python sequences.
|
|
|
|
The following attributes and methods are proposed for the core
|
|
implementation.
|
|
|
|
Attributes::
|
|
|
|
.name: e.g. "add", "subtract", etc.
|
|
.nargs: number of total arguments
|
|
.iargs: number of input arguments
|
|
.cfuncs (mapping): the set C functions
|
|
.doc: documentation string
|
|
|
|
Methods::
|
|
|
|
__init__(): initialization
|
|
__del__(): destruction
|
|
__repr__(): representation
|
|
__call__(): look-up and dispatch method
|
|
initrule(): initialize coercion rule
|
|
uninitrule(): uninitialize coercion rule
|
|
register(): register a CUFunc
|
|
unregister(): unregister a CUFunc
|
|
|
|
C-API: This still needs to be fleshed-out.
|
|
|
|
3. Array:
|
|
|
|
This class contains information about the array, such as shape,
|
|
type, endian-ness of the data, etc.. Its operators, '+', '-',
|
|
etc. just invoke the corresponding UFunc function, e.g.
|
|
|
|
::
|
|
|
|
def __add__(self, other):
|
|
return ufunc.add(self, other)
|
|
|
|
The following attributes, methods, and functions are proposed for
|
|
the core implementation.
|
|
|
|
Attributes::
|
|
|
|
.shape: shape of the array
|
|
.format: type of the array
|
|
.real (only complex): real part of a complex array
|
|
.imag (only complex): imaginary part of a complex array
|
|
|
|
Methods::
|
|
|
|
__init__(): initialization
|
|
__del__(): destruction
|
|
__repr_(): representation
|
|
__str__(): pretty representation
|
|
__cmp__(): rich comparison
|
|
__len__():
|
|
__getitem__():
|
|
__setitem__():
|
|
__getslice__():
|
|
__setslice__():
|
|
numeric methods:
|
|
copy(): copy of array
|
|
aslist(): create list from array
|
|
asstring(): create string from array
|
|
|
|
Functions::
|
|
|
|
fromlist(): create array from sequence
|
|
fromstring(): create array from string
|
|
array(): create array with shape and value
|
|
concat(): concatenate two arrays
|
|
resize(): resize array
|
|
|
|
C-API: This still needs to be fleshed-out.
|
|
|
|
4. ArrayView
|
|
|
|
This class is similar to the Array class except that the reshape
|
|
and flat methods will raise exceptions, since non-contiguous
|
|
arrays cannot be reshaped or flattened using just pointer and
|
|
step-size information.
|
|
|
|
C-API: This still needs to be fleshed-out.
|
|
|
|
5. C-extension modules:
|
|
|
|
Numeric2 will have several C-extension modules.
|
|
|
|
a. _ufunc:
|
|
|
|
The primary module of this set is the _ufuncmodule.c. The
|
|
intention of this module is to do the bare minimum,
|
|
i.e. iterate over arrays using a specified C function. The
|
|
interface of these functions is the same as Numeric 1, i.e.
|
|
|
|
::
|
|
|
|
int (*CFunc)(char *data, int *steps, int repeat, void *func);
|
|
|
|
and their functionality is expected to be the same, i.e. they
|
|
iterate over the inner-most dimension.
|
|
|
|
The following attributes and methods are proposed for the core
|
|
implementation.
|
|
|
|
Attributes:
|
|
|
|
Methods::
|
|
|
|
compute():
|
|
|
|
C-API: This still needs to be fleshed-out.
|
|
|
|
b. _int32, _real64, etc.:
|
|
|
|
There will also be C-extension modules for each array type,
|
|
e.g. _int32module.c, _real64module.c, etc. As mentioned
|
|
previously, when these modules are imported by the UFunc
|
|
module, they will automatically register their functions and
|
|
coercion rules. New or improved versions of these modules can
|
|
be easily implemented and used without affecting the rest of
|
|
Numeric 2.
|
|
|
|
|
|
Open Issues
|
|
===========
|
|
|
|
1. Does slicing syntax default to copy or view behavior?
|
|
|
|
The default behavior of Python is to return a copy of a sub-list
|
|
or tuple when slicing syntax is used, whereas Numeric 1 returns a
|
|
view into the array. The choice made for Numeric 1 is apparently
|
|
for reasons of performance: the developers wish to avoid the
|
|
penalty of allocating and copying the data buffer during each
|
|
array operation and feel that the need for a deep copy of an array
|
|
to be rare. Yet, some have argued that Numeric's slice notation
|
|
should also have copy behavior to be consistent with Python lists.
|
|
In this case the performance penalty associated with copy behavior
|
|
can be minimized by implementing copy-on-write. This scheme has
|
|
both arrays sharing one data buffer (as in view behavior) until
|
|
either array is assigned new data at which point a copy of the
|
|
data buffer is made. View behavior would then be implemented by
|
|
an ArrayView class, whose behavior be similar to Numeric 1 arrays,
|
|
i.e. .shape is not settable for non-contiguous arrays. The use of
|
|
an ArrayView class also makes explicit what type of data the array
|
|
contains.
|
|
|
|
2. Does item syntax default to copy or view behavior?
|
|
|
|
A similar question arises with the item syntax. For example, if
|
|
``a = [[0,1,2], [3,4,5]]`` and ``b = a[0]``, then changing ``b[0]`` also changes
|
|
``a[0][0]``, because ``a[0]`` is a reference or view of the first row of a.
|
|
Therefore, if c is a 2-d array, it would appear that ``c[i]``
|
|
should return a 1-d array which is a view into, instead of a copy
|
|
of, c for consistency. Yet, ``c[i]`` can be considered just a
|
|
shorthand for ``c[i,:]`` which would imply copy behavior assuming
|
|
slicing syntax returns a copy. Should Numeric 2 behave the same
|
|
way as lists and return a view or should it return a copy.
|
|
|
|
3. How is scalar coercion implemented?
|
|
|
|
Python has fewer numeric types than Numeric which can cause
|
|
coercion problems. For example, when multiplying a Python scalar
|
|
of type float and a Numeric array of type float, the Numeric array
|
|
is converted to a double, since the Python float type is actually
|
|
a double. This is often not the desired behavior, since the
|
|
Numeric array will be doubled in size which is likely to be
|
|
annoying, particularly for very large arrays. We prefer that the
|
|
array type trumps the python type for the same type class, namely
|
|
integer, float, and complex. Therefore, an operation between a
|
|
Python integer and an Int16 (short) array will return an Int16
|
|
array. Whereas an operation between a Python float and an Int16
|
|
array would return a Float64 (double) array. Operations between
|
|
two arrays use normal coercion rules.
|
|
|
|
4. How is integer division handled?
|
|
|
|
In a future version of Python, the behavior of integer division
|
|
will change. The operands will be converted to floats, so the
|
|
result will be a float. If we implement the proposed scalar
|
|
coercion rules where arrays have precedence over Python scalars,
|
|
then dividing an array by an integer will return an integer array
|
|
and will not be consistent with a future version of Python which
|
|
would return an array of type double. Scientific programmers are
|
|
familiar with the distinction between integer and float-point
|
|
division, so should Numeric 2 continue with this behavior?
|
|
|
|
5. How should records be implemented?
|
|
|
|
There are two approaches to implementing records depending on your
|
|
point-of-view. The first is two divide arrays into separate
|
|
classes depending on the behavior of their types. For example,
|
|
numeric arrays are one class, strings a second, and records a
|
|
third, because the range and type of operations of each class
|
|
differ. As such, a record array is not a new type, but a
|
|
mechanism for a more flexible form of array. To easily access and
|
|
manipulate such complex data, the class is comprised of numeric
|
|
arrays having different byte offsets into the data buffer. For
|
|
example, one might have a table consisting of an array of Int16,
|
|
Real32 values. Two numeric arrays, one with an offset of 0 bytes
|
|
and a stride of 6 bytes to be interpreted as Int16, and one with an
|
|
offset of 2 bytes and a stride of 6 bytes to be interpreted as
|
|
Real32 would represent the record array. Both numeric arrays
|
|
would refer to the same data buffer, but have different offset and
|
|
stride attributes, and a different numeric type.
|
|
|
|
The second approach is to consider a record as one of many array
|
|
types, albeit with fewer, and possibly different, array operations
|
|
than for numeric arrays. This approach considers an array type to
|
|
be a mapping of a fixed-length string. The mapping can either be
|
|
simple, like integer and floating-point numbers, or complex, like
|
|
a complex number, a byte string, and a C-structure. The record
|
|
type effectively merges the struct and Numeric modules into a
|
|
multi-dimensional struct array. This approach implies certain
|
|
changes to the array interface. For example, the 'typecode'
|
|
keyword argument should probably be changed to the more
|
|
descriptive 'format' keyword.
|
|
|
|
a. How are record semantics defined and implemented?
|
|
|
|
Which ever implementation approach is taken for records, the
|
|
syntax and semantics of how they are to be accessed and
|
|
manipulated must be decided, if one wishes to have access to
|
|
sub-fields of records. In this case, the record type can
|
|
essentially be considered an inhomogeneous list, like a tuple
|
|
returned by the unpack method of the struct module; and a 1-d
|
|
array of records may be interpreted as a 2-d array with the
|
|
second dimension being the index into the list of fields.
|
|
This enhanced array semantics makes access to an array of one
|
|
or more of the fields easy and straightforward. It also
|
|
allows a user to do array operations on a field in a natural
|
|
and intuitive way. If we assume that records are implemented
|
|
as an array type, then last dimension defaults to 0 and can
|
|
therefore be neglected for arrays comprised of simple types,
|
|
like numeric.
|
|
|
|
6. How are masked-arrays implemented?
|
|
|
|
Masked-arrays in Numeric 1 are implemented as a separate array
|
|
class. With the ability to add new array types to Numeric 2, it
|
|
is possible that masked-arrays in Numeric 2 could be implemented
|
|
as a new array type instead of an array class.
|
|
|
|
7. How are numerical errors handled (IEEE floating-point errors in
|
|
particular)?
|
|
|
|
It is not clear to the proposers (Paul Barrett and Travis
|
|
Oliphant) what is the best or preferred way of handling errors.
|
|
Since most of the C functions that do the operation, iterate over
|
|
the inner-most (last) dimension of the array. This dimension
|
|
could contain a thousand or more items having one or more errors
|
|
of differing type, such as divide-by-zero, underflow, and
|
|
overflow. Additionally, keeping track of these errors may come at
|
|
the expense of performance. Therefore, we suggest several
|
|
options:
|
|
|
|
a. Print a message of the most severe error, leaving it to
|
|
the user to locate the errors.
|
|
|
|
b. Print a message of all errors that occurred and the number
|
|
of occurrences, leaving it to the user to locate the errors.
|
|
|
|
c. Print a message of all errors that occurred and a list of
|
|
where they occurred.
|
|
|
|
d. Or use a hybrid approach, printing only the most severe
|
|
error, yet keeping track of what and where the errors
|
|
occurred. This would allow the user to locate the errors
|
|
while keeping the error message brief.
|
|
|
|
8. What features are needed to ease the integration of FORTRAN
|
|
libraries and code?
|
|
|
|
It would be a good idea at this stage to consider how to ease the
|
|
integration of FORTRAN libraries and user code in Numeric 2.
|
|
|
|
|
|
Implementation Steps
|
|
====================
|
|
|
|
1. Implement basic UFunc capability
|
|
|
|
a. Minimal Array class:
|
|
|
|
Necessary class attributes and methods, e.g. .shape, .data,
|
|
.type, etc.
|
|
|
|
b. Minimal ArrayType class:
|
|
|
|
Int32, Real64, Complex64, Char, Object
|
|
|
|
c. Minimal UFunc class:
|
|
|
|
UFunc instantiation, CFunction registration, UFunc call for
|
|
1-D arrays including the rules for doing alignment,
|
|
byte-swapping, and coercion.
|
|
|
|
d. Minimal C-extension module:
|
|
|
|
_UFunc, which does the innermost array loop in C.
|
|
|
|
This step implements whatever is needed to do: 'c = add(a, b)'
|
|
where a, b, and c are 1-D arrays. It teaches us how to add
|
|
new UFuncs, to coerce the arrays, to pass the necessary
|
|
information to a C iterator method and to do the actually
|
|
computation.
|
|
|
|
2. Continue enhancing the UFunc iterator and Array class
|
|
|
|
a. Implement some access methods for the Array class:
|
|
print, repr, getitem, setitem, etc.
|
|
|
|
b. Implement multidimensional arrays
|
|
|
|
c. Implement some of basic Array methods using UFuncs:
|
|
+, -, \*, /, etc.
|
|
|
|
d. Enable UFuncs to use Python sequences.
|
|
|
|
3. Complete the standard UFunc and Array class behavior
|
|
|
|
a. Implement getslice and setslice behavior
|
|
|
|
b. Work on Array broadcasting rules
|
|
|
|
c. Implement Record type
|
|
|
|
4. Add additional functionality
|
|
|
|
a. Add more UFuncs
|
|
|
|
b. Implement buffer or mmap access
|
|
|
|
|
|
Incompatibilities
|
|
=================
|
|
|
|
The following is a list of incompatibilities in behavior between
|
|
Numeric 1 and Numeric 2.
|
|
|
|
1. Scalar coercion rules
|
|
|
|
Numeric 1 has single set of coercion rules for array and Python
|
|
numeric types. This can cause unexpected and annoying problems
|
|
during the calculation of an array expression. Numeric 2 intends
|
|
to overcome these problems by having two sets of coercion rules:
|
|
one for arrays and Python numeric types, and another just for
|
|
arrays.
|
|
|
|
2. No savespace attribute
|
|
|
|
The savespace attribute in Numeric 1 makes arrays with this
|
|
attribute set take precedence over those that do not have it set.
|
|
Numeric 2 will not have such an attribute and therefore normal
|
|
array coercion rules will be in effect.
|
|
|
|
3. Slicing syntax returns a copy
|
|
|
|
The slicing syntax in Numeric 1 returns a view into the original
|
|
array. The slicing behavior for Numeric 2 will be a copy. You
|
|
should use the ArrayView class to get a view into an array.
|
|
|
|
4. Boolean comparisons return a boolean array
|
|
|
|
A comparison between arrays in Numeric 1 results in a Boolean
|
|
scalar, because of current limitations in Python. The advent of
|
|
Rich Comparisons in Python 2.1 will allow an array of Booleans to
|
|
be returned.
|
|
|
|
5. Type characters are deprecated
|
|
|
|
Numeric 2 will have an ArrayType class composed of Type instances,
|
|
for example Int8, Int16, Int32, and Int for signed integers. The
|
|
typecode scheme in Numeric 1 will be available for backward
|
|
compatibility, but will be deprecated.
|
|
|
|
|
|
Appendices
|
|
==========
|
|
|
|
A. Implicit sub-arrays iteration
|
|
|
|
A computer animation is composed of a number of 2-D images or
|
|
frames of identical shape. By stacking these images into a single
|
|
block of memory, a 3-D array is created. Yet the operations to be
|
|
performed are not meant for the entire 3-D array, but on the set
|
|
of 2-D sub-arrays. In most array languages, each frame has to be
|
|
extracted, operated on, and then reinserted into the output array
|
|
using a for-like loop. The J language allows the programmer to
|
|
perform such operations implicitly by having a rank for the frame
|
|
and array. By default these ranks will be the same during the
|
|
creation of the array. It was the intention of the Numeric 1
|
|
developers to implement this feature, since it is based on the
|
|
language J. The Numeric 1 code has the required variables for
|
|
implementing this behavior, but was never implemented. We intend
|
|
to implement implicit sub-array iteration in Numeric 2, if the
|
|
array broadcasting rules found in Numeric 1 do not fully support
|
|
this behavior.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain.
|
|
|
|
|
|
Related PEPs
|
|
============
|
|
|
|
* :pep:`207`: Rich Comparisons
|
|
by Guido van Rossum and David Ascher
|
|
|
|
* :pep:`208`: Reworking the Coercion Model
|
|
by Neil Schemenauer and Marc-Andre' Lemburg
|
|
|
|
* :pep:`211`: Adding New Linear Algebra Operators to Python
|
|
by Greg Wilson
|
|
|
|
* :pep:`225`: Elementwise/Objectwise Operators
|
|
by Huaiyu Zhu
|
|
|
|
* :pep:`228`: Reworking Python's Numeric Model
|
|
by Moshe Zadka
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] P. Greenfield 2000. private communication.
|