Change PyBuffer to Py_buffer in PEP-3118 and clear up the locking mechanism. There are four locking modes (standard read, standard write, read lock --- no other object can write to the array while the current object holds the lock, and exclusive write lock -- no other object can read or write to the data-buffer.
This commit is contained in:
Travis E. Oliphant
parent
b0c2bd0cab
commit
ca6cf4fb2b
132
pep-3118.txt
132
pep-3118.txt
|
|
@ -87,7 +87,7 @@ has issues:
|
|||
NumPy's strided memory model is used more often in computational
|
||||
libraries and because it is so simple it makes sense to support
|
||||
memory sharing using this model. The PIL memory model is sometimes
|
||||
used in C-code where a 2-d array can be then accessed using double
|
||||
used in C-code where a 2-d array can then be accessed using double
|
||||
pointer indirection: e.g. ``image[i][j]``.
|
||||
|
||||
The buffer interface should allow the object to export either of these
|
||||
|
|
@ -188,31 +188,38 @@ without that information, then a ``PyErr_BufferError`` should be raised.
|
|||
``PyBUF_SIMPLE``
|
||||
|
||||
This is the default flag state (0). The returned buffer may or may
|
||||
not have writeable memory. The format will be assumed to be
|
||||
not have writable memory. The format will be assumed to be
|
||||
unsigned bytes . This is a "stand-alone" flag constant. It never
|
||||
needs to be \|'d to the others. The exporter will raise an error if
|
||||
it cannot provide such a contiguous buffer of bytes.
|
||||
|
||||
``PyBUF_CHARACTER``
|
||||
``PyBUF_WRITABLE``
|
||||
|
||||
This essentially replaces the separate function for getting a character
|
||||
buffer (which is useful in at least the unicode type). If an object
|
||||
should do something different when it is requested as a character
|
||||
buffer then it should detect this flag and respond differently.
|
||||
|
||||
``PyBUF_WRITEABLE``
|
||||
|
||||
The returned buffer must be writeable. If it is not writeable,
|
||||
The returned buffer must be writable. If it is not writable,
|
||||
then raise an error.
|
||||
|
||||
``PyBUF_LOCKDATA``
|
||||
``PyBUF_LOCK``
|
||||
|
||||
The returned buffer must be readonly. If the object is already
|
||||
read-only or it can make its memory read-only (and there are no
|
||||
other writeable views on the object) then it should do so and
|
||||
return the buffer information. If the object does not have
|
||||
read-only memory (or cannot make it read-only), then an error
|
||||
should be raised.
|
||||
This flag is used to request a lock (either a read-lock or an
|
||||
exclusive write lock) on the data-area of the object before the
|
||||
buffer is returned. If the lock cannot be obtained, then an error
|
||||
should be raised. In conjunction with the PyBUF_WRITABLE flag, the
|
||||
PyBUF_LOCK flag creates four different access modes on the
|
||||
data-area of the buffer memory: read, read-with-write-lock, write,
|
||||
and exclusive write. The access modes are
|
||||
|
||||
================ ================= ==========================
|
||||
flags Description Other Requests Can
|
||||
================ ================= ==========================
|
||||
neither read read and write
|
||||
WRITABLE write read and write
|
||||
LOCK locked read read but not write
|
||||
WRITABLE | LOCK exclusive write neither read nor write
|
||||
================ ================= ==========================
|
||||
|
||||
Care should be taken not to LOCK the buffer unless it is really
|
||||
necessary (especially the exlcusive write lock) as it makes the
|
||||
object unable to share its memory until the lock is released.
|
||||
|
||||
``PyBUF_FORMAT``
|
||||
|
||||
|
|
@ -223,14 +230,14 @@ without that information, then a ``PyErr_BufferError`` should be raised.
|
|||
format is not explicitly requested then the format must be returned
|
||||
as ``NULL`` (which means "B")
|
||||
|
||||
``PyBUF_ALW_ND``
|
||||
``PyBUF_ND``
|
||||
|
||||
The returned buffer must provide shape information. The memory will
|
||||
be assumed C-style contiguous (last dimension varies the fastest).
|
||||
The exporter may raise an error if it cannot provide this kind of
|
||||
contiguous buffer. If this is not given then shape will be NULL.
|
||||
|
||||
``PyBUF_ALW_STRIDES`` (implies ``PyBUF_ALW_ND``)
|
||||
``PyBUF_STRIDES`` (implies ``PyBUF_ND``)
|
||||
|
||||
The returned buffer must provide strides information (i.e. the
|
||||
strides cannot be NULL). This would be used when the consumer can
|
||||
|
|
@ -246,11 +253,11 @@ without that information, then a ``PyErr_BufferError`` should be raised.
|
|||
These flags indicate that the returned buffer must be respectively,
|
||||
C-contiguous (last dimension varies the fastest), Fortran
|
||||
contiguous (first dimension varies the fastest) or either one.
|
||||
All of these flags imply PyBUF_ALW_STRIDES and guarantee that the
|
||||
All of these flags imply PyBUF_STRIDES and guarantee that the
|
||||
strides buffer info structure will be filled in correctly.
|
||||
|
||||
|
||||
``PyBUF_ALW_INDIRECT`` (implies ``PyBUF_ALW_STRIDES``)
|
||||
``PyBUF_INDIRECT`` (implies ``PyBUF_STRIDES``)
|
||||
|
||||
The returned buffer must have suboffsets information (which can be
|
||||
NULL if no suboffsets are needed). This would be used when the
|
||||
|
|
@ -262,27 +269,31 @@ Specialized combinations of flags for specific kinds of memory_sharing.
|
|||
|
||||
Multi-dimensional (but contiguous)
|
||||
|
||||
| ``PyBUF_CONTIG`` (``PyBUF_ALW_ND | PyBUF_WRITEABLE``)
|
||||
| ``PyBUF_CONTIG_RO`` (``PyBUF_ALW_ND``)
|
||||
| ``PyBUF_CONTIG_LCK`` (``PyBUF_ALW_ND | PyBUF_LOCKDATA``)
|
||||
| ``PyBUF_CONTIG`` (``PyBUF_ND | PyBUF_WRITABLE``)
|
||||
| ``PyBUF_CONTIG_RO`` (``PyBUF_ND``)
|
||||
| ``PyBUF_CONTIG_LCK`` (``PyBUF_ND | PyBUF_LOCK``)
|
||||
| ``PyBUF_CONTIG_XLCK`` (``PyBUF_ND | PyBUF_WRITABLE | PyBUF_LOCK``)
|
||||
|
||||
Multi-dimensional using strides but aligned
|
||||
|
||||
| ``PyBUF_STRIDED`` (``PyBUF_ALW_STRIDES | PyBUF_WRITEABLE``)
|
||||
| ``PyBUF_STRIDED_RO`` (``PyBUF_ALW_STRIDES``)
|
||||
| ``PyBUF_STRIDED_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_LOCKDATA``)
|
||||
| ``PyBUF_STRIDED`` (``PyBUF_STRIDES | PyBUF_WRITABLE``)
|
||||
| ``PyBUF_STRIDED_RO`` (``PyBUF_STRIDES``)
|
||||
| ``PyBUF_STRIDED_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK``)
|
||||
| ``PyBUF_STRIDED_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_WRITABLE``)
|
||||
|
||||
Multi-dimensional using strides and not necessarily aligned
|
||||
|
||||
| ``PyBUF_RECORDS`` (``PyBUF_ALW_STRIDES | PyBUF_WRITEABLE | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS_RO`` (``PyBUF_ALW_STRIDES | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_LOCKDATA | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS`` (``PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS_RO`` (``PyBUF_STRIDES | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT``)
|
||||
| ``PyBUF_RECORDS_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
|
||||
|
||||
Multi-dimensional using sub-offsets
|
||||
|
||||
| ``PyBUF_FULL`` (``PyBUF_ALW_INDIRECT | PyBUF_WRITEABLE | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL_RO`` (``PyBUF_ALW_INDIRECT | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL_LCK`` (``PyBUF_ALW_INDIRECT | PyBUF_LOCKDATA | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL`` (``PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL_RO`` (``PyBUF_INDIRECT | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL_LCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT``)
|
||||
| ``PyBUF_FULL_XLCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
|
||||
|
||||
Thus, the consumer simply wanting a contiguous chunk of bytes from
|
||||
the object would use ``PyBUF_SIMPLE``, while a consumer that understands
|
||||
|
|
@ -309,7 +320,7 @@ The bufferinfo structure is::
|
|||
Py_ssize_t *suboffsets;
|
||||
Py_ssize_t itemsize;
|
||||
void *internal;
|
||||
} PyBuffer;
|
||||
} Py_buffer;
|
||||
|
||||
Before calling this function, the bufferinfo structure can be filled
|
||||
with whatever. Upon return from getbufferproc, the bufferinfo
|
||||
|
|
@ -332,11 +343,14 @@ The members of the bufferinfo structure are:
|
|||
|
||||
``readonly``
|
||||
an integer variable to hold whether or not the memory is readonly.
|
||||
1 means the memory is readonly, zero means the memory is
|
||||
writeable, and -1 means the memory was "locked" (changed from
|
||||
writeable to readonly) when this PyBuffer structure was filled-in
|
||||
and therefore should be unlocked when this PyBuffer structure is
|
||||
"released" (this is supported only by some exporters).
|
||||
1 means the memory is readonly, zero means the memory is writable,
|
||||
-1 means the memory was "locked" (changed from writable to
|
||||
readonly) when this Py_buffer structure was filled-in after a
|
||||
readable request and therefore should be unlocked when this
|
||||
Py_buffer structure is "released" (this is supported only by some
|
||||
exporters). A -2 means this Py_buffer structure has an
|
||||
exclusive-write lock on the memory. This should be unlocked when
|
||||
the Py_buffer structure is released.
|
||||
|
||||
``format``
|
||||
a NULL-terminated format-string (following the struct-style syntax
|
||||
|
|
@ -361,7 +375,7 @@ The members of the bufferinfo structure are:
|
|||
pointer to an array of ``Py_ssize_t`` of length ``ndims`` (or ``NULL``
|
||||
if ``ndims`` is 0). indicating the number of bytes to skip to get to
|
||||
the next element in each dimension. If this is not requested by
|
||||
the caller (``PyBUF_ALW_STRIDES`` is not set), then this should be set
|
||||
the caller (``PyBUF_STRIDES`` is not set), then this should be set
|
||||
to NULL which indicates a C-style contiguous array or a
|
||||
PyExc_BufferError raised if this is not possible.
|
||||
|
||||
|
|
@ -375,7 +389,7 @@ The members of the bufferinfo structure are:
|
|||
should occur (striding in a contiguous memory block). If all
|
||||
suboffsets are negative (i.e. no de-referencing is needed, then
|
||||
this must be NULL (the default value). If this is not requested
|
||||
by the caller (PyBUF_ALW_INDIRECT is not set), then this should be
|
||||
by the caller (PyBUF_INDIRECT is not set), then this should be
|
||||
set to NULL or an PyExc_BufferError raised if this is not possible.
|
||||
|
||||
For clarity, here is a function that returns a pointer to the
|
||||
|
|
@ -428,9 +442,9 @@ when releasebuffer is called.
|
|||
|
||||
The same bufferinfo struct should be used in the release-buffer
|
||||
interface call. The caller is responsible for the memory of the
|
||||
PyBuffer structure itself.
|
||||
Py_buffer structure itself.
|
||||
|
||||
``typedef void (*releasebufferproc)(PyObject *obj, PyBuffer *view)``
|
||||
``typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)``
|
||||
Callers of getbufferproc must make sure that this function is
|
||||
called when memory previously acquired from the object is no
|
||||
longer needed. The exporter of the interface must make sure that
|
||||
|
|
@ -468,7 +482,7 @@ Return 1 if the getbuffer function is available otherwise 0.
|
|||
|
||||
::
|
||||
|
||||
int PyObject_GetBuffer(PyObject *obj, PyBuffer *view,
|
||||
int PyObject_GetBuffer(PyObject *obj, Py_buffer *view,
|
||||
int flags)
|
||||
|
||||
This is a C-API version of the getbuffer function call. It checks to
|
||||
|
|
@ -478,7 +492,7 @@ success.
|
|||
|
||||
::
|
||||
|
||||
void PyObject_ReleaseBuffer(PyObject *obj, PyBuffer *view)
|
||||
void PyObject_ReleaseBuffer(PyObject *obj, Py_buffer *view)
|
||||
|
||||
This is a C-API version of the releasebuffer function call. It checks
|
||||
to make sure the object has the required function pointer and issues
|
||||
|
|
@ -498,7 +512,7 @@ simple 1-d memory-view object). Its C-structure is ::
|
|||
typedef struct {
|
||||
PyObject_HEAD
|
||||
PyObject *base;
|
||||
PyBuffer view;
|
||||
Py_buffer view;
|
||||
} PyMemoryViewObject;
|
||||
|
||||
This is functionally similar to the current buffer object except a
|
||||
|
|
@ -566,14 +580,14 @@ the base object for the returned memory view object.
|
|||
|
||||
The buffertype argument can be PyBUF_READ, PyBUF_WRITE,
|
||||
PyBUF_UPDATEIFCOPY to determine whether the returned buffer should be
|
||||
readable, writeable, or set to update the original buffer if a copy
|
||||
readable, writable, or set to update the original buffer if a copy
|
||||
must be made. If buffertype is PyBUF_WRITE and the buffer is not
|
||||
contiguous an error will be raised. In this circumstance, the user
|
||||
can use PyBUF_UPDATEIFCOPY to ensure that a a writeable temporary
|
||||
can use PyBUF_UPDATEIFCOPY to ensure that a a writable temporary
|
||||
contiguous buffer is returned. The contents of this contiguous buffer
|
||||
will be copied back into the original object after the memoryview
|
||||
object is deleted as long as the original object is writeable and
|
||||
allows setting its memory to "readonly". If this is not allowed by
|
||||
object is deleted as long as the original object is writable and
|
||||
allows applying a read-write lock. If this is not allowed by
|
||||
the original object, then a BufferError is raised.
|
||||
|
||||
If the object is multi-dimensional, then if fortran is 'F', the first
|
||||
|
|
@ -595,7 +609,7 @@ the memoryview object is deleted).
|
|||
Copy ``len`` bytes of data pointed to by the contiguous chunk of
|
||||
memory pointed to by ``buf`` into the buffer exported by obj. Return
|
||||
0 on success and return -1 and raise an error on failure. If the
|
||||
object does not have a writeable buffer, then an error is raised. If
|
||||
object does not have a writable buffer, then an error is raised. If
|
||||
fortran is 'F', then if the object is multi-dimensional, then the data
|
||||
will be copied into the array in Fortran-style (first dimension varies
|
||||
the fastest). If fortran is 'C', then the data will be copied into
|
||||
|
|
@ -614,7 +628,7 @@ their work.
|
|||
|
||||
::
|
||||
|
||||
int PyBuffer_IsContiguous(PyBuffer *view, char fortran)
|
||||
int PyBuffer_IsContiguous(Py_buffer *view, char fortran)
|
||||
|
||||
Return 1 if the memory defined by the view object is C-style (fortran
|
||||
= 'C') or Fortran-style (fortran = 'F') contiguous or either one
|
||||
|
|
@ -632,7 +646,7 @@ shape with the given number of bytes per element.
|
|||
|
||||
::
|
||||
|
||||
int PyBuffer_FillInfo(PyBuffer *view, void *buf,
|
||||
int PyBuffer_FillInfo(Py_buffer *view, void *buf,
|
||||
Py_ssize_t len, int readonly, int infoflags)
|
||||
|
||||
Fills in a buffer-info structure correctly for an exporter that can
|
||||
|
|
@ -679,8 +693,6 @@ Character Description
|
|||
'X{}' pointer to a function (optional function
|
||||
signature inside {} with any return value
|
||||
preceeded by -> and placed at the end)
|
||||
' ', '\\n', \\t' ignored (allow better readability)
|
||||
-- this may already be true
|
||||
================ ===========
|
||||
|
||||
The struct module will be changed to understand these as well and
|
||||
|
|
@ -862,7 +874,7 @@ In order to access, say, the red value of the pixel at x=30, y=50, you'd use "li
|
|||
|
||||
So what does ImageObject's getbuffer do? Leaving error checking out::
|
||||
|
||||
int Image_getbuffer(PyObject *self, PyBuffer *view, int flags) {
|
||||
int Image_getbuffer(PyObject *self, Py_buffer *view, int flags) {
|
||||
|
||||
static Py_ssize_t suboffsets[2] = { 0, -1};
|
||||
|
||||
|
|
@ -884,7 +896,7 @@ So what does ImageObject's getbuffer do? Leaving error checking out::
|
|||
}
|
||||
|
||||
|
||||
int Image_releasebuffer(PyObject *self, PyBuffer *view) {
|
||||
int Image_releasebuffer(PyObject *self, Py_buffer *view) {
|
||||
self->view_count--;
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -899,7 +911,7 @@ alive) would do that.
|
|||
|
||||
::
|
||||
|
||||
int myobject_getbuffer(PyObject *self, PyBuffer *view, int flags) {
|
||||
int myobject_getbuffer(PyObject *self, Py_buffer *view, int flags) {
|
||||
|
||||
void *buf;
|
||||
Py_ssize_t len;
|
||||
|
|
@ -924,7 +936,7 @@ from a Python object, obj would do the following:
|
|||
|
||||
::
|
||||
|
||||
PyBuffer view;
|
||||
Py_buffer view;
|
||||
int ret;
|
||||
|
||||
if (PyObject_GetBuffer(obj, &view, Py_BUF_SIMPLE) < 0) {
|
||||
|
|
|
|||
Loading…
Reference in New Issue