Changes to pep-3118.txt
This commit is contained in:
parent
ac9a2701b5
commit
5f33b4b632
525
pep-3118.txt
525
pep-3118.txt
|
@ -1,8 +1,9 @@
|
|||
|
||||
PEP: 3118
|
||||
Title: Revising the buffer protocol
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Travis Oliphant <oliphant@ee.byu.edu>, Carl Banks <pythondev@aerojockey.com>
|
||||
Authors: Travis Oliphant <oliphant@ee.byu.edu>, Carl Banks <pythondev@aerojockey.com>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
|
@ -13,8 +14,8 @@ Abstract
|
|||
========
|
||||
|
||||
This PEP proposes re-designing the buffer interface (PyBufferProcs
|
||||
function pointers) to improve the way Python allows memory sharing
|
||||
in Python 3.0
|
||||
function pointers) to improve the way Python allows memory sharing in
|
||||
Python 3.0
|
||||
|
||||
In particular, it is proposed that the character buffer portion
|
||||
of the API be elminated and the multiple-segment portion be
|
||||
|
@ -73,7 +74,7 @@ has issues:
|
|||
|
||||
NumPy uses the notion of constant striding in each dimension as its
|
||||
basic concept of an array. With this concept, a simple sub-region
|
||||
of a larger array can be described without copying the data. T
|
||||
of a larger array can be described without copying the data.
|
||||
Thus, stride information is the additional information that must be
|
||||
shared.
|
||||
|
||||
|
@ -151,7 +152,7 @@ Change the PyBufferProcs structure to
|
|||
|
||||
::
|
||||
|
||||
typedef int (*getbufferproc)(PyObject *obj, struct bufferinfo *view, int flags)
|
||||
typedef int (*getbufferproc)(PyObject *obj, PyBuffer *view, int flags)
|
||||
|
||||
This function returns 0 on success and -1 on failure (and raises an
|
||||
error). The first variable is the "exporting" object. The second
|
||||
|
@ -160,54 +161,137 @@ then no information is returned but a lock on the memory is still
|
|||
obtained. In this case, the corresponding releasebuffer should also
|
||||
be called with NULL.
|
||||
|
||||
The third argument indicates what kind of buffer the exporter is allowed to return. It tells the
|
||||
exporter what elements the bufferinfo structure the consumer is going to make use of. This
|
||||
allows the exporter to simplify and/or raise an error if it can't support the operation.
|
||||
The third argument indicates what kind of buffer the consumer is
|
||||
prepared to deal with and therefore what kind of buffer the exporter
|
||||
is allowed to return. The new buffer interface allows for much more
|
||||
complicated memory sharing possibilities. Some consumers may not be
|
||||
able to handle all the complexibity but may want to see if the
|
||||
exporter will let them take a simpler view to its memory.
|
||||
|
||||
It also allows the caller to make a request for a simple "view" and
|
||||
receive it or have an error raised if it's not possible.
|
||||
In addition, some exporters may not be able to share memory in every
|
||||
possible way and may need to raise errors to signal to some consumers
|
||||
that something is just not possible. These errors should be
|
||||
PyErr_BufferError unless there is another error that is actually
|
||||
causing the problem. The exporter can use flags information to
|
||||
simplify how much of the PyBuffer structure is filled in with
|
||||
non-default values and/or raise an error if the object can't support a
|
||||
simpler view of its memory.
|
||||
|
||||
All of the following assume that at least buf, len, and readonly will be
|
||||
utilized by the caller.
|
||||
The exporter should always fill in all elements of the buffer
|
||||
structure (with defaults if nothing else is requested).
|
||||
|
||||
Py_BUF_SIMPLE
|
||||
The returned buffer will only be assumed to be readable (the object
|
||||
may or may not have writeable memory). Only the buf, len, and
|
||||
readonly variables may be accessed. The format will be
|
||||
assumed to be unsigned bytes. This is a "stand-alone" flag constant.
|
||||
It never needs to be \|'d to the others.
|
||||
Some flags are useful for requesting a specific kind of memory
|
||||
segment, while others indicate to the exporter what kind of
|
||||
information the consumer can deal with. If certain information is not
|
||||
asked for by the consumer, but the exporter cannot share its memory
|
||||
without that information, then a PyErr_BufferError should be raised.
|
||||
|
||||
Py_BUF_WRITEABLE
|
||||
The returned buffer must be writeable. If it cannot be, then raise an error.
|
||||
|
||||
Py_BUF_READONLY
|
||||
The returned buffer must be readonly and the underlying object should make
|
||||
its memory readonly if that is possible.
|
||||
PyBUF_SIMPLE
|
||||
|
||||
Py_BUF_FORMAT
|
||||
The consumer will be using the format string information so make sure that
|
||||
member is filled correctly.
|
||||
This is the default flag state (0). The returned buffer may or may
|
||||
not have writeable memory. The format will be assumed to be
|
||||
unsigned bytes . This is a "stand-alone" flag constant. It never
|
||||
needs to be |'d to the others. The exporter will raise an error if
|
||||
it cannot provide such a contiguous buffer of bytes.
|
||||
|
||||
Py_BUF_SHAPE
|
||||
The consumer can (and might) make use of using the ndims and shape members of the structure
|
||||
so make sure they are filled in correctly.
|
||||
PyBUF_REQ_WRITEABLE
|
||||
|
||||
The returned buffer must be writeable. If it is not writeable,
|
||||
then raise an error.
|
||||
|
||||
PyBUF_REQ_LOCKDATA
|
||||
|
||||
The returned buffer must be readonly. If the object is already
|
||||
read-only or it can make its memory read-only (and there are no
|
||||
other views on the object) then it should do so and return the
|
||||
buffer information. If the object does not have read-only memory
|
||||
(or cannot make it read-only), then an error should be raised.
|
||||
|
||||
PyBUF_REQ_FORMAT
|
||||
|
||||
The returned buffer must have true format information if this flag
|
||||
is provided. This would be used when the consumer is going to be
|
||||
checking for what 'kind' of data is actually stored. An exporter
|
||||
should always be able to provide this information if requested. If
|
||||
format is not explicitly requested then the format must be returned
|
||||
as NULL (which means "B")
|
||||
|
||||
PyBUF_REQ_ALIGNED (implies PyBUF_REQ_FORMAT)
|
||||
|
||||
The returned buffer must have all (primitive data-type entries)
|
||||
aligned as the compiler would align them
|
||||
|
||||
PyBUF_ALW_ND
|
||||
|
||||
The returned buffer must provide shape information. The memory will
|
||||
be assumed C-style contiguous (last dimension varies the fastest).
|
||||
The exporter may raise an error if it cannot provide this kind of
|
||||
contiguous buffer.
|
||||
|
||||
Py_BUF_STRIDES (implies SHAPE)
|
||||
The consumer can (and might) make use of the strides member of the structure (as well
|
||||
as ndims and shape)
|
||||
PyBUF_ALW_STRIDES (implies PyBUF_ALW_ND)
|
||||
|
||||
Py_BUF_OFFSETS (implies STRIDES)
|
||||
The consumer can (and might) make use of the suboffsets member (as well as
|
||||
ndims, shape, and strides)
|
||||
The returned buffer must provide strides information. This would be
|
||||
used when the consumer can handle strided, discontiguous arrays.
|
||||
Handling strides automatically assumes you can handle shape.
|
||||
The exporter may raise an error if cannot provide a strided-only
|
||||
representation of the data (i.e. without the suboffsets).
|
||||
|
||||
Thus, the consumer simply wanting an contiguous chunk of bytes from
|
||||
the object would use Py_BUF_SIMPLE, while a consumer that understands
|
||||
how to make use of the most complicated cases would use
|
||||
Py_BUF_OFFSETS.
|
||||
PyBUF_REQ_C_CONTIGUOUS
|
||||
PyBUF_REQ_F_CONTIGUOUS
|
||||
PyBUF_REQ_ANY_CONTIGUOUS
|
||||
|
||||
These flags indicate that the returned buffer must be respectively,
|
||||
C-contiguous (last dimension varies the fastest), Fortran
|
||||
contiguous (first dimension varies the fastest) or either one.
|
||||
All of these flags imply PyBUF_ALW_STRIDES and guarantee that the
|
||||
strides buffer info structure will be filled in correctly.
|
||||
|
||||
|
||||
PyBUF_ALW_INDIRECT (implies PyBUF_ALW_STRIDES)
|
||||
|
||||
The returned buffer must have suboffsets information. This would
|
||||
be used when the consumer can handle indirect array referencing
|
||||
implied by these suboffsets.
|
||||
|
||||
|
||||
Specialized combinations of flags for specific kinds of memory_sharing.
|
||||
|
||||
Multi-dimensional (but contiguous)
|
||||
|
||||
PyBUF_CONTIG (PyBUF_ALW_ND | PyBUF_REQ_WRITEABLE | PyBUF_REQ_ALIGNED)
|
||||
PyBUF_CONTIG_RO (PyBUF_ALW_ND | PyBUF_REQ_ALIGNED)
|
||||
PyBUF_CONTIG_LCK (PyBUF_ALW_ND | PyBUF_REQ_LOCKDATA | PyBUF_REQ_ALIGNED)
|
||||
|
||||
Multi-dimensional using strides but aligned
|
||||
|
||||
PyBUF_STRIDED (PyBUF_ALW_STRIDES | PyBUF_REQ_WRITEABLE | PyBUF_REQ_ALIGNED)
|
||||
PyBUF_STRIDED_RO (PyBUF_ALW_STRIDES | PyBUF_REQ_ALIGNED)
|
||||
PyBUF_STRIDED_LCK (PyBUF_ALW_STRIDES | PyBUF_REQ_LOCKDATA | PyBUF_REQ_ALIGNED)
|
||||
|
||||
Multi-dimensional using strides and not necessarily aligned
|
||||
|
||||
PyBUF_RECORDS (PyBUF_ALW_STRIDES | PyBUF_REQ_WRITEABLE | PyBUF_REQ_FORMAT)
|
||||
PyBUF_RECORDS_RO (PyBUF_ALW_STRIDES | PyBUF_REQ_FORMAT)
|
||||
PyBUF_RECORDS_LCK (PyBUF_ALW_STRIDES | PyBUF_REQ_LOCKDATA | PyBUF_REQ_FORMAT)
|
||||
|
||||
Multi-dimensional using sub-offsets
|
||||
|
||||
PyBUF_FULL (PyBUF_ALW_INDIRECT | PyBUF_REQ_WRITEABLE | PyBUF_REQ_FORMAT)
|
||||
PyBUF_FULL_RO (PyBUF_ALW_INDIRECT | PyBUF_REQ_FORMAT)
|
||||
PyBUF_FULL_LCK (PyBUF_ALW_INDIRECT | PyBUF_REQ_LOCKDATA | PyBUF_REQ_FORMAT)
|
||||
|
||||
Thus, the consumer simply wanting a contiguous chunk of bytes from
|
||||
the object would use PyBUF_SIMPLE, while a consumer that understands
|
||||
how to make use of the most complicated cases could use PyBUF_FULL
|
||||
|
||||
The format information is only guaranteed to be non-NULL if
|
||||
PyBUF_REQ_FORMAT is in the flag argument, otherwise it is expected the
|
||||
consumer will assume unsigned bytes.
|
||||
|
||||
There is a C-API that simple exporting objects can use to fill-in the
|
||||
buffer info structure correctly according to the provided flags if a
|
||||
contiguous chunk of memory is all that can be exported.
|
||||
contiguous chunk of "unsigned bytes" is all that can be exported.
|
||||
|
||||
|
||||
The bufferinfo structure is::
|
||||
|
@ -216,21 +300,23 @@ The bufferinfo structure is::
|
|||
void *buf;
|
||||
Py_ssize_t len;
|
||||
int readonly;
|
||||
char *format;
|
||||
int ndims;
|
||||
const char *format;
|
||||
int ndim;
|
||||
Py_ssize_t *shape;
|
||||
Py_ssize_t *strides;
|
||||
Py_ssize_t *suboffsets;
|
||||
int itemsize;
|
||||
void *internal;
|
||||
};
|
||||
} PyBuffer;
|
||||
|
||||
Before calling this function, the bufferinfo structure can be filled with
|
||||
whatever. Upon return from getbufferproc, the bufferinfo structure is filled in
|
||||
with relevant information about the buffer. This same bufferinfo
|
||||
structure must be passed to bf_releasebuffer (if available) when the
|
||||
consumer is done with the memory. The caller is responsible for
|
||||
keeping a reference to obj until releasebuffer is called (i.e. this
|
||||
call does not alter the reference count of obj).
|
||||
Before calling this function, the bufferinfo structure can be filled
|
||||
with whatever. Upon return from getbufferproc, the bufferinfo
|
||||
structure is filled in with relevant information about the buffer.
|
||||
This same bufferinfo structure must be passed to bf_releasebuffer (if
|
||||
available) when the consumer is done with the memory. The caller is
|
||||
responsible for keeping a reference to obj until releasebuffer is
|
||||
called (i.e. the call to bf_getbuffer does not alter the reference
|
||||
count of obj).
|
||||
|
||||
The members of the bufferinfo structure are:
|
||||
|
||||
|
@ -242,33 +328,39 @@ len
|
|||
same as the product of the shape array multiplied by the number of
|
||||
bytes per item of memory.
|
||||
|
||||
readonly
|
||||
an integer variable to hold whether or not the memory is
|
||||
readonly. 1 means the memory is readonly, zero means the
|
||||
memory is writeable.
|
||||
readonly
|
||||
an integer variable to hold whether or not the memory is readonly.
|
||||
1 means the memory is readonly, zero means the memory is
|
||||
writeable, and -1 means the memory was "locked" (changed from
|
||||
writeable to readonly) when this PyBuffer structure was filled-in
|
||||
and therefore should be unlocked when this PyBuffer structure is
|
||||
"released" (this is supported only by some exporters).
|
||||
|
||||
format
|
||||
a NULL-terminated format-string (following the struct-style syntax
|
||||
including extensions) indicating what is in each element of
|
||||
memory. The number of elements is len / itemsize, where itemsize
|
||||
is the number of bytes implied by the format. For standard
|
||||
unsigned bytes use a format string of "B".
|
||||
is the number of bytes implied by the format. This can be NULL which
|
||||
implies standard unsigned bytes ("B").
|
||||
|
||||
ndims
|
||||
a variable storing the number of dimensions the memory represents.
|
||||
Must be >=0.
|
||||
Must be >=0. A value of 0 means that shape and strides and suboffsets
|
||||
must be NULL (i.e. the memory represents a scalar).
|
||||
|
||||
shape
|
||||
an array of ``Py_ssize_t`` of length ``ndims`` indicating the
|
||||
shape of the memory as an N-D array. Note that ``((*shape)[0] *
|
||||
... * (*shape)[ndims-1])*itemsize = len``.
|
||||
... * (*shape)[ndims-1])*itemsize = len``. If ndims is 0 (indicating
|
||||
a scalar), then this must be NULL.
|
||||
|
||||
strides
|
||||
address of a ``Py_ssize_t*`` variable that will be filled with a
|
||||
pointer to an array of ``Py_ssize_t`` of length ``*ndims``
|
||||
indicating the number of bytes to skip to get to the next element
|
||||
in each dimension. For C-style contiguous arrays (where the
|
||||
last-dimension varies the fastest) this must be filled in.
|
||||
pointer to an array of ``Py_ssize_t`` of length ``ndims`` (or NULL
|
||||
if ndims is 0). indicating the number of bytes to skip to get to
|
||||
the next element in each dimension. If this is not requested by
|
||||
the caller (PyBUF_ALW_STRIDES is not set), then this should be set
|
||||
to NULL which indicates a C-style contiguous array.
|
||||
|
||||
suboffsets
|
||||
address of a ``Py_ssize_t *`` variable that will be filled with a
|
||||
|
@ -279,14 +371,14 @@ suboffsets
|
|||
suboffset value that it negative indicates that no de-referencing
|
||||
should occur (striding in a contiguous memory block). If all
|
||||
suboffsets are negative (i.e. no de-referencing is needed, then
|
||||
this must be NULL.
|
||||
this must be NULL (the default value).
|
||||
|
||||
For clarity, here is a function that returns a pointer to the
|
||||
element in an N-D array pointed to by an N-dimesional index when
|
||||
there are both strides and suboffsets.::
|
||||
there are both non-NULL strides and suboffsets.::
|
||||
|
||||
void* get_item_pointer(int ndim, void* buf, Py_ssize_t* strides,
|
||||
Py_ssize_t* suboffsets, Py_ssize_t *indices) {
|
||||
Py_ssize_t* suboffsets, Py_ssize_t *indices) {
|
||||
char* pointer = (char*)buf;
|
||||
int i;
|
||||
for (i = 0; i < ndim; i++) {
|
||||
|
@ -304,28 +396,36 @@ suboffsets
|
|||
the location of the starting pointer directly (i.e. buf would
|
||||
be modified).
|
||||
|
||||
itemsize
|
||||
This is a storage for the itemsize of each element of the shared
|
||||
memory. It is strictly un-necessary as it can be obtained using
|
||||
PyBuffer_SizeFromFormat, however an exporter may know this
|
||||
information without parsing the format string and it is necessary
|
||||
to know the itemsize for proper interpretation of striding.
|
||||
Therefore, storing it is more convenient and faster.
|
||||
|
||||
internal
|
||||
This is for use internally by the exporting object. For example,
|
||||
this might be re-cast as an integer by the exporter and used to
|
||||
store flags about whether or not the shape, strides, and suboffsets
|
||||
arrays must be freed when the buffer is released. The consumer
|
||||
should never touch this value.
|
||||
should never alter this value.
|
||||
|
||||
|
||||
The exporter is responsible for making sure the memory pointed to by
|
||||
buf, format, shape, strides, and suboffsets is valid until
|
||||
The exporter is responsible for making sure that any memory pointed to
|
||||
by buf, format, shape, strides, and suboffsets is valid until
|
||||
releasebuffer is called. If the exporter wants to be able to change
|
||||
shape, strides, and/or suboffsets before releasebuffer is called then
|
||||
it should allocate those arrays when getbuffer is called (pointing to
|
||||
them in the buffer-info structure provided) and free them when
|
||||
releasebuffer is called.
|
||||
an object's shape, strides, and/or suboffsets before releasebuffer is
|
||||
called then it should allocate those arrays when getbuffer is called
|
||||
(pointing to them in the buffer-info structure provided) and free them
|
||||
when releasebuffer is called.
|
||||
|
||||
|
||||
The same bufferinfo struct should be used in the release-buffer
|
||||
interface call. The caller is responsible for the memory of the
|
||||
bufferinfo structure itself.
|
||||
|
||||
``typedef int (*releasebufferproc)(PyObject *obj, struct bufferinfo *view)``
|
||||
``typedef int (*releasebufferproc)(PyObject *obj, PyBuffer *view)``
|
||||
Callers of getbufferproc must make sure that this function is
|
||||
called when memory previously acquired from the object is no
|
||||
longer needed. The exporter of the interface must make sure that
|
||||
|
@ -348,7 +448,8 @@ each object.
|
|||
|
||||
All that is specifically required by the exporter, however, is to
|
||||
ensure that any memory shared through the bufferinfo structure remains
|
||||
valid until releasebuffer is called on the bufferinfo structure.
|
||||
valid until releasebuffer is called on the bufferinfo structure
|
||||
exporting that memory.
|
||||
|
||||
|
||||
New C-API calls are proposed
|
||||
|
@ -362,69 +463,94 @@ Return 1 if the getbuffer function is available otherwise 0.
|
|||
|
||||
::
|
||||
|
||||
int PyObject_GetBuffer(PyObject *obj, struct bufferinfo *view, int flags)
|
||||
int PyObject_GetBuffer(PyObject *obj, PyBuffer *view,
|
||||
int flags)
|
||||
|
||||
This is a C-API version of the getbuffer function call. It checks to
|
||||
make sure object has the required function pointer and issues the
|
||||
call. Returns -1 and raises an error on failure and returns 0 on
|
||||
success.
|
||||
call. Returns -1 and raises an error on failure and returns 0 on
|
||||
success.
|
||||
|
||||
::
|
||||
int PyObject_ReleaseBuffer(PyObject *obj, PyBuffer *view)
|
||||
|
||||
int PyObject_ReleaseBuffer(PyObject *obj, struct bufferinfo *view)
|
||||
|
||||
This is a C-API version of the releasebuffer function call. It checks to
|
||||
make sure the object has the required function pointer and issues the call. Returns 0
|
||||
on success and -1 (with an error raised) on failure. This function always
|
||||
succeeds if there is no releasebuffer function for the object.
|
||||
This is a C-API version of the releasebuffer function call. It checks
|
||||
to make sure the object has the required function pointer and issues
|
||||
the call. Returns 0 on success and -1 (with an error raised) on
|
||||
failure. This function always succeeds if there is no releasebuffer
|
||||
function for the object.
|
||||
|
||||
::
|
||||
|
||||
PyObject *PyObject_GetMemoryView(PyObject *obj)
|
||||
|
||||
Return a memory-view object from an object that defines the buffer interface.
|
||||
If make_ro is non-zero then request that the memory is made read-only until
|
||||
release buffer is called.
|
||||
Return a memory-view object from an object that defines the buffer interface.
|
||||
|
||||
A memory-view object is an extended buffer object that should replace
|
||||
the buffer object in Python 3K. It's C-structure is::
|
||||
A memory-view object is an extended buffer object that could replace
|
||||
the buffer object (but doesn't have to as that could be kept as a
|
||||
simple 1-d memory-view object). It's C-structure is
|
||||
|
||||
::
|
||||
|
||||
typedef struct {
|
||||
PyObject_HEAD
|
||||
PyObject *base;
|
||||
struct bufferinfo view;
|
||||
int itemsize;
|
||||
int flags;
|
||||
int ndims;
|
||||
Py_ssize_t *starts; /* slice starts */
|
||||
Py_ssize_t *stops; /* slice stops */
|
||||
Py_ssize_t *steps; /* slice steps */
|
||||
} PyMemoryViewObject;
|
||||
|
||||
This is very similar to the current buffer object except offset has
|
||||
been removed because ptr can just be modified by offset and a single
|
||||
offset is not sufficient for the sub-offsets. Also the hash has been
|
||||
removed because using the buffer object as a hash even if it is
|
||||
read-only is rarely useful.
|
||||
This is functionally similar to the current buffer object except only
|
||||
a reference to base is kept. The actual memory for base must be
|
||||
re-grabbed using the buffer-protocol, whenever it is actually
|
||||
needed (after which it is immediately released).
|
||||
|
||||
Also, the format, ndims, shape, strides, and suboffsets have been
|
||||
added. These additions will allow multi-dimensional slicing of the
|
||||
memory-view object which can be added at some point. This object
|
||||
always owns it's own shape, strides, and suboffsets arrays and it's
|
||||
own format string, but always borrows the memory from the object
|
||||
pointed to by base.
|
||||
The getbuffer and releasebuffer for this object use the underlying
|
||||
base object (adjusted as needed using the slice information). If the
|
||||
number of dimensions of the base object (or the strides or the size)
|
||||
has changed when a new view is requested, then the getbuffer will
|
||||
trigger an error. Methods on the MemoryView object will allow
|
||||
manual locking and unlocking of the underlying buffer.
|
||||
|
||||
The itemsize is a convenience and specifies the number of bytes
|
||||
indicated by the format string if positive.
|
||||
This memory-view object will support mult-dimensional slicing and be
|
||||
the first object provided with Python to do so. Slices of the
|
||||
memory-view object are other memory-view objects. When an "element"
|
||||
from the memory-view is returned it is always a bytes object whose
|
||||
format should be interpreted by the format attribute of the memoryview
|
||||
object. The struct module can be used to "decode" the bytes in Python
|
||||
if desired.
|
||||
|
||||
This object never reallocates ptr, shape, strides, subboffsets or
|
||||
format and therefore does not need to keep track of how many views it
|
||||
has exported.
|
||||
The Python name will be
|
||||
|
||||
__builtin__.memory
|
||||
|
||||
Methods:
|
||||
|
||||
lock
|
||||
unlock
|
||||
__getitem__ (will support multi-dimensional slicing)
|
||||
__setitem__ (will support multi-dimensional slicing)
|
||||
tobytes (obtain a bytes-object of everything in the memory).
|
||||
tolist (obtain a "nested" list of the memory. Everything
|
||||
is interpreted into standard Python objects
|
||||
as the struct module unpack would do).
|
||||
|
||||
Attributes (taken from the memory of the base object)
|
||||
|
||||
format
|
||||
itemsize
|
||||
shape
|
||||
strides
|
||||
suboffsets
|
||||
size
|
||||
readonly
|
||||
ndim
|
||||
|
||||
It exports a view using the base object. It releases a view by
|
||||
releasing the view on the base object. Because, it will never
|
||||
re-allocate memory, it does not need to keep track of how many it has
|
||||
exported but simple reference counting will suffice.
|
||||
|
||||
::
|
||||
|
||||
int PyObject_SizeFromFormat(char *)
|
||||
int PyBuffer_SizeFromFormat(const char *)
|
||||
|
||||
Return the implied itemsize of the data-format area from a struct-style
|
||||
description.
|
||||
|
@ -432,65 +558,82 @@ description.
|
|||
::
|
||||
|
||||
int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t *len,
|
||||
int fortran)
|
||||
char **format, char fortran)
|
||||
|
||||
Return a contiguous chunk of memory representing the buffer. If a
|
||||
copy is made then return 1. If no copy was needed return 0. If an
|
||||
error occurred in probing the buffer interface, then return -1. The
|
||||
contiguous chunk of memory is pointed to by ``*buf`` and the length of
|
||||
that memory is ``*len``. If the object is multi-dimensional, then if
|
||||
fortran is 1, the first dimension of the underlying array will vary
|
||||
the fastest in the buffer. If fortran is 0, then the last dimension
|
||||
will vary the fastest (C-style contiguous). If fortran is -1, then it
|
||||
fortran is 'F', the first dimension of the underlying array will vary
|
||||
the fastest in the buffer. If fortran is 'C', then the last dimension
|
||||
will vary the fastest (C-style contiguous). If fortran is 'A', then it
|
||||
does not matter and you will get whatever the object decides is more
|
||||
efficient.
|
||||
efficient. If a copy is made, then the memory must be freed by calling
|
||||
``PyMem_Free``.
|
||||
|
||||
::
|
||||
|
||||
int PyObject_CopyToObject(PyObject *obj, void *buf, Py_ssize_t len,
|
||||
int fortran)
|
||||
char fortran)
|
||||
|
||||
Copy ``len`` bytes of data pointed to by the contiguous chunk of
|
||||
memory pointed to by ``buf`` into the buffer exported by obj. Return
|
||||
0 on success and return -1 and raise an error on failure. If the
|
||||
object does not have a writeable buffer, then an error is raised. If
|
||||
fortran is 1, then if the object is multi-dimensional, then the data
|
||||
fortran is 'F', then if the object is multi-dimensional, then the data
|
||||
will be copied into the array in Fortran-style (first dimension varies
|
||||
the fastest). If fortran is 0, then the data will be copied into the
|
||||
array in C-style (last dimension varies the fastest). If fortran is -1, then
|
||||
the fastest). If fortran is 'C', then the data will be copied into the
|
||||
array in C-style (last dimension varies the fastest). If fortran is 'A', then
|
||||
it does not matter and the copy will be made in whatever way is more
|
||||
efficient.
|
||||
|
||||
The last two C-API calls allow a standard way of getting data in and
|
||||
|
||||
These last three C-API calls allow a standard way of getting data in and
|
||||
out of Python objects into contiguous memory areas no matter how it is
|
||||
actually stored. These calls use the extended buffer interface to perform
|
||||
their work.
|
||||
|
||||
::
|
||||
|
||||
int PyObject_IsContiguous(struct bufferinfo *view, int fortran);
|
||||
int PyBuffer_IsContiguous(PyBuffer *view, char fortran);
|
||||
|
||||
Return 1 if the memory defined by the view object is C-style (fortran = 0)
|
||||
or Fortran-style (fortran = 1) contiguous. Return 0 otherwise.
|
||||
Return 1 if the memory defined by the view object is C-style (fortran
|
||||
= 'C') or Fortran-style (fortran = 'F') contiguous or either one
|
||||
(fortran = 'A'). Return 0 otherwise.
|
||||
|
||||
::
|
||||
int PyBuffer_IsAligned(PyBuffer *view);
|
||||
|
||||
Return 1 if the memory at all elements of the array implied by the
|
||||
view object is aligned
|
||||
|
||||
::
|
||||
|
||||
void PyObject_FillContiguousStrides(int *ndims, Py_ssize_t *shape,
|
||||
void PyBuffer_FillContiguousStrides(int *ndims, Py_ssize_t *shape,
|
||||
int itemsize,
|
||||
Py_ssize_t *strides, int fortran)
|
||||
Py_ssize_t *strides, char fortran)
|
||||
|
||||
Fill the strides array with byte-strides of a contiguous (C-style if
|
||||
fortran is 0 or Fortran-style if fortran is 1) array of the given
|
||||
shape with the given number of bytes per element.
|
||||
|
||||
::
|
||||
int PyBuffer_FillInfo(PyBuffer *view, void *buf,
|
||||
Py_ssize_t len, int readonly, int infoflags)
|
||||
|
||||
int PyObject_FillBufferInfo(struct bufferinfo *view, void *buf, Py_ssize_t len,
|
||||
int readonly, int infoflags)
|
||||
Fills in a buffer-info structure correctly for an exporter that can
|
||||
only share a contiguous chunk of memory of "unsigned bytes" of the
|
||||
given length. Returns 0 on success and -1 (with raising an error) on
|
||||
error.
|
||||
|
||||
Fills in a buffer-info structure correctly for an exporter that can only share
|
||||
a contiguous chunk of memory of "unsigned bytes" of the given length. Returns 0 on success
|
||||
and -1 (with raising an error) on error
|
||||
::
|
||||
PyErr_BufferError
|
||||
|
||||
A new error object for returning buffer errors which arise because an
|
||||
exporter cannot provide the kind of buffer that a consumer expects.
|
||||
This will also be raised when a consumer requests a buffer from an
|
||||
object that does not provide the protocol.
|
||||
|
||||
|
||||
Additions to the struct string-syntax
|
||||
|
@ -521,22 +664,29 @@ Character Description
|
|||
':name:' optional name of the preceeding element
|
||||
'X{}' pointer to a function (optional function
|
||||
signature inside {})
|
||||
' \n\t' ignored (allow better readability) -- this may already be true
|
||||
' \n\t' ignored (allow better readability)
|
||||
-- this may already be true
|
||||
================ ===========
|
||||
|
||||
The struct module will be changed to understand these as well and
|
||||
return appropriate Python objects on unpacking. Un-packing a
|
||||
return appropriate Python objects on unpacking. Unpacking a
|
||||
long-double will return a decimal object or a ctypes long-double.
|
||||
Unpacking 'u' or 'w' will return Python unicode. Unpacking a
|
||||
multi-dimensional array will return a list of lists. Un-packing a
|
||||
pointer will return a ctypes pointer object. Un-packing a bit will
|
||||
return a Python Bool. Spaces in the struct-string syntax will be
|
||||
ignored. Unpacking a named-object will return a Python class with
|
||||
attributes having those names.
|
||||
multi-dimensional array will return a list (of lists if >1d).
|
||||
Unpacking a pointer will return a ctypes pointer object. Unpacking a
|
||||
function pointer will return a ctypes call-object (perhaps). Unpacking
|
||||
a bit will return a Python Bool. White-space in the struct-string
|
||||
syntax will be ignored if it isn't already. Unpacking a named-object
|
||||
will return some kind of named-tuple-like object that acts like a
|
||||
tuple but whose entries can also be accessed by name. Unpacking a
|
||||
nested structure will return a nested tuple.
|
||||
|
||||
Endian-specification ('=','>','<') is also allowed inside the
|
||||
string so that it can change if needed. The previously-specified
|
||||
endian string is in force until changed. The default endian is '='.
|
||||
Endian-specification ('!', '@','=','>','<', '^') is also allowed
|
||||
inside the string so that it can change if needed. The
|
||||
previously-specified endian string is in force until changed. The
|
||||
default endian is '@' which means native data-types and alignment. If
|
||||
un-aligned, native data-types are requested, then the endian
|
||||
specification is '^'.
|
||||
|
||||
According to the struct-module, a number can preceed a character
|
||||
code to specify how many of that type there are. The
|
||||
|
@ -551,16 +701,21 @@ Examples of Data-Format Descriptions
|
|||
====================================
|
||||
|
||||
Here are some examples of C-structures and how they would be
|
||||
represented using the struct-style syntax:
|
||||
represented using the struct-style syntax.
|
||||
|
||||
<named> is the constructor for a named-tuple (not-specified yet).
|
||||
|
||||
float
|
||||
'f'
|
||||
'f' <--> Python float
|
||||
complex double
|
||||
'Zd'
|
||||
'Zd' <--> Python complex
|
||||
RGB Pixel data
|
||||
'BBB' or 'B:r: B:g: B:b:'
|
||||
'BBB' <--> (int, int, int)
|
||||
'B:r: B:g: B:b:' <--> <named>((int, int, int), ('r','g','b'))
|
||||
|
||||
Mixed endian (weird but possible)
|
||||
'>i:big: <i:little:'
|
||||
'>i:big: <i:little:' <--> <named>((int, int), ('big', 'little'))
|
||||
|
||||
Nested structure
|
||||
::
|
||||
|
||||
|
@ -586,7 +741,9 @@ Nested array
|
|||
int ival;
|
||||
double data[16*4];
|
||||
}
|
||||
'i:ival: (16,4)d:data:'
|
||||
"""i:ival:
|
||||
(16,4)d:data:
|
||||
"""
|
||||
|
||||
|
||||
Code to be affected
|
||||
|
@ -662,10 +819,12 @@ Examples
|
|||
=========
|
||||
|
||||
Ex. 1
|
||||
----------
|
||||
-----------
|
||||
|
||||
This example shows how an image object that uses contiguous lines might expose its buffer.::
|
||||
|
||||
::
|
||||
|
||||
struct rgba {
|
||||
unsigned char r, g, b, a;
|
||||
};
|
||||
|
@ -688,7 +847,9 @@ In order to access, say, the red value of the pixel at x=30, y=50, you'd use "li
|
|||
|
||||
So what does ImageObject's getbuffer do? Leaving error checking out::
|
||||
|
||||
int Image_getbuffer(PyObject *self, struct bufferinfo *view, int flags) {
|
||||
::
|
||||
|
||||
int Image_getbuffer(PyObject *self, PyBuffer *view, int flags) {
|
||||
|
||||
static Py_ssize_t suboffsets[2] = { -1, 0 };
|
||||
|
||||
|
@ -710,7 +871,7 @@ So what does ImageObject's getbuffer do? Leaving error checking out::
|
|||
}
|
||||
|
||||
|
||||
int Image_releasebuffer(PyObject *self, struct bufferinfo *view) {
|
||||
int Image_releasebuffer(PyObject *self, PyBuffer *view) {
|
||||
self->view_count--;
|
||||
return 0;
|
||||
}
|
||||
|
@ -721,33 +882,36 @@ Ex. 2
|
|||
|
||||
This example shows how an object that wants to expose a contiguous
|
||||
chunk of memory (which will never be re-allocated while the object is
|
||||
alive) would do that.::
|
||||
alive) would do that.
|
||||
|
||||
int myobject_getbuffer(PyObject *self, struct bufferinfo *view, int flags) {
|
||||
::
|
||||
|
||||
void *buf;
|
||||
Py_ssize_t len;
|
||||
int readonly=0;
|
||||
int myobject_getbuffer(PyObject *self, PyBuffer *view, int flags) {
|
||||
|
||||
void *buf;
|
||||
Py_ssize_t len;
|
||||
int readonly=0;
|
||||
|
||||
buf = /* Point to buffer */
|
||||
len = /* Set to size of buffer */
|
||||
readonly = /* Set to 1 if readonly */
|
||||
|
||||
return PyObject_FillBufferInfo(view, buf, len, readonly, flags);
|
||||
buf = /* Point to buffer */
|
||||
len = /* Set to size of buffer */
|
||||
readonly = /* Set to 1 if readonly */
|
||||
|
||||
return PyObject_FillBufferInfo(view, buf, len, readonly, flags);
|
||||
}
|
||||
|
||||
/* No releasebuffer is necessary because the memory will never
|
||||
be re-allocated so the locking mechanism is not needed
|
||||
*/
|
||||
/* No releasebuffer is necessary because the memory will never
|
||||
be re-allocated so the locking mechanism is not needed
|
||||
*/
|
||||
|
||||
Ex. 3
|
||||
-----------
|
||||
|
||||
A consumer that wants to only get a simple contiguous chunk of bytes
|
||||
from a Python object, obj would do the following::
|
||||
from a Python object, obj would do the following:
|
||||
|
||||
::
|
||||
|
||||
struct bufferinfo view;
|
||||
PyBuffer view;
|
||||
int ret;
|
||||
|
||||
if (PyObject_GetBuffer(obj, &view, Py_BUF_SIMPLE) < 0) {
|
||||
|
@ -767,7 +931,36 @@ from a Python object, obj would do the following::
|
|||
}
|
||||
|
||||
|
||||
Ex. 4
|
||||
-----------
|
||||
|
||||
A consumer that wants to be able to use any object's memory but is
|
||||
writing an algorithm that only handle contiguous memory could do the following:
|
||||
|
||||
::
|
||||
|
||||
void *buf;
|
||||
Py_ssize_t len;
|
||||
char *format;
|
||||
|
||||
if (PyObject_GetContiguous(obj, &buf, &len, &format, 0) < 0) {
|
||||
/* error return */
|
||||
}
|
||||
|
||||
/* process memory pointed to by buffer if format is correct */
|
||||
|
||||
/* Optional:
|
||||
|
||||
if, after processing, we want to copy data from buffer back
|
||||
into the the object
|
||||
|
||||
we could do
|
||||
*/
|
||||
|
||||
if (PyObject_CopyToObject(obj, buf, len, 0) < 0) {
|
||||
/* error return */
|
||||
}
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
|
Loading…
Reference in New Issue