diff --git a/pep-3118.txt b/pep-3118.txt index 786f95e78..97438b153 100644 --- a/pep-3118.txt +++ b/pep-3118.txt @@ -175,7 +175,8 @@ non-default values and/or raise an error if the object can't support a simpler view of its memory. The exporter should always fill in all elements of the buffer -structure (with defaults if nothing else is requested). +structure (with defaults or NULLs if nothing else is requested). The +PyBuffer_FillInfo function can be used for simple cases. Some flags are useful for requesting a specific kind of memory segment, while others indicate to the exporter what kind of @@ -201,9 +202,10 @@ without that information, then a ``PyErr_BufferError`` should be raised. The returned buffer must be readonly. If the object is already read-only or it can make its memory read-only (and there are no - other views on the object) then it should do so and return the - buffer information. If the object does not have read-only memory - (or cannot make it read-only), then an error should be raised. + other writeable views on the object) then it should do so and + return the buffer information. If the object does not have + read-only memory (or cannot make it read-only), then an error + should be raised. ``PyBUF_REQ_FORMAT`` @@ -214,25 +216,21 @@ without that information, then a ``PyErr_BufferError`` should be raised. format is not explicitly requested then the format must be returned as ``NULL`` (which means "B") -``PyBUF_REQ_ALIGNED`` (implies ``PyBUF_REQ_FORMAT``) - - The returned buffer must have all (primitive data-type entries) - aligned as the compiler would align them - ``PyBUF_ALW_ND`` The returned buffer must provide shape information. The memory will be assumed C-style contiguous (last dimension varies the fastest). The exporter may raise an error if it cannot provide this kind of - contiguous buffer. + contiguous buffer. If this is not given then shape will be NULL. ``PyBUF_ALW_STRIDES`` (implies ``PyBUF_ALW_ND``) - The returned buffer must provide strides information. This would be - used when the consumer can handle strided, discontiguous arrays. - Handling strides automatically assumes you can handle shape. - The exporter may raise an error if cannot provide a strided-only - representation of the data (i.e. without the suboffsets). + The returned buffer must provide strides information (i.e. the + strides cannot be NULL). This would be used when the consumer can + handle strided, discontiguous arrays. Handling strides + automatically assumes you can handle shape. The exporter may raise + an error if cannot provide a strided-only representation of the + data (i.e. without the suboffsets). | ``PyBUF_REQ_C_CONTIGUOUS`` | ``PyBUF_REQ_F_CONTIGUOUS`` @@ -247,24 +245,25 @@ without that information, then a ``PyErr_BufferError`` should be raised. ``PyBUF_ALW_INDIRECT`` (implies ``PyBUF_ALW_STRIDES``) - The returned buffer must have suboffsets information. This would - be used when the consumer can handle indirect array referencing - implied by these suboffsets. + The returned buffer must have suboffsets information (which can be + NULL if no suboffsets are needed). This would be used when the + consumer can handle indirect array referencing implied by these + suboffsets. Specialized combinations of flags for specific kinds of memory_sharing. Multi-dimensional (but contiguous) - | ``PyBUF_CONTIG`` (``PyBUF_ALW_ND | PyBUF_REQ_WRITEABLE | PyBUF_REQ_ALIGNED``) - | ``PyBUF_CONTIG_RO`` (``PyBUF_ALW_ND | PyBUF_REQ_ALIGNED``) - | ``PyBUF_CONTIG_LCK`` (``PyBUF_ALW_ND | PyBUF_REQ_LOCKDATA | PyBUF_REQ_ALIGNED``) + | ``PyBUF_CONTIG`` (``PyBUF_ALW_ND | PyBUF_REQ_WRITEABLE``) + | ``PyBUF_CONTIG_RO`` (``PyBUF_ALW_ND``) + | ``PyBUF_CONTIG_LCK`` (``PyBUF_ALW_ND | PyBUF_REQ_LOCKDATA``) Multi-dimensional using strides but aligned - | ``PyBUF_STRIDED`` (``PyBUF_ALW_STRIDES | PyBUF_REQ_WRITEABLE | PyBUF_REQ_ALIGNED``) - | ``PyBUF_STRIDED_RO`` (``PyBUF_ALW_STRIDES | PyBUF_REQ_ALIGNED``) - | ``PyBUF_STRIDED_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_REQ_LOCKDATA | PyBUF_REQ_ALIGNED``) + | ``PyBUF_STRIDED`` (``PyBUF_ALW_STRIDES | PyBUF_REQ_WRITEABLE``) + | ``PyBUF_STRIDED_RO`` (``PyBUF_ALW_STRIDES``) + | ``PyBUF_STRIDED_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_REQ_LOCKDATA``) Multi-dimensional using strides and not necessarily aligned @@ -290,7 +289,6 @@ There is a C-API that simple exporting objects can use to fill-in the buffer info structure correctly according to the provided flags if a contiguous chunk of "unsigned bytes" is all that can be exported. - The bufferinfo structure is:: struct bufferinfo { @@ -357,7 +355,8 @@ The members of the bufferinfo structure are: if ``ndims`` is 0). indicating the number of bytes to skip to get to the next element in each dimension. If this is not requested by the caller (``PyBUF_ALW_STRIDES`` is not set), then this should be set - to NULL which indicates a C-style contiguous array. + to NULL which indicates a C-style contiguous array or a + PyExc_BufferError raised if this is not possible. ``suboffsets`` address of a ``Py_ssize_t *`` variable that will be filled with a @@ -368,7 +367,9 @@ The members of the bufferinfo structure are: suboffset value that it negative indicates that no de-referencing should occur (striding in a contiguous memory block). If all suboffsets are negative (i.e. no de-referencing is needed, then - this must be NULL (the default value). + this must be NULL (the default value). If this is not requested + by the caller (PyBUF_ALW_INDIRECT is not set), then this should be + set to NULL or an PyExc_BufferError raised if this is not possible. For clarity, here is a function that returns a pointer to the element in an N-D array pointed to by an N-dimesional index when @@ -395,7 +396,7 @@ The members of the bufferinfo structure are: ``itemsize`` This is a storage for the itemsize of each element of the shared - memory. It is strictly un-necessary as it can be obtained using + memory. It is technically un-necessary as it can be obtained using ``PyBuffer_SizeFromFormat``, however an exporter may know this information without parsing the format string and it is necessary to know the itemsize for proper interpretation of striding. @@ -420,9 +421,9 @@ when releasebuffer is called. The same bufferinfo struct should be used in the release-buffer interface call. The caller is responsible for the memory of the -bufferinfo structure itself. +PyBuffer structure itself. -``typedef int (*releasebufferproc)(PyObject *obj, PyBuffer *view)`` +``typedef void (*releasebufferproc)(PyObject *obj, PyBuffer *view)`` Callers of getbufferproc must make sure that this function is called when memory previously acquired from the object is no longer needed. The exporter of the interface must make sure that @@ -469,13 +470,11 @@ call. Returns -1 and raises an error on failure and returns 0 on success. :: - - int PyObject_ReleaseBuffer(PyObject *obj, PyBuffer *view) + void PyObject_ReleaseBuffer(PyObject *obj, PyBuffer *view) This is a C-API version of the releasebuffer function call. It checks to make sure the object has the required function pointer and issues -the call. Returns 0 on success and -1 (with an error raised) on -failure. This function always succeeds if there is no releasebuffer +the call. This function always succeeds even if there is no releasebuffer function for the object. :: @@ -554,19 +553,24 @@ description. :: int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t *len, - char **format, char fortran) + char **format, int writeable, char fortran) Return a contiguous chunk of memory representing the buffer. If a copy is made then return 1. If no copy was needed return 0. If an error occurred in probing the buffer interface, then return -1. The contiguous chunk of memory is pointed to by ``*buf`` and the length of -that memory is ``*len``. If the object is multi-dimensional, then if -fortran is 'F', the first dimension of the underlying array will vary -the fastest in the buffer. If fortran is 'C', then the last dimension -will vary the fastest (C-style contiguous). If fortran is 'A', then it -does not matter and you will get whatever the object decides is more -efficient. If a copy is made, then the memory must be freed by calling -``PyMem_Free``. +that memory is ``*len``. If writeable is 1, then the returned memory +must be writeable or else an error is raised. Otherwise, the memory +may or may not be writeable. If the object is multi-dimensional, then +if fortran is 'F', the first dimension of the underlying array will +vary the fastest in the buffer. If fortran is 'C', then the last +dimension will vary the fastest (C-style contiguous). If fortran is +'A', then it does not matter and you will get whatever the object +decides is more efficient. If a copy is made, then the memory must be +freed by calling ``PyMem_Free``. + +PyObject_ReleaseBuffer must be called on obj after you are done with the +memory even if a copy is made. :: @@ -579,11 +583,13 @@ memory pointed to by ``buf`` into the buffer exported by obj. Return object does not have a writeable buffer, then an error is raised. If fortran is 'F', then if the object is multi-dimensional, then the data will be copied into the array in Fortran-style (first dimension varies -the fastest). If fortran is 'C', then the data will be copied into the -array in C-style (last dimension varies the fastest). If fortran is 'A', then -it does not matter and the copy will be made in whatever way is more -efficient. +the fastest). If fortran is 'C', then the data will be copied into +the array in C-style (last dimension varies the fastest). If fortran +is 'A', then it does not matter and the copy will be made in whatever +way is more efficient. +:: + int PyObject_CopyData(PyObject *dest, PyObject *src) These last three C-API calls allow a standard way of getting data in and out of Python objects into contiguous memory areas no matter how it is @@ -600,19 +606,12 @@ Return 1 if the memory defined by the view object is C-style (fortran :: - int PyBuffer_IsAligned(PyBuffer *view) - -Return 1 if the memory at all elements of the array implied by the -view object is aligned - -:: - - void PyBuffer_FillContiguousStrides(int *ndims, Py_ssize_t *shape, - int itemsize, - Py_ssize_t *strides, char fortran) + void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, + Py_ssize_t *strides, int itemsize, + char fortran) Fill the strides array with byte-strides of a contiguous (C-style if -fortran is 0 or Fortran-style if fortran is 1) array of the given +fortran is 'C' or Fortran-style if fortran is 'F' array of the given shape with the given number of bytes per element. :: @@ -627,7 +626,7 @@ error. :: - PyErr_BufferError + PyExc_BufferError A new error object for returning buffer errors which arise because an exporter cannot provide the kind of buffer that a consumer expects. @@ -754,6 +753,7 @@ buffer interface will be modified. Here is a partial list. * buffer object * bytes object * string object +* unicode object * array module * struct module * mmap module @@ -847,7 +847,7 @@ So what does ImageObject's getbuffer do? Leaving error checking out:: int Image_getbuffer(PyObject *self, PyBuffer *view, int flags) { - static Py_ssize_t suboffsets[2] = { -1, 0 }; + static Py_ssize_t suboffsets[2] = { 0, -1}; view->buf = self->lines; view->len = self->height*self->width; @@ -938,8 +938,10 @@ writing an algorithm that only handle contiguous memory could do the following: void *buf; Py_ssize_t len; char *format; + int copy; - if (PyObject_GetContiguous(obj, &buf, &len, &format, 0) < 0) { + copy = PyObject_GetContiguous(obj, &buf, &len, &format, 0, 'A'); + if (copy < 0) { /* error return */ } @@ -953,9 +955,12 @@ writing an algorithm that only handle contiguous memory could do the following: we could do */ - if (PyObject_CopyToObject(obj, buf, len, 0) < 0) { + if (PyObject_CopyToObject(obj, buf, len, 'A') < 0) { /* error return */ } + + /* Make sure that if a copy was made, the memory is freed */ + if (copy == 1) PyMem_Free(buf); Copyright