diff --git a/pep-3118.txt b/pep-3118.txt index 0019040ae..cfbd2d4ad 100644 --- a/pep-3118.txt +++ b/pep-3118.txt @@ -87,7 +87,7 @@ has issues: NumPy's strided memory model is used more often in computational libraries and because it is so simple it makes sense to support memory sharing using this model. The PIL memory model is sometimes - used in C-code where a 2-d array can be then accessed using double + used in C-code where a 2-d array can then be accessed using double pointer indirection: e.g. ``image[i][j]``. The buffer interface should allow the object to export either of these @@ -188,31 +188,38 @@ without that information, then a ``PyErr_BufferError`` should be raised. ``PyBUF_SIMPLE`` This is the default flag state (0). The returned buffer may or may - not have writeable memory. The format will be assumed to be + not have writable memory. The format will be assumed to be unsigned bytes . This is a "stand-alone" flag constant. It never needs to be \|'d to the others. The exporter will raise an error if it cannot provide such a contiguous buffer of bytes. -``PyBUF_CHARACTER`` +``PyBUF_WRITABLE`` - This essentially replaces the separate function for getting a character - buffer (which is useful in at least the unicode type). If an object - should do something different when it is requested as a character - buffer then it should detect this flag and respond differently. - -``PyBUF_WRITEABLE`` - - The returned buffer must be writeable. If it is not writeable, + The returned buffer must be writable. If it is not writable, then raise an error. -``PyBUF_LOCKDATA`` +``PyBUF_LOCK`` - The returned buffer must be readonly. If the object is already - read-only or it can make its memory read-only (and there are no - other writeable views on the object) then it should do so and - return the buffer information. If the object does not have - read-only memory (or cannot make it read-only), then an error - should be raised. + This flag is used to request a lock (either a read-lock or an + exclusive write lock) on the data-area of the object before the + buffer is returned. If the lock cannot be obtained, then an error + should be raised. In conjunction with the PyBUF_WRITABLE flag, the + PyBUF_LOCK flag creates four different access modes on the + data-area of the buffer memory: read, read-with-write-lock, write, + and exclusive write. The access modes are + + ================ ================= ========================== + flags Description Other Requests Can + ================ ================= ========================== + neither read read and write + WRITABLE write read and write + LOCK locked read read but not write + WRITABLE | LOCK exclusive write neither read nor write + ================ ================= ========================== + + Care should be taken not to LOCK the buffer unless it is really + necessary (especially the exlcusive write lock) as it makes the + object unable to share its memory until the lock is released. ``PyBUF_FORMAT`` @@ -223,14 +230,14 @@ without that information, then a ``PyErr_BufferError`` should be raised. format is not explicitly requested then the format must be returned as ``NULL`` (which means "B") -``PyBUF_ALW_ND`` +``PyBUF_ND`` The returned buffer must provide shape information. The memory will be assumed C-style contiguous (last dimension varies the fastest). The exporter may raise an error if it cannot provide this kind of contiguous buffer. If this is not given then shape will be NULL. -``PyBUF_ALW_STRIDES`` (implies ``PyBUF_ALW_ND``) +``PyBUF_STRIDES`` (implies ``PyBUF_ND``) The returned buffer must provide strides information (i.e. the strides cannot be NULL). This would be used when the consumer can @@ -246,11 +253,11 @@ without that information, then a ``PyErr_BufferError`` should be raised. These flags indicate that the returned buffer must be respectively, C-contiguous (last dimension varies the fastest), Fortran contiguous (first dimension varies the fastest) or either one. - All of these flags imply PyBUF_ALW_STRIDES and guarantee that the + All of these flags imply PyBUF_STRIDES and guarantee that the strides buffer info structure will be filled in correctly. -``PyBUF_ALW_INDIRECT`` (implies ``PyBUF_ALW_STRIDES``) +``PyBUF_INDIRECT`` (implies ``PyBUF_STRIDES``) The returned buffer must have suboffsets information (which can be NULL if no suboffsets are needed). This would be used when the @@ -262,27 +269,31 @@ Specialized combinations of flags for specific kinds of memory_sharing. Multi-dimensional (but contiguous) - | ``PyBUF_CONTIG`` (``PyBUF_ALW_ND | PyBUF_WRITEABLE``) - | ``PyBUF_CONTIG_RO`` (``PyBUF_ALW_ND``) - | ``PyBUF_CONTIG_LCK`` (``PyBUF_ALW_ND | PyBUF_LOCKDATA``) + | ``PyBUF_CONTIG`` (``PyBUF_ND | PyBUF_WRITABLE``) + | ``PyBUF_CONTIG_RO`` (``PyBUF_ND``) + | ``PyBUF_CONTIG_LCK`` (``PyBUF_ND | PyBUF_LOCK``) + | ``PyBUF_CONTIG_XLCK`` (``PyBUF_ND | PyBUF_WRITABLE | PyBUF_LOCK``) Multi-dimensional using strides but aligned - | ``PyBUF_STRIDED`` (``PyBUF_ALW_STRIDES | PyBUF_WRITEABLE``) - | ``PyBUF_STRIDED_RO`` (``PyBUF_ALW_STRIDES``) - | ``PyBUF_STRIDED_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_LOCKDATA``) + | ``PyBUF_STRIDED`` (``PyBUF_STRIDES | PyBUF_WRITABLE``) + | ``PyBUF_STRIDED_RO`` (``PyBUF_STRIDES``) + | ``PyBUF_STRIDED_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK``) + | ``PyBUF_STRIDED_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_WRITABLE``) Multi-dimensional using strides and not necessarily aligned - | ``PyBUF_RECORDS`` (``PyBUF_ALW_STRIDES | PyBUF_WRITEABLE | PyBUF_FORMAT``) - | ``PyBUF_RECORDS_RO`` (``PyBUF_ALW_STRIDES | PyBUF_FORMAT``) - | ``PyBUF_RECORDS_LCK`` (``PyBUF_ALW_STRIDES | PyBUF_LOCKDATA | PyBUF_FORMAT``) + | ``PyBUF_RECORDS`` (``PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT``) + | ``PyBUF_RECORDS_RO`` (``PyBUF_STRIDES | PyBUF_FORMAT``) + | ``PyBUF_RECORDS_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT``) + | ``PyBUF_RECORDS_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``) Multi-dimensional using sub-offsets - | ``PyBUF_FULL`` (``PyBUF_ALW_INDIRECT | PyBUF_WRITEABLE | PyBUF_FORMAT``) - | ``PyBUF_FULL_RO`` (``PyBUF_ALW_INDIRECT | PyBUF_FORMAT``) - | ``PyBUF_FULL_LCK`` (``PyBUF_ALW_INDIRECT | PyBUF_LOCKDATA | PyBUF_FORMAT``) + | ``PyBUF_FULL`` (``PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT``) + | ``PyBUF_FULL_RO`` (``PyBUF_INDIRECT | PyBUF_FORMAT``) + | ``PyBUF_FULL_LCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT``) + | ``PyBUF_FULL_XLCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``) Thus, the consumer simply wanting a contiguous chunk of bytes from the object would use ``PyBUF_SIMPLE``, while a consumer that understands @@ -309,7 +320,7 @@ The bufferinfo structure is:: Py_ssize_t *suboffsets; Py_ssize_t itemsize; void *internal; - } PyBuffer; + } Py_buffer; Before calling this function, the bufferinfo structure can be filled with whatever. Upon return from getbufferproc, the bufferinfo @@ -332,11 +343,14 @@ The members of the bufferinfo structure are: ``readonly`` an integer variable to hold whether or not the memory is readonly. - 1 means the memory is readonly, zero means the memory is - writeable, and -1 means the memory was "locked" (changed from - writeable to readonly) when this PyBuffer structure was filled-in - and therefore should be unlocked when this PyBuffer structure is - "released" (this is supported only by some exporters). + 1 means the memory is readonly, zero means the memory is writable, + -1 means the memory was "locked" (changed from writable to + readonly) when this Py_buffer structure was filled-in after a + readable request and therefore should be unlocked when this + Py_buffer structure is "released" (this is supported only by some + exporters). A -2 means this Py_buffer structure has an + exclusive-write lock on the memory. This should be unlocked when + the Py_buffer structure is released. ``format`` a NULL-terminated format-string (following the struct-style syntax @@ -361,7 +375,7 @@ The members of the bufferinfo structure are: pointer to an array of ``Py_ssize_t`` of length ``ndims`` (or ``NULL`` if ``ndims`` is 0). indicating the number of bytes to skip to get to the next element in each dimension. If this is not requested by - the caller (``PyBUF_ALW_STRIDES`` is not set), then this should be set + the caller (``PyBUF_STRIDES`` is not set), then this should be set to NULL which indicates a C-style contiguous array or a PyExc_BufferError raised if this is not possible. @@ -375,7 +389,7 @@ The members of the bufferinfo structure are: should occur (striding in a contiguous memory block). If all suboffsets are negative (i.e. no de-referencing is needed, then this must be NULL (the default value). If this is not requested - by the caller (PyBUF_ALW_INDIRECT is not set), then this should be + by the caller (PyBUF_INDIRECT is not set), then this should be set to NULL or an PyExc_BufferError raised if this is not possible. For clarity, here is a function that returns a pointer to the @@ -428,9 +442,9 @@ when releasebuffer is called. The same bufferinfo struct should be used in the release-buffer interface call. The caller is responsible for the memory of the -PyBuffer structure itself. +Py_buffer structure itself. -``typedef void (*releasebufferproc)(PyObject *obj, PyBuffer *view)`` +``typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)`` Callers of getbufferproc must make sure that this function is called when memory previously acquired from the object is no longer needed. The exporter of the interface must make sure that @@ -468,7 +482,7 @@ Return 1 if the getbuffer function is available otherwise 0. :: - int PyObject_GetBuffer(PyObject *obj, PyBuffer *view, + int PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags) This is a C-API version of the getbuffer function call. It checks to @@ -478,7 +492,7 @@ success. :: - void PyObject_ReleaseBuffer(PyObject *obj, PyBuffer *view) + void PyObject_ReleaseBuffer(PyObject *obj, Py_buffer *view) This is a C-API version of the releasebuffer function call. It checks to make sure the object has the required function pointer and issues @@ -498,7 +512,7 @@ simple 1-d memory-view object). Its C-structure is :: typedef struct { PyObject_HEAD PyObject *base; - PyBuffer view; + Py_buffer view; } PyMemoryViewObject; This is functionally similar to the current buffer object except a @@ -566,14 +580,14 @@ the base object for the returned memory view object. The buffertype argument can be PyBUF_READ, PyBUF_WRITE, PyBUF_UPDATEIFCOPY to determine whether the returned buffer should be -readable, writeable, or set to update the original buffer if a copy +readable, writable, or set to update the original buffer if a copy must be made. If buffertype is PyBUF_WRITE and the buffer is not contiguous an error will be raised. In this circumstance, the user -can use PyBUF_UPDATEIFCOPY to ensure that a a writeable temporary +can use PyBUF_UPDATEIFCOPY to ensure that a a writable temporary contiguous buffer is returned. The contents of this contiguous buffer will be copied back into the original object after the memoryview -object is deleted as long as the original object is writeable and -allows setting its memory to "readonly". If this is not allowed by +object is deleted as long as the original object is writable and +allows applying a read-write lock. If this is not allowed by the original object, then a BufferError is raised. If the object is multi-dimensional, then if fortran is 'F', the first @@ -595,7 +609,7 @@ the memoryview object is deleted). Copy ``len`` bytes of data pointed to by the contiguous chunk of memory pointed to by ``buf`` into the buffer exported by obj. Return 0 on success and return -1 and raise an error on failure. If the -object does not have a writeable buffer, then an error is raised. If +object does not have a writable buffer, then an error is raised. If fortran is 'F', then if the object is multi-dimensional, then the data will be copied into the array in Fortran-style (first dimension varies the fastest). If fortran is 'C', then the data will be copied into @@ -614,7 +628,7 @@ their work. :: - int PyBuffer_IsContiguous(PyBuffer *view, char fortran) + int PyBuffer_IsContiguous(Py_buffer *view, char fortran) Return 1 if the memory defined by the view object is C-style (fortran = 'C') or Fortran-style (fortran = 'F') contiguous or either one @@ -632,7 +646,7 @@ shape with the given number of bytes per element. :: - int PyBuffer_FillInfo(PyBuffer *view, void *buf, + int PyBuffer_FillInfo(Py_buffer *view, void *buf, Py_ssize_t len, int readonly, int infoflags) Fills in a buffer-info structure correctly for an exporter that can @@ -679,8 +693,6 @@ Character Description 'X{}' pointer to a function (optional function signature inside {} with any return value preceeded by -> and placed at the end) -' ', '\\n', \\t' ignored (allow better readability) - -- this may already be true ================ =========== The struct module will be changed to understand these as well and @@ -862,7 +874,7 @@ In order to access, say, the red value of the pixel at x=30, y=50, you'd use "li So what does ImageObject's getbuffer do? Leaving error checking out:: - int Image_getbuffer(PyObject *self, PyBuffer *view, int flags) { + int Image_getbuffer(PyObject *self, Py_buffer *view, int flags) { static Py_ssize_t suboffsets[2] = { 0, -1}; @@ -884,7 +896,7 @@ So what does ImageObject's getbuffer do? Leaving error checking out:: } - int Image_releasebuffer(PyObject *self, PyBuffer *view) { + int Image_releasebuffer(PyObject *self, Py_buffer *view) { self->view_count--; return 0; } @@ -899,7 +911,7 @@ alive) would do that. :: - int myobject_getbuffer(PyObject *self, PyBuffer *view, int flags) { + int myobject_getbuffer(PyObject *self, Py_buffer *view, int flags) { void *buf; Py_ssize_t len; @@ -924,7 +936,7 @@ from a Python object, obj would do the following: :: - PyBuffer view; + Py_buffer view; int ret; if (PyObject_GetBuffer(obj, &view, Py_BUF_SIMPLE) < 0) {