iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove locking scheme from PEP 3118 buffer protocol. See issue 3046 in python bug tracker.

This commit is contained in:
Travis E. Oliphant 2008-06-06 16:33:50 +00:00
parent 351a920ce3
commit 617bc3a168
1 changed files with 52 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

137
pep-3118.txt
View File

@ -147,16 +147,16 @@ Change the ``PyBufferProcs`` structure to ::
releasebufferproc bf_releasebuffer;
} PyBufferProcs;
Both of these routines are optional for a type object
::
typedef int (*getbufferproc)(PyObject *obj, PyBuffer *view, int flags)
This function returns ``0`` on success and ``-1`` on failure (and raises an
error). The first variable is the "exporting" object. The second
argument is the address to a bufferinfo structure. If view is ``NULL``,
then no information is returned but a lock on the memory is still
obtained. In this case, the corresponding releasebuffer should also
be called with ``NULL``.
argument is the address to a bufferinfo structure. Both arguments must
never be NULL.
The third argument indicates what kind of buffer the consumer is
prepared to deal with and therefore what kind of buffer the exporter
@ -178,13 +178,16 @@ The exporter should always fill in all elements of the buffer
structure (with defaults or NULLs if nothing else is requested). The
PyBuffer_FillInfo function can be used for simple cases.
Access flags
------------
Some flags are useful for requesting a specific kind of memory
segment, while others indicate to the exporter what kind of
information the consumer can deal with. If certain information is not
asked for by the consumer, but the exporter cannot share its memory
without that information, then a ``PyErr_BufferError`` should be raised.
``PyBUF_SIMPLE``
This is the default flag state (0). The returned buffer may or may
@ -198,29 +201,6 @@ without that information, then a ``PyErr_BufferError`` should be raised.
The returned buffer must be writable. If it is not writable,
then raise an error.
``PyBUF_LOCK``
This flag is used to request a lock (either a read-lock or an
exclusive write lock) on the data-area of the object before the
buffer is returned. If the lock cannot be obtained, then an error
should be raised. In conjunction with the PyBUF_WRITABLE flag, the
PyBUF_LOCK flag creates four different access modes on the
data-area of the buffer memory: read, read-with-write-lock, write,
and exclusive write. The access modes are
================ ================= ==========================
flags Description Other Requests Can
================ ================= ==========================
neither read read and write
WRITABLE write read and write
LOCK locked read read but not write
WRITABLE | LOCK exclusive write neither read nor write
================ ================= ==========================
Care should be taken not to LOCK the buffer unless it is really
necessary (especially the exclusive write lock) as it makes the
object unable to share its memory until the lock is released.
``PyBUF_FORMAT``
The returned buffer must have true format information if this flag
@ -256,7 +236,6 @@ without that information, then a ``PyErr_BufferError`` should be raised.
All of these flags imply PyBUF_STRIDES and guarantee that the
strides buffer info structure will be filled in correctly.
``PyBUF_INDIRECT`` (implies ``PyBUF_STRIDES``)
The returned buffer must have suboffsets information (which can be
@ -271,29 +250,21 @@ Specialized combinations of flags for specific kinds of memory_sharing.
| ``PyBUF_CONTIG`` (``PyBUF_ND | PyBUF_WRITABLE``)
| ``PyBUF_CONTIG_RO`` (``PyBUF_ND``)
| ``PyBUF_CONTIG_LCK`` (``PyBUF_ND | PyBUF_LOCK``)
| ``PyBUF_CONTIG_XLCK`` (``PyBUF_ND | PyBUF_WRITABLE | PyBUF_LOCK``)
Multi-dimensional using strides but aligned
| ``PyBUF_STRIDED`` (``PyBUF_STRIDES | PyBUF_WRITABLE``)
| ``PyBUF_STRIDED_RO`` (``PyBUF_STRIDES``)
| ``PyBUF_STRIDED_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK``)
| ``PyBUF_STRIDED_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_WRITABLE``)
Multi-dimensional using strides and not necessarily aligned
| ``PyBUF_RECORDS`` (``PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT``)
| ``PyBUF_RECORDS_RO`` (``PyBUF_STRIDES | PyBUF_FORMAT``)
| ``PyBUF_RECORDS_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT``)
| ``PyBUF_RECORDS_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
Multi-dimensional using sub-offsets
| ``PyBUF_FULL`` (``PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT``)
| ``PyBUF_FULL_RO`` (``PyBUF_INDIRECT | PyBUF_FORMAT``)
| ``PyBUF_FULL_LCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT``)
| ``PyBUF_FULL_XLCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
Thus, the consumer simply wanting a contiguous chunk of bytes from
the object would use ``PyBUF_SIMPLE``, while a consumer that understands
@ -307,6 +278,10 @@ There is a C-API that simple exporting objects can use to fill-in the
buffer info structure correctly according to the provided flags if a
contiguous chunk of "unsigned bytes" is all that can be exported.
The Py_buffer struct
--------------------
The bufferinfo structure is::
struct bufferinfo {
@ -322,14 +297,15 @@ The bufferinfo structure is::
void *internal;
} Py_buffer;
Before calling the bf_getbuffer function, the bufferinfo structure can be
filled with whatever. Upon return from bf_getbuffer, the bufferinfo
structure is filled in with relevant information about the buffer.
This same bufferinfo structure must be passed to bf_releasebuffer (if
available) when the consumer is done with the memory. The caller is
responsible for keeping a reference to obj until releasebuffer is
called (i.e. the call to bf_getbuffer does not alter the reference
count of obj).
Before calling the bf_getbuffer function, the bufferinfo structure can
be filled with whatever, but the ``buf`` field must be NULL when
requesting a new buffer. Upon return from bf_getbuffer, the
bufferinfo structure is filled in with relevant information about the
buffer. This same bufferinfo structure must be passed to
bf_releasebuffer (if available) when the consumer is done with the
memory. The caller is responsible for keeping a reference to obj until
releasebuffer is called (i.e. the call to bf_getbuffer does not alter
the reference count of obj).
The members of the bufferinfo structure are:
@ -343,14 +319,7 @@ The members of the bufferinfo structure are:
``readonly``
an integer variable to hold whether or not the memory is readonly.
1 means the memory is readonly, zero means the memory is writable,
-1 means the memory was read "locked" when this Py_buffer
structure was filled-in therefore should be unlocked when this
Py_buffer structure is "released." A -2 means this Py_buffer
structure has an exclusive-write lock on the memory. This should
be unlocked when the Py_buffer structure is released. The concept
of locking is not supported by all objects that expose the buffer
protocol.
1 means the memory is readonly, zero means the memory is writable.
``format``
a NULL-terminated format-string (following the struct-style syntax
@ -359,7 +328,7 @@ The members of the bufferinfo structure are:
is the number of bytes implied by the format. This can be NULL which
implies standard unsigned bytes ("B").
``ndims``
``ndim``
a variable storing the number of dimensions the memory represents.
Must be >=0. A value of 0 means that shape and strides and suboffsets
must be ``NULL`` (i.e. the memory represents a scalar).
@ -440,21 +409,25 @@ called then it should allocate those arrays when getbuffer is called
when releasebuffer is called.
Releasing the buffer
--------------------
The same bufferinfo struct should be used in the release-buffer
interface call. The caller is responsible for the memory of the
Py_buffer structure itself.
interface call. The caller is responsible for the memory of the
Py_buffer structure itself.
``typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)``
Callers of getbufferproc must make sure that this function is
called when memory previously acquired from the object is no
longer needed. The exporter of the interface must make sure that
any memory pointed to in the bufferinfo structure remains valid
until releasebuffer is called.
::
Both of these routines are optional for a type object
typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)
If the bf_releasebuffer function is not provided (i.e. it is NULL),
then it does not ever need to be called.
Callers of getbufferproc must make sure that this function is called
when memory previously acquired from the object is no longer needed.
The exporter of the interface must make sure that any memory pointed
to in the bufferinfo structure remains valid until releasebuffer is
called.
If the bf_releasebuffer function is not provided (i.e. it is NULL),
then it does not ever need to be called.
Exporters will need to define a bf_releasebuffer function if they can
re-allocate their memory, strides, shape, suboffsets, or format
@ -520,10 +493,6 @@ reference to base is kept and the memory view is not re-grabbed.
Thus, this memory view object holds on to the memory of base until it
is deleted.
The bf_getbuffer and bf_releasebuffer for this object increments and
decrements the lock on base, but passes on the contents of view to the
caller.
This memory-view object will support multi-dimensional slicing and be
the first object provided with Python to do so. Slices of the
memory-view object are other memory-view objects with the same base
@ -571,7 +540,7 @@ description.
::
PyObject * PyMemoryView_GetContiguous(PyObject *obj, int buffertype,
char fort)
char fortran)
Return a memoryview object to a contiguous chunk of memory represented
by obj. If a copy must be made (because the memory pointed to by obj
@ -586,9 +555,8 @@ contiguous an error will be raised. In this circumstance, the user
can use PyBUF_UPDATEIFCOPY to ensure that a a writable temporary
contiguous buffer is returned. The contents of this contiguous buffer
will be copied back into the original object after the memoryview
object is deleted as long as the original object is writable and
allows applying a read-write lock. If this is not allowed by
the original object, then a BufferError is raised.
object is deleted as long as the original object is writable. If this
is not allowed by the original object, then a BufferError is raised.
If the object is multi-dimensional, then if fortran is 'F', the first
dimension of the underlying array will vary the fastest in the buffer.
@ -597,9 +565,7 @@ If fortran is 'C', then the last dimension will vary the fastest
you will get whatever the object decides is more efficient. If a copy
is made, then the memory must be freed by calling ``PyMem_Free``.
You receive a new reference to the memoryview object which will also
hold a lock on the objects data area (this lock will be released when
the memoryview object is deleted).
You receive a new reference to the memoryview object.
::
@ -816,12 +782,15 @@ It is intended that this PEP will be back-ported to Python 2.6 by
adding the C-API and the two functions to the existing buffer
protocol.
The proposed locking mechanism relies entirely on the exporter object
to not invalidate any of the memory pointed to by the buffer structure
until a corresponding releasebuffer is called. If it wants to be able
to change its own shape and/or strides arrays, then it needs to create
memory for these in the bufferinfo structure and copy information
over.
Previous versions of this PEP proposed a read/write locking scheme,
but it was later perceived as a) too complicated for common simple use
cases that do not require any locking and b) too simple for use cases
that required concurrent read/write access to a buffer with changing,
short-living locks. It is therefore left to users to implement their
own specific locking scheme around buffer objects if they require
consistent views across concurrent read/write access. A future PEP
may be proposed which includes a separate locking API after some
experience with these user-schemes is obtained
The sharing of strided memory and suboffsets is new and can be seen as
a modification of the multiple-segment interface. It is motivated by
@ -860,8 +829,6 @@ The authors of the PEP promise to contribute and maintain the code for
this proposal but will welcome any help.
Examples
========
@ -943,7 +910,7 @@ alive) would do that.
}
/* No releasebuffer is necessary because the memory will never
be re-allocated so the locking mechanism is not needed
be re-allocated
*/
Ex. 3