175 lines
5.5 KiB
Plaintext
175 lines
5.5 KiB
Plaintext
PEP: 298
|
||
Title: The Fixed Buffer Interface
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Thomas Heller <theller@python.net>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Created: 26-Jul-2002
|
||
Python-Version: 2.3
|
||
Post-History:
|
||
|
||
|
||
Abstract
|
||
|
||
This PEP proposes an extension to the buffer interface called the
|
||
'fixed buffer interface'.
|
||
|
||
The fixed buffer interface fixes the flaws of the 'old' buffer
|
||
interface as defined in Python versions up to and including 2.2,
|
||
see [1]:
|
||
|
||
The lifetime of the retrieved pointer is clearly defined.
|
||
|
||
The buffer size is returned as a 'size_t' data type, which
|
||
allows access to large buffers on platforms where sizeof(int)
|
||
!= sizeof(void *).
|
||
|
||
|
||
Specification
|
||
|
||
The fixed buffer interface exposes new functions which return the
|
||
size and the pointer to the internal memory block of any python
|
||
object which chooses to implement this interface.
|
||
|
||
The size and pointer returned must be valid as long as the object
|
||
is alive (has a positive reference count). So, only objects which
|
||
never reallocate or resize the memory block are allowed to
|
||
implement this interface.
|
||
|
||
The fixed buffer interface omits the memory segment model which is
|
||
present in the old buffer interface - only a single memory block
|
||
can be exposed.
|
||
|
||
|
||
Implementation
|
||
|
||
Define a new flag in Include/object.h:
|
||
|
||
/* PyBufferProcs contains bf_getfixedreadbuffer
|
||
and bf_getfixedwritebuffer */
|
||
#define Py_TPFLAGS_HAVE_GETFIXEDBUFFER (1L<<15)
|
||
|
||
|
||
This flag would be included in Py_TPFLAGS_DEFAULT:
|
||
|
||
#define Py_TPFLAGS_DEFAULT ( \
|
||
....
|
||
Py_TPFLAGS_HAVE_GETFIXEDBUFFER | \
|
||
....
|
||
0)
|
||
|
||
|
||
Extend the PyBufferProcs structure by new fields in
|
||
Include/object.h:
|
||
|
||
typedef size_t (*getfixedreadbufferproc)(PyObject *, void **);
|
||
typedef size_t (*getfixedwritebufferproc)(PyObject *, void **);
|
||
|
||
typedef struct {
|
||
getreadbufferproc bf_getreadbuffer;
|
||
getwritebufferproc bf_getwritebuffer;
|
||
getsegcountproc bf_getsegcount;
|
||
getcharbufferproc bf_getcharbuffer;
|
||
/* fixed buffer interface functions */
|
||
getfixedreadbufferproc bf_getfixedreadbufferproc;
|
||
getfixedwritebufferproc bf_getfixedwritebufferproc;
|
||
} PyBufferProcs;
|
||
|
||
|
||
The new fields are present if the Py_TPFLAGS_HAVE_GETFIXEDBUFFER
|
||
flag is set in the object's type.
|
||
|
||
The Py_TPFLAGS_HAVE_GETFIXEDBUFFER flag implies the
|
||
Py_TPFLAGS_HAVE_GETCHARBUFFER flag.
|
||
|
||
The getfixedreadbufferproc and getfixedwritebufferproc functions
|
||
return the size in bytes of the memory block on success, and fill
|
||
in the passed void * pointer on success. If these functions fail
|
||
- either because an error occurs or no memory block is exposed -
|
||
they must set the void * pointer to NULL and raise an exception.
|
||
The return value is undefined in these cases and should not be
|
||
used.
|
||
|
||
Usually the getfixedwritebufferproc and getfixedreadbufferproc
|
||
functions aren't called directly, they are called through
|
||
convenience functions declared in Include/abstract.h:
|
||
|
||
int PyObject_AsFixedReadBuffer(PyObject *obj,
|
||
void **buffer,
|
||
size_t *buffer_len);
|
||
|
||
int PyObject_AsFixedWriteBuffer(PyObject *obj,
|
||
void **buffer,
|
||
size_t *buffer_len);
|
||
|
||
These functions return 0 on success, set buffer to the memory
|
||
location and buffer_len to the length of the memory block in
|
||
bytes. On failure, or if the fixed buffer interface is not
|
||
implemented by obj, they return -1 and set an exception.
|
||
|
||
|
||
Backward Compatibility
|
||
|
||
The size of the PyBufferProcs structure changes if this proposal
|
||
is implemented, but the type's tp_flags slot can be used to
|
||
determine if the additional fields are present.
|
||
|
||
|
||
Reference Implementation
|
||
|
||
Will be uploaded to the SourceForge patch manager by the author.
|
||
|
||
|
||
Additional Notes/Comments
|
||
|
||
Python strings, Unicode strings, mmap objects, and maybe other
|
||
types would expose the fixed buffer interface, but the array type
|
||
would *not*, because its memory block may be reallocated during
|
||
its lifetime.
|
||
|
||
|
||
Community Feedback
|
||
|
||
Greg Ewing doubts the fixed buffer interface is needed at all, he
|
||
thinks the normal buffer interface could be used if the pointer is
|
||
(re)fetched each time it's used. This seems to be dangerous,
|
||
because even innocent looking calls to the Python API like
|
||
Py_DECREF() may trigger execution of arbitrary Python code.
|
||
|
||
Neil Hodgson wants to expose pointers to memory blocks with
|
||
limited lifetime: do some kind of lock operation on the object,
|
||
retrieve the pointer, use it, and unlock the object again. While
|
||
the author sees the need for this, it cannot be addressed by this
|
||
proposal. Beeing required to call a function after not using the
|
||
pointer received by the getfixedbufferprocs any more seems too
|
||
error prone.
|
||
|
||
|
||
Credits
|
||
|
||
Scott Gilbert came up with the name 'fixed buffer interface'.
|
||
|
||
|
||
References
|
||
|
||
[1] The buffer interface
|
||
http://mail.python.org/pipermail/python-dev/2000-October/009974.html
|
||
|
||
[2] The Buffer Problem
|
||
http://www.python.org/peps/pep-0296.html
|
||
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
End:
|