PEP 574: change buffer callback semantics (#742)

`buffer_callback` now takes a single buffer argument and can return a true-ish
value if the buffer is to be considered in-band.
This commit is contained in:
Antoine Pitrou 2018-07-20 13:25:18 +02:00 committed by Nick Coghlan
parent 200fe676b4
commit 1ed3043948
1 changed files with 29 additions and 19 deletions

View File

@ -227,16 +227,17 @@ Consumer API
class Pickler:
def __init__(self, file, protocol=None, ..., buffer_callback=None):
"""
If *buffer_callback* is not None, then it is called with a list
of out-of-band buffer views when deemed necessary (this could be
once every buffer, or only after a certain size is reached,
or once at the end, depending on implementation details). The
callback should arrange to store or transmit those buffers without
changing their order.
If *buffer_callback* is None (the default), buffer views are
serialized into *file* as part of the pickle stream.
If *buffer_callback* is not None, then it can be called any number
of times with a buffer view. If the callback returns a false value
(such as None), the given buffer is out-of-band; otherwise the
buffer is serialized in-band, i.e. inside the pickle stream.
The callback should arrange to store or transmit out-of-band buffers
without changing their order.
It is an error if *buffer_callback* is not None and *protocol* is
None or smaller than 5.
"""
@ -281,20 +282,29 @@ Three new opcodes are introduced:
it on the stack.
* ``READONLY_BUFFER`` makes a readonly view of the top of the stack.
When pickling encounters a ``PickleBuffer``, there can be four cases:
When pickling encounters a ``PickleBuffer``, that buffer can be considered
in-band or out-of-band depending on the following conditions:
* If a ``buffer_callback`` is given and the ``PickleBuffer`` is writable,
the ``PickleBuffer`` is given to the callback and a ``NEXT_BUFFER`` opcode
is appended to the pickle stream.
* If a ``buffer_callback`` is given and the ``PickleBuffer`` is readonly,
the ``PickleBuffer`` is given to the callback and a ``NEXT_BUFFER`` opcode
is appended to the pickle stream, followed by a ``READONLY_BUFFER`` opcode.
* If no ``buffer_callback`` is given and the ``PickleBuffer`` is writable,
it is serialized into the pickle stream as if it were a ``bytearray`` object.
* If no ``buffer_callback`` is given and the ``PickleBuffer`` is readonly,
it is serialized into the pickle stream as if it were a ``bytes`` object.
* if no ``buffer_callback`` is given, the buffer is in-band;
* if a ``buffer_callback`` is given, it is called with the buffer. If the
callback returns a true value, the buffer is in-band; if the callback
returns a false value, the buffer is out-of-band.
The distinction between readonly and writable buffers is explained below
An in-band buffer is serialized as follows:
* If the buffer is writable, it is serialized into the pickle stream as if
it were a ``bytearray`` object.
* If the buffer is readonly, it is serialized into the pickle stream as if
it were a ``bytes`` object.
An out-of-band buffer is serialized as follows:
* If the buffer is writable, a ``NEXT_BUFFER`` opcode is appended to the
pickle stream.
* If the buffer is readonly, a ``NEXT_BUFFER`` opcode is appended to the
pickle stream, followed by a ``READONLY_BUFFER`` opcode.
The distinction between readonly and writable buffers is motivated below
(see "Mutability").