Add some comments and acknowledgements

This commit is contained in:
Antoine Pitrou 2013-04-27 13:24:45 +02:00
parent 455dd3f556
commit e1ba742ebc
1 changed files with 41 additions and 9 deletions

View File

@ -7,8 +7,9 @@ Status: Draft
Type: Standards Track Type: Standards Track
Content-Type: text/x-rst Content-Type: text/x-rst
Created: 2011-08-11 Created: 2011-08-11
Python-Version: 3.3 Python-Version: 3.4
Post-History: Post-History:
http://mail.python.org/pipermail/python-dev/2011-August/112821.html
Resolution: TBD Resolution: TBD
@ -36,10 +37,10 @@ language (mainly, unicode strings by default and the new bytes
object). The opportunity was not taken at the time to improve the object). The opportunity was not taken at the time to improve the
protocol in other ways. protocol in other ways.
This PEP is an attempt to foster a number of small incremental This PEP is an attempt to foster a number of incremental improvements
improvements in a future new protocol version. The PEP process is in a new pickle protocol version. The PEP process is used in order to
used in order to gather as many improvements as possible, because the gather as many improvements as possible, because the introduction of a
introduction of a new protocol version should be a rare occurrence. new pickle protocol should be a rare occurrence.
Proposed changes Proposed changes
@ -74,11 +75,15 @@ to straddle frame boundaries. The pickler takes care not to produce such
pickles, and the unpickler refuses them. Also, there is no "last frame" pickles, and the unpickler refuses them. Also, there is no "last frame"
marker. The last frame is simply the one which ends with a STOP opcode. marker. The last frame is simply the one which ends with a STOP opcode.
A well-written C implementation doesn't need additional memory copies
for the framing layer, preserving general (un)pickling efficiency.
.. note:: .. note::
How the pickler partitions the pickle stream into frames is an How the pickler decides to partition the pickle stream into frames is an
implementation detail. "Closing" a frame as soon as it reaches implementation detail. For example, "closing" a frame as soon as it
~64 KiB should be ok for both performance and pickle size overhead. reaches ~64 KiB is a reasonable choice for both performance and pickle
size overhead.
Binary encoding for all opcodes Binary encoding for all opcodes
------------------------------- -------------------------------
@ -138,10 +143,31 @@ integer, which is wasteful. A specific opcode with a 1-byte length
would make many pickles smaller. would make many pickles smaller.
Alternative ideas
=================
Prefetching
-----------
Serhiy Storchaka suggested to replace framing with a special PREFETCH
opcode (with a 2- or 4-bytes argument) to declare known pickle chunks
explicitly. Large data may be pickled outside such chunks. A naïve
unpickler should be able to skip the PREFETCH opcode and still decode
pickles properly, but good error handling would require checking that
the PREFETCH length falls on an opcode boundary.
Acknowledgments Acknowledgments
=============== ===============
(...) In alphabetic order:
* Alexandre Vassalotti, for starting the second PEP 3154 implementation [6]_
* Serhiy Storchaka, for discussing the framing proposal [6]_
* Stefan Mihaila, for starting the first PEP 3154 implementation as a
Google Summer of Code project mentored by Alexandre Vassalotti [7]_.
References References
@ -162,6 +188,12 @@ References
.. [5] Lib/multiprocessing/forking.py: .. [5] Lib/multiprocessing/forking.py:
http://hg.python.org/cpython/file/baea9f5f973c/Lib/multiprocessing/forking.py#l54 http://hg.python.org/cpython/file/baea9f5f973c/Lib/multiprocessing/forking.py#l54
.. [6] Implement PEP 3154, by Alexandre Vassalotti
http://bugs.python.org/issue17810
.. [7] Implement PEP 3154, by Stefan Mihaila
http://bugs.python.org/issue15642
Copyright Copyright
========= =========