diff --git a/pep-3154.txt b/pep-3154.txt index e98adbf89..8e0d4e368 100644 --- a/pep-3154.txt +++ b/pep-3154.txt @@ -7,8 +7,9 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2011-08-11 -Python-Version: 3.3 +Python-Version: 3.4 Post-History: + http://mail.python.org/pipermail/python-dev/2011-August/112821.html Resolution: TBD @@ -36,10 +37,10 @@ language (mainly, unicode strings by default and the new bytes object). The opportunity was not taken at the time to improve the protocol in other ways. -This PEP is an attempt to foster a number of small incremental -improvements in a future new protocol version. The PEP process is -used in order to gather as many improvements as possible, because the -introduction of a new protocol version should be a rare occurrence. +This PEP is an attempt to foster a number of incremental improvements +in a new pickle protocol version. The PEP process is used in order to +gather as many improvements as possible, because the introduction of a +new pickle protocol should be a rare occurrence. Proposed changes @@ -74,11 +75,15 @@ to straddle frame boundaries. The pickler takes care not to produce such pickles, and the unpickler refuses them. Also, there is no "last frame" marker. The last frame is simply the one which ends with a STOP opcode. +A well-written C implementation doesn't need additional memory copies +for the framing layer, preserving general (un)pickling efficiency. + .. note:: - How the pickler partitions the pickle stream into frames is an - implementation detail. "Closing" a frame as soon as it reaches - ~64 KiB should be ok for both performance and pickle size overhead. + How the pickler decides to partition the pickle stream into frames is an + implementation detail. For example, "closing" a frame as soon as it + reaches ~64 KiB is a reasonable choice for both performance and pickle + size overhead. Binary encoding for all opcodes ------------------------------- @@ -138,10 +143,31 @@ integer, which is wasteful. A specific opcode with a 1-byte length would make many pickles smaller. +Alternative ideas +================= + +Prefetching +----------- + +Serhiy Storchaka suggested to replace framing with a special PREFETCH +opcode (with a 2- or 4-bytes argument) to declare known pickle chunks +explicitly. Large data may be pickled outside such chunks. A naïve +unpickler should be able to skip the PREFETCH opcode and still decode +pickles properly, but good error handling would require checking that +the PREFETCH length falls on an opcode boundary. + + Acknowledgments =============== -(...) +In alphabetic order: + +* Alexandre Vassalotti, for starting the second PEP 3154 implementation [6]_ + +* Serhiy Storchaka, for discussing the framing proposal [6]_ + +* Stefan Mihaila, for starting the first PEP 3154 implementation as a + Google Summer of Code project mentored by Alexandre Vassalotti [7]_. References @@ -162,6 +188,12 @@ References .. [5] Lib/multiprocessing/forking.py: http://hg.python.org/cpython/file/baea9f5f973c/Lib/multiprocessing/forking.py#l54 +.. [6] Implement PEP 3154, by Alexandre Vassalotti + http://bugs.python.org/issue17810 + +.. [7] Implement PEP 3154, by Stefan Mihaila + http://bugs.python.org/issue15642 + Copyright =========