PEP 445: cleanup

Avoid "should", "may" and "might". Rephrase some sentences

pep-0445.txt

@@ -12,7 +12,8 @@ Python-Version: 3.4
 Abstract
 ========
 
-Add new APIs to customize Python memory allocators.
+Add new Application Programming Interfaces (API) to customize Python
+memory allocators.
 
 
 Rationale
@@ -20,21 +21,22 @@ Rationale
 
 Use cases:
 
-* Application embedding Python may want to isolate Python memory from
+* Applications embedding Python which want to isolate Python memory from
-  the memory of the application, or may want to use a different memory
+  the memory of the application, or want to use a different memory
   allocator optimized for its Python usage
 * Python running on embedded devices with low memory and slow CPU.
-  A custom memory allocator may be required to use efficiently the
+  A custom memory allocator can be used for efficiency and/or to get
-  memory and/or to be able to use all the memory of the device.
+  access all the memory of the device.
-* Debug tool to:
+* Debug tools for memory allocators:
 
-  - track the memory usage (memory leaks)
+  - track the memory usage (find memory leaks)
-  - get the Python filename and line number where an object was
+  - get the location of a memory allocation: Python filename and line
-    allocated
+    number, and the size of a memory block
-  - detect buffer underflow, buffer overflow and detect misuse of Python
+  - detect buffer underflow, buffer overflow and misuse of Python
-    allocator APIs (builtin Python debug hooks)
+    allocator APIs (see `Redesign Debug Checks on Memory Block
-  - force allocation to fail to test handling of ``MemoryError``
+    Allocators as Hooks`_)
-    exception
+  - force memory allocations to fail to test handling of the
+    ``MemoryError`` exception
 
 
 Proposal
@@ -56,8 +58,7 @@ New Functions and Structures
 * Add a new ``PyMemAllocator`` structure::
 
     typedef struct {
-        /* user context passed as the first argument
+        /* user context passed as the first argument to the 3 functions */
-           to the 3 functions */
         void *ctx;
 
         /* allocate a memory block */
@@ -82,7 +83,7 @@ New Functions and Structures
   - ``PYMEM_DOMAIN_OBJ``: ``PyObject_Malloc()``, ``PyObject_Realloc()``
     and ``PyObject_Free()``
 
-* Add new functions to get and set memory allocators:
+* Add new functions to get and set memory block allocators:
 
   - ``void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
   - ``void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
@@ -94,8 +95,7 @@ New Functions and Structures
 * Add a new ``PyObjectArenaAllocator`` structure::
 
     typedef struct {
-        /* user context passed as the first argument
+        /* user context passed as the first argument to the 2 functions */
-           to the 2 functions */
         void *ctx;
 
         /* allocate an arena */
@@ -111,18 +111,17 @@ New Functions and Structures
   - ``void PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)``
   - ``void PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)``
 
-* Add a new function to setup the debug checks on memory allocators when
+* Add a new function to reinstall the debug checks on memory allocators when
-  a memory allocator is replaced:
+  a memory allocator is replaced with ``PyMem_SetAllocator()``:
 
   - ``void PyMem_SetupDebugHooks(void)``
-  - Install the debug hook on all memory block allocators. The function
+  - Install the debug hooks on all memory block allocators. The function can be
-    can be called more than once, hooks are not reinstalled if they
+    called more than once, hooks are only installed once.
-    were already installed.
+  - The function does nothing is Python is not compiled in debug mode.
-  - The function does nothing is Python is not compiled in debug mode
 
-* Memory allocators always returns *NULL* if size is greater than
+* Memory block allocators always return *NULL* if *size* is greater than
-  ``PY_SSIZE_T_MAX``. The check is done before calling the
+  ``PY_SSIZE_T_MAX``. The check is done before calling the inner
-  inner function.
+  function.
 
 The *pymalloc* allocator is optimized for objects smaller than 512 bytes
 with a short lifetime. It uses memory mappings with a fixed size of 256
@@ -140,8 +139,8 @@ Default allocators:
   and ``free()``
 
 
-Redesign Debug Checks on  Memory Allocators as Hooks
+Redesign Debug Checks on Memory Block Allocators as Hooks
-----------------------------------------------------
+---------------------------------------------------------
 
 Since Python 2.3, Python implements different checks on memory
 allocators in debug mode:
@@ -157,7 +156,8 @@ In Python 3.3, the checks are installed by replacing ``PyMem_Malloc()``,
 ``PyMem_Realloc()``, ``PyMem_Free()``, ``PyObject_Malloc()``,
 ``PyObject_Realloc()`` and ``PyObject_Free()`` using macros. The new
 allocator allocates a larger buffer and write a pattern to detect buffer
-underflow and overflow. It uses the original ``PyObject_Malloc()``
+underflow, buffer overflow and use after free (fill the buffer with the
+pattern ``0xDB``). It uses the original ``PyObject_Malloc()``
 function to allocate memory. So ``PyMem_Malloc()`` and
 ``PyMem_Realloc()`` call indirectly ``PyObject_Malloc()`` and
 ``PyObject_Realloc()``.
@@ -178,13 +178,14 @@ Call traces when the hooks are installed (debug mode):
 * ``PyObject_Free()`` => ``_PyMem_DebugFree()``
   => ``_PyObject_Free()``
 
-As a result, ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call
+As a result, ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now call
-``malloc()`` and ``realloc()``, instead of calling ``PyObject_Malloc()``
+``malloc()`` and ``realloc()`` in release mode and in debug mode,
-and ``PyObject_Realloc()`` in debug mode.
+instead of calling ``PyObject_Malloc()`` and ``PyObject_Realloc()`` in
+debug mode.
 
 When at least one memory allocator is replaced with
 ``PyMem_SetAllocator()``, the ``PyMem_SetupDebugHooks()`` function must
-be called to install the debug hooks on top on the new allocator.
+be called to reinstall the debug hooks on top on the new allocator.
 
 
 Don't call malloc() directly anymore
@@ -195,7 +196,7 @@ Don't call malloc() directly anymore
 ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
 ``realloc()``
 
-Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
+Direct calls to ``malloc()`` are replaced with ``PyMem_Malloc()``, or
 ``PyMem_RawMalloc()`` if the GIL is not held.
 
 Configure external libraries like zlib or OpenSSL to allocate memory
@@ -205,22 +206,22 @@ Python is embedded in an application.
 
 For the "track memory usage" use case, it is important to track memory
 allocated in external libraries to have accurate reports, because these
-allocations may be large.
+allocations can be large (can raise a ``MemoryError`` exception).
 
 If an hook is used to the track memory usage, the memory allocated by
-``malloc()`` will not be tracked. Remaining ``malloc()`` in external
+direct calls to ``malloc()`` will not be tracked. Remaining ``malloc()``
-libraries like OpenSSL or bz2 may allocate large memory blocks and so
+in external libraries like OpenSSL or bz2 can allocate large memory
-would be missed in memory usage reports.
+blocks and so would be missed in memory usage reports.
 
 
 Examples
 ========
 
-Use case 1: Replace Memory Allocator, keep pymalloc
+Use case 1: Replace Memory Allocators, keep pymalloc
 ----------------------------------------------------
 
 Dummy example wasting 2 bytes per memory block,
-and 10 bytes per memory mapping::
+and 10 bytes per *pymalloc* arena::
 
     #include <stdlib.h>
 
@@ -267,6 +268,7 @@ and 10 bytes per memory mapping::
 
         PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
         PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
+        /* leave PYMEM_DOMAIN_OBJ unchanged, use pymalloc */
 
         arena.ctx = &arena_padding;
         arena.alloc = my_alloc_arena;
@@ -277,7 +279,7 @@ and 10 bytes per memory mapping::
     }
 
 
-Use case 2: Replace Memory Allocator, override pymalloc
+Use case 2: Replace Memory Allocators, override pymalloc
 --------------------------------------------------------
 
 If your allocator is optimized for allocations of objects smaller than
@@ -322,11 +324,14 @@ Dummy example wasting 2 bytes per memory block::
         PyMem_SetupDebugHooks();
     }
 
+The *pymalloc* arena does not need to be replaced, because it is no more
+used by the new allocator.
 
-Use case 3: Setup Allocator Hooks
----------------------------------
 
-Example to setup hooks on all memory allocators::
+Use case 3: Setup Hooks On Memory Block Allocators
+--------------------------------------------------
+
+Example to setup hooks on all memory block allocators::
 
     struct {
         PyMemAllocator raw;
@@ -390,22 +395,23 @@ Example to setup hooks on all memory allocators::
     }
 
 .. note::
-   ``PyMem_SetupDebugHooks()`` does not need to be called because the
+   ``PyMem_SetupDebugHooks()`` does not need to be called because
-   allocator is not replaced: Python debug hooks are installed
+   memory allocator are not replaced: the debug checks on memory
-   automatically at startup.
+   block allocators are installed automatically at startup.
 
 
 Performances
 ============
 
+The implementation of this PEP (issue #3329) has no visible overhead on
+the Python benchmark suite.
+
 Results of the `Python benchmarks suite
 <http://hg.python.org/benchmarks>`_ (-b 2n3): some tests are 1.04x
-faster, some tests are 1.04 slower, significant is between 115 and -191.
+faster, some tests are 1.04 slower. Results of pybench microbenchmark:
+"+0.1%" slower globally (diff between -4.9% and +5.6%).
 
-Results of pybench benchmark: "+0.1%" slower globally (diff between
+The full output of benchmarks is attached to the issue #3329.
--4.9% and +5.6%).
-
-The full reports are attached to the issue #3329.
 
 
 Rejected Alternatives
@@ -428,8 +434,9 @@ with:
 * ``void PyMem_SetAllocator(PyMemAllocator *allocator)``
 * ``void PyObject_SetAllocator(PyMemAllocator *allocator)``
 
-With more specific functions, it becomes more difficult to write generic
+This alternative was rejected because it is not possible to write
-code, like reusing the same code for different allocator domains.
+generic code with more specific functions: code must be duplicated for
+each memory allocator domain.
 
 
 Make PyMem_Malloc() reuse PyMem_RawMalloc() by default
@@ -439,25 +446,25 @@ If ``PyMem_Malloc()`` would call ``PyMem_RawMalloc()`` by default,
 calling ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, alloc)`` would also also
 patch ``PyMem_Malloc()`` indirectly.
 
-This option was rejected because ``PyMem_SetAllocator()`` would have a
+This alternative was rejected because ``PyMem_SetAllocator()`` would
-different behaviour depending on the domain. Always having the same
+have a different behaviour depending on the domain. Always having the
-behaviour is less error-prone.
+same behaviour is less error-prone.
 
 
 Add a new PYDEBUGMALLOC environment variable
 --------------------------------------------
 
-To be able to use the Python builtin debug hooks even when a custom
+Add a new ``PYDEBUGMALLOC`` environment variable to enable debug checks
-memory allocator replaces the default Python allocator, an environment
+on memory block allocators. The environment variable replaces the new
-variable ``PYDEBUGMALLOC`` can be added to setup these debug function
+function ``PyMem_SetupDebugHooks()`` which is not needed anymore.
-hooks, instead of adding the new function ``PyMem_SetupDebugHooks()``.
+Another advantage is to allow to enable debug checks even in release
-If the environment variable is present, ``PyMem_SetRawAllocator()``,
+mode: debug checks are always compiled, but only enabled when the
-``PyMem_SetAllocator()`` and ``PyObject_SetAllocator()`` will reinstall
+environment variable is present and non-empty.
-automatically the hook on top of the new allocator.
 
-A new environment variable would make the Python initialization even
+This alternative was rejected because a new environment variable would
-more complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_
+make the Python initialization even more complex. The `PEP 432
-tries to simply the CPython startup sequence.
+<http://www.python.org/dev/peps/pep-0432/>`_ tries to simply the CPython
+startup sequence.
 
 
 Use macros to get customizable allocators
@@ -503,7 +510,7 @@ Example of ``PyMem_Malloc`` macro with the modified
     void* _PyMem_MallocTrace(const char *filename, int lineno,
                              size_t size);
 
-    /* need also a function for the Python stable ABI */
+    /* the function is still needed for the Python stable ABI */
     void* PyMem_Malloc(size_t size);
 
     #define PyMem_Malloc(size) \
@@ -527,19 +534,19 @@ In Python 3.3, when Python is compiled in debug mode, ``PyMem_Malloc()``
 calls indirectly ``PyObject_Malloc()`` which requires the GIL to be
 held.  That's why ``PyMem_Malloc()`` must be called with the GIL held.
 
-This PEP proposes changes ``PyMem_Malloc()``: it now always call
+This PEP changes ``PyMem_Malloc()``: it now always call ``malloc()``.
-``malloc()``.  The "GIL must be held" restriction can be removed from
+The "GIL must be held" restriction could be removed from
 ``PyMem_Malloc()``.
 
 This alternative was rejected because allowing to call
-``PyMem_Malloc()`` without holding the GIL might break applications
+``PyMem_Malloc()`` without holding the GIL can break applications
 which setup their own allocators or allocator hooks.  Holding the GIL is
 convinient to develop a custom allocator: no need to care of other
 threads. It is also convinient for a debug allocator hook: Python
 internal objects can be safetly inspected.
 
 Calling ``PyGILState_Ensure()`` in
-a memory allocator may have unexpected behaviour, especially at Python
+a memory allocator has unexpected behaviour, especially at Python
 startup and at creation of a new Python thread state.
 
 
@@ -552,13 +559,14 @@ held.  Otherwise, keep ``malloc()`` unchanged.
 The ``PyMem_Malloc()`` is used without the GIL held in some Python
 functions.  For example, the ``main()`` and ``Py_Main()`` functions of
 Python call ``PyMem_Malloc()`` whereas the GIL do not exist yet. In this
-case, ``PyMem_Malloc()`` should be replaced with ``malloc()`` (or
+case, ``PyMem_Malloc()`` would be replaced with ``malloc()`` (or
 ``PyMem_RawMalloc()``).
 
-If an hook is used to the track memory usage, the memory allocated by
+This alternative was rejected because ``PyMem_RawMalloc()`` is required
-direct calls to ``malloc()`` will not be tracked. External libraries
+for accurate reports of the memory usage. When a debug hook is used to
-like OpenSSL or bz2 should not call ``malloc()`` directly, so large
+track the memory usage, the memory allocated by direct calls to
-allocated will be included in memory usage reports.
+``malloc()`` cannot be tracked. ``PyMem_RawMalloc()`` can be hooked and
+so all the memory allocated by Python can be tracked.
 
 
 Use existing debug tools to analyze the memory
@@ -571,11 +579,11 @@ examples: `Valgrind <http://valgrind.org/>`_, `Purify
 <http://www.nongnu.org/failmalloc/>`_, etc.
 
 The problem is to retrieve the Python object related to a memory pointer
-to read its type and/or content. Another issue is to retrieve the
+to read its type and/or its content. Another issue is to retrieve the
 location of the memory allocation: the C backtrace is usually useless
-(same reasoning than macros using ``__FILE__`` and ``__LINE__``), the
+(same reasoning than macros using ``__FILE__`` and ``__LINE__``, see
-Python filename and line number (or even the Python traceback) is more
+`Pass the C filename and line number`_), the Python filename and line
-useful.
+number (or even the Python traceback) is more useful.
 
 This alternative was rejected because classic tools are unable to
 introspect Python internals to collect such information. Being able to
@@ -586,8 +594,8 @@ lot of useful data from Python internals.
 Add a msize() function
 ----------------------
 
-Add another field to ``PyMemAllocator`` and ``PyObjectArenaAllocator``
+Add another function to ``PyMemAllocator`` and
-structures::
+``PyObjectArenaAllocator`` structures::
 
     size_t msize(void *ptr);
 
@@ -607,8 +615,8 @@ The additional ``msize()`` function was rejected because only few
 platforms implement it. For example, Linux with the GNU libc does not
 provide a function to get the size of a memory block. ``msize()`` is not
 currently used in the Python source code. The function is only used to
-track the memory usage, but makes the API more complex. A debug hook can
+track the memory usage, and makes the API more complex. A debug hook can
-implemente the function internally, there is no need to add it to
+implement the function internally, there is no need to add it to
 ``PyMemAllocator`` and ``PyObjectArenaAllocator`` structures.
 
 
@@ -653,17 +661,19 @@ Libraries used by Python:
 * lzma: `LZMA SDK - How to Use
   <http://www.asawicki.info/news_1368_lzma_sdk_-_how_to_use.html>`_,
   pass an opaque pointer
-* lipmpdec doesn't have this extra *ctx* parameter
+* lipmpdec: no opaque pointer (classic malloc API)
 
 Other libraries:
 
 * glib: `g_mem_set_vtable()
   <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
-* libxml2: `xmlGcMemSetup() <http://xmlsoft.org/html/libxml-xmlmemory.html>`_,
+* libxml2:
+  `xmlGcMemSetup() <http://xmlsoft.org/html/libxml-xmlmemory.html>`_,
   global
 * Oracle's OCI: `Oracle Call Interface Programmer's Guide,
   Release 2 (9.2)
-  <http://docs.oracle.com/cd/B10501_01/appdev.920/a96584/oci15re4.htm>`_
+  <http://docs.oracle.com/cd/B10501_01/appdev.920/a96584/oci15re4.htm>`_,
+  pass an opaque pointer
 
 See also the `GNU libc: Memory Allocation Hooks
 <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_.
@@ -676,30 +686,30 @@ The C standard library provides the well known ``malloc()`` function.
 Its implementation depends on the platform and of the C library. The GNU
 C library uses a modified ptmalloc2, based on "Doug Lea's Malloc"
 (dlmalloc). FreeBSD uses `jemalloc
-<http://www.canonware.com/jemalloc/>`_. Google provides tcmalloc which
+<http://www.canonware.com/jemalloc/>`_. Google provides *tcmalloc* which
 is part of `gperftools <http://code.google.com/p/gperftools/>`_.
 
 ``malloc()`` uses two kinds of memory: heap and memory mappings. Memory
 mappings are usually used for large allocations (ex: larger than 256
 KB), whereas the heap is used for small allocations.
 
-On UNIX, the heap is handled by ``brk()`` and ``sbrk()`` system calls on
+On UNIX, the heap is handled by ``brk()`` and ``sbrk()`` system calls,
-Linux, and it is contiguous.  On Windows, the heap is handled by
+and it is contiguous.  On Windows, the heap is handled by
-``HeapAlloc()`` and may be discontiguous. Memory mappings are handled by
+``HeapAlloc()`` and can be discontiguous. Memory mappings are handled by
-``mmap()`` on UNIX and ``VirtualAlloc()`` on Windows, they may be
+``mmap()`` on UNIX and ``VirtualAlloc()`` on Windows, they can be
 discontiguous.
 
 Releasing a memory mapping gives back immediatly the memory to the
-system. On UNIX, heap memory is only given back to the system if it is
+system. On UNIX, the heap memory is only given back to the system if the
-at the end of the heap. Otherwise, the memory will only be given back to
+released block is located at the end of the heap. Otherwise, the memory
-the system when all the memory located after the released memory are
+will only be given back to the system when all the memory located after
-also released.
+the released memory is also released.
 
-To allocate memory in the heap, the allocator tries to reuse free space.
+To allocate memory on the heap, an allocator tries to reuse free space.
-If there is no contiguous space big enough, the heap must be increased,
+If there is no contiguous space big enough, the heap must be enlarged,
-even if we have more free space than required size.  This issue is
+even if there is more free space than required size.  This issue is
 called the "memory fragmentation": the memory usage seen by the system
-may be much higher than real usage. On Windows, ``HeapAlloc()`` creates
+is higher than real usage. On Windows, ``HeapAlloc()`` creates
 a new memory mapping with ``VirtualAlloc()`` if there is not enough free
 contiguous memory.
 
@@ -730,8 +740,8 @@ CPython issues related to memory allocation:
   <http://bugs.python.org/issue3329>`_
 * `Issue #13483: Use VirtualAlloc to allocate memory arenas
   <http://bugs.python.org/issue13483>`_
-* `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which
+* `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline,
-  isn't thread safe <http://bugs.python.org/issue16742>`_
+  which isn't thread safe <http://bugs.python.org/issue16742>`_
 * `Issue #18203: Replace calls to malloc() with PyMem_Malloc() or
   PyMem_RawMalloc() <http://bugs.python.org/issue18203>`_
 * `Issue #18227: Use Python memory allocators in external libraries like