PEP 454: remove filters on tracing

* Remove add_filter(), get_filters(), clear_filters()
* Remove the default filter on tracemalloc.py
* Give more explanation on the impacts of set_traceback_limit()
This commit is contained in:
Victor Stinner 2013-11-09 03:13:58 +01:00
parent d2dea92fa8
commit 20e6c78e68
1 changed files with 33 additions and 81 deletions

View File

@ -99,10 +99,6 @@ recent frame (1 frame). To store 25 frames at startup: set the
tracemalloc=25`` command line option. The ``set_traceback_limit()`` tracemalloc=25`` command line option. The ``set_traceback_limit()``
function can be used at runtime to set the limit. function can be used at runtime to set the limit.
By default, Python memory blocks allocated in the ``tracemalloc`` module
are ignored using a filter. Use ``clear_filters()`` to trace also these
memory allocations.
Main functions Main functions
-------------- --------------
@ -136,22 +132,17 @@ Main functions
``start()`` function: ``start()`` function:
Start tracing Python memory allocations. Start tracing Python memory allocations: install hooks on Python
memory allocators.
The function installs hooks on Python memory allocators. These hooks
have important overhead in term of performances and memory usage:
see `Filter functions`_ to limit the overhead.
See also ``stop()`` and ``is_tracing()`` functions. See also ``stop()`` and ``is_tracing()`` functions.
``stop()`` function: ``stop()`` function:
Stop tracing Python memory allocations and clear traces of memory Stop tracing Python memory allocations: uninstall hooks on Python
blocks allocated by Python. memory allocators. Clear also traces of memory blocks allocated by
Python
The function uninstalls hooks on Python memory allocators, so the
overhead of the module becomes null.
Call ``take_snapshot()`` function to take a snapshot of traces Call ``take_snapshot()`` function to take a snapshot of traces
before clearing them. before clearing them.
@ -165,8 +156,7 @@ Main functions
Return a new ``Snapshot`` instance. Return a new ``Snapshot`` instance.
The snapshot does not include memory blocks allocated before the The snapshot does not include memory blocks allocated before the
``tracemalloc`` module started to trace memory allocations nor ``tracemalloc`` module started to trace memory allocations.
memory blocks ignored by filters (see ``get_filters()``).
Tracebacks of traces are limited to ``get_traceback_limit()`` Tracebacks of traces are limited to ``get_traceback_limit()``
frames. Use ``set_traceback_limit()`` to store more frames. frames. Use ``set_traceback_limit()`` to store more frames.
@ -209,10 +199,14 @@ allocation occurred. See the ``Trace`` class.
Set the maximum number of frames stored in the traceback of a trace. Set the maximum number of frames stored in the traceback of a trace.
Storing the traceback of each memory allocation has an important Storing more frames allocates more memory of the ``tracemalloc``
overhead on the memory usage. Use the ``get_tracemalloc_memory()`` module and makes Python slower. Use the ``get_tracemalloc_memory()``
function to measure the overhead and the ``add_filter()`` function function to measure how much memory is used by the ``tracemalloc``
to select which memory allocations are traced. module.
Storing more than ``1`` frame is only useful to compute statistics
grouped by ``'traceback'`` or to compute cumulative statistics: see
the ``Snapshot.statistics()`` method.
If the limit is set to ``0`` frame, a traceback with a frame will be If the limit is set to ``0`` frame, a traceback with a frame will be
used for all traces: filename ``'<unknown>'`` at line number ``0``. used for all traces: filename ``'<unknown>'`` at line number ``0``.
@ -224,70 +218,25 @@ allocation occurred. See the ``Trace`` class.
command line option can be used to set the limit at startup. command line option can be used to set the limit at startup.
Filter functions
----------------
Tracing all Python memroy allocations has an important overhead on performances
and on the memory usage.
To limit the overhead, some files can be excluded or tracing can be restricted
to a set of files using filters. Examples: ``add_filter(Filter(True,
subprocess.__file__))`` only traces memory allocations in the ``subprocess``
module, and ``add_filter(Filter(False, tracemalloc.__file__))`` ignores
memory allocations in the ``tracemalloc`` module
By default, there is one exclusive filter to ignore Python memory blocks
allocated by the ``tracemalloc`` module.
Use the ``get_tracemalloc_memory()`` function to measure the memory usage.
See also the ``set_traceback_limit()`` function to configure how many
frames are stored.
``add_filter(filter)`` function:
Add a new filter on Python memory allocations, *filter* is a
``Filter`` instance.
All inclusive filters are applied at once, a memory allocation is
ignored if no inclusive filters match its trace. A memory allocation
is ignored if at least one exclusive filter matchs its trace.
The new filter is not applied on already collected traces. Use the
``clear_traces()`` function to ensure that all traces match the new
filter.
``clear_filters()`` function:
Clear the filter list.
See also the ``get_filters()`` function.
``get_filters()`` function:
Get the filters on Python memory allocations. Return a list of
``Filter`` instances.
See also the ``clear_filters()`` function.
Filter Filter
------ ------
``Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False)`` class: ``Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False)`` class:
Filter to select which memory allocations are traced. Filters can be Filter on traces of memory blocks.
used to reduce the memory usage of the ``tracemalloc`` module, which
can be read using the ``get_tracemalloc_memory()`` function.
The ``'*'`` joker character can be used in *filename_pattern* to See the ``fnmatch.fnmatch()`` function for the syntax of
match any substring, including empty string. The ``'.pyc'`` and *filename_pattern*. The ``'.pyc'`` and ``'.pyo'`` file extensions
``'.pyo'`` file extensions are replaced with ``'.py'``. On Windows, are replaced with ``'.py'``.
the comparison is case insensitive and the alternative separator
``'/'`` is replaced with the standard separator ``'\'``.
Use ``Filter(False, "<unknown>")`` to exclude empty tracebacks. Examples:
* ``Filter(True, subprocess.__file__)`` only includes traces of the
``subprocess`` module
* ``Filter(False, tracemalloc.__file__)``
excludes traces of the ``tracemalloc`` module.
* ``Filter(False,
"<unknown>")`` excludes empty tracebacks
``inclusive`` attribute: ``inclusive`` attribute:
@ -348,11 +297,14 @@ Snapshot
``apply_filters(filters)`` method: ``apply_filters(filters)`` method:
Create a new ``Snapshot`` instance with the filtered ``traces`` Create a new ``Snapshot`` instance with a filtered ``traces``
sequence, *filters* is a list of ``Filter`` instances. sequence, *filters* is a list of ``Filter`` instances. If *filters*
is an empty list, return a new ``Snapshot`` instance with a copy of
the traces.
If *filters* is an empty list, return a new ``Snapshot`` instance All inclusive filters are applied at once, a trace is ignored if no
with a copy of the traces. inclusive filters match it. A trace is ignored if at least one
exclusive filter matchs it.
``compare_to(old_snapshot: Snapshot, group_by: str, cumulative: bool=False)`` method: ``compare_to(old_snapshot: Snapshot, group_by: str, cumulative: bool=False)`` method: