PEP 454: second round based on the tracemalloc repository at changeset 21f7c3df0f15
This commit is contained in:
parent
2b83b4a265
commit
0cb64f9c69
740
pep-0454.txt
740
pep-0454.txt
|
@ -13,7 +13,7 @@ Python-Version: 3.4
|
||||||
Abstract
|
Abstract
|
||||||
========
|
========
|
||||||
|
|
||||||
Add a new ``tracemalloc`` module to trace Python memory allocations.
|
Add a new ``tracemalloc`` module to trace memory blocks allocated by Python.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@ -50,11 +50,11 @@ This PEP proposes to add a new ``tracemalloc`` module. It is a debug
|
||||||
tool to trace memory allocations made by Python. The module provides the
|
tool to trace memory allocations made by Python. The module provides the
|
||||||
following information:
|
following information:
|
||||||
|
|
||||||
* Statistics on Python memory allocations per Python filename and line
|
* Compute the differences between two snapshots to detect memory leaks
|
||||||
number: size, number, and average size of allocations
|
* Statistics on allocated memory blocks per filename and per line number:
|
||||||
* Compute differences between two snapshots of Python memory allocations
|
total size, number and average size of allocated memory blocks
|
||||||
* Location of a Python memory allocation: size in bytes, Python filename
|
* For each allocated memory block: its size and the traceback where the block
|
||||||
and line number
|
was allocated
|
||||||
|
|
||||||
The API of the tracemalloc module is similar to the API of the
|
The API of the tracemalloc module is similar to the API of the
|
||||||
faulthandler module: ``enable()``, ``disable()`` and ``is_enabled()``
|
faulthandler module: ``enable()``, ``disable()`` and ``is_enabled()``
|
||||||
|
@ -71,77 +71,110 @@ implementations of Python may not provide it.
|
||||||
API
|
API
|
||||||
===
|
===
|
||||||
|
|
||||||
To trace the most Python memory allocations, the module should be
|
To trace most memory blocks allocated by Python, the module should be
|
||||||
enabled as early as possible in your application by calling
|
enabled as early as possible by calling ``tracemalloc.enable()``
|
||||||
``tracemalloc.enable()`` function, by setting the ``PYTHONTRACEMALLOC``
|
function, by setting the ``PYTHONTRACEMALLOC`` environment variable to
|
||||||
environment variable to ``1``, or by using ``-X tracemalloc`` command
|
``1``, or by using ``-X tracemalloc`` command line option.
|
||||||
line option.
|
|
||||||
|
|
||||||
By default, tracemalloc only stores one ``frame`` instance per memory
|
By default, the ``Trace.traceback`` attribute only stores one ``Frame``
|
||||||
allocation. Use ``tracemalloc.set_number_frame()`` to store more frames.
|
instance per allocated memory block. Use ``set_traceback_limit()`` to
|
||||||
|
store more frames.
|
||||||
|
|
||||||
|
|
||||||
Functions
|
Functions
|
||||||
---------
|
---------
|
||||||
|
|
||||||
``add_filter(include: bool, filename: str, lineno: int=None)`` function:
|
``add_filter(filter)`` function:
|
||||||
|
|
||||||
Add a filter. If *include* is ``True``, only trace memory blocks
|
Add a new filter on Python memory allocations, *filter* is a
|
||||||
allocated in a file with a name matching *filename*. If
|
``Filter`` instance.
|
||||||
*include* is ``False``, don't trace memory blocks allocated in a
|
|
||||||
file with a name matching *filename*.
|
|
||||||
|
|
||||||
The match is done using *filename* as a prefix. For example,
|
All inclusive filters are applied at once, a memory allocation is
|
||||||
``'/usr/bin/'`` only matchs files the ``/usr/bin`` directories. The
|
only ignored if no inclusive filter match its trace. A memory
|
||||||
``.pyc`` and ``.pyo`` suffixes are automatically replaced with
|
allocation is ignored if at least one exclusive filter matchs its
|
||||||
``.py`` when matching the filename.
|
trace.
|
||||||
|
|
||||||
|
The new filter is not applied on already collected traces. Use
|
||||||
|
``clear_traces()`` to ensure that all traces match the new filter.
|
||||||
|
|
||||||
|
|
||||||
|
``add_include_filter(filename: str, lineno: int=None, traceback: bool=False)`` function:
|
||||||
|
|
||||||
|
Add an inclusive filter: helper for ``add_filter()`` creating a
|
||||||
|
``Filter`` instance with ``include`` attribute set to ``True``.
|
||||||
|
|
||||||
|
Example: ``tracemalloc.add_include_filter(tracemalloc.__file__)``
|
||||||
|
only includes memory blocks allocated by the ``tracemalloc`` module.
|
||||||
|
|
||||||
|
|
||||||
|
``add_exclude_filter(filename: str, lineno: int=None, traceback: bool=False)`` function:
|
||||||
|
|
||||||
|
Add an exclusive filter: helper for ``add_filter()`` creating a
|
||||||
|
``Filter`` instance with ``include`` attribute set to ``False``.
|
||||||
|
|
||||||
|
Example: ``tracemalloc.add_exclude_filter(tracemalloc.__file__)``
|
||||||
|
ignores memory blocks allocated by the ``tracemalloc`` module.
|
||||||
|
|
||||||
*lineno* is a line number. If *lineno* is ``None`` or lesser than
|
|
||||||
``1``, it matches any line number.
|
|
||||||
|
|
||||||
``clear_filters()`` function:
|
``clear_filters()`` function:
|
||||||
|
|
||||||
Reset the filter list.
|
Reset the filter list.
|
||||||
|
|
||||||
|
|
||||||
``clear_traces()`` function:
|
``clear_traces()`` function:
|
||||||
|
|
||||||
Clear all traces and statistics of memory allocations.
|
Clear all traces and statistics on Python memory allocations, and
|
||||||
|
reset the ``get_traced_memory()`` counter.
|
||||||
|
|
||||||
|
|
||||||
``disable()`` function:
|
``disable()`` function:
|
||||||
|
|
||||||
Stop tracing Python memory allocations and stop the timer started by
|
Stop tracing Python memory allocations and stop the timer started by
|
||||||
``start_timer()``.
|
``start_timer()``.
|
||||||
|
|
||||||
|
See also ``enable()`` and ``is_enabled()`` functions.
|
||||||
|
|
||||||
|
|
||||||
``enable()`` function:
|
``enable()`` function:
|
||||||
|
|
||||||
Start tracing Python memory allocations.
|
Start tracing Python memory allocations.
|
||||||
|
|
||||||
|
See also ``disable()`` and ``is_enabled()`` functions.
|
||||||
|
|
||||||
|
|
||||||
``get_filters()`` function:
|
``get_filters()`` function:
|
||||||
|
|
||||||
Get the filters as list of
|
Get the filters on Python memory allocations as list of ``Filter``
|
||||||
``(include: bool, filename: str, lineno: int)`` tuples.
|
instances.
|
||||||
|
|
||||||
If *lineno* is ``None``, a filter matchs any line number.
|
|
||||||
|
|
||||||
By default, the filename of the Python tracemalloc module
|
``get_traceback_limit()`` function:
|
||||||
(``tracemalloc.py``) is excluded.
|
|
||||||
|
|
||||||
``get_number_frame()`` function:
|
Get the maximum number of ``Frame`` instances stored in the
|
||||||
|
``traceback`` attribute of a ``Trace`` instance.
|
||||||
|
|
||||||
|
Use ``set_traceback_limit()`` to change the limit.
|
||||||
|
|
||||||
Get the maximum number of frames stored in a trace of a memory
|
|
||||||
allocation.
|
|
||||||
|
|
||||||
``get_object_address(obj)`` function:
|
``get_object_address(obj)`` function:
|
||||||
|
|
||||||
Get the address of the memory block of the specified Python object.
|
Get the address of the memory block of the specified Python object.
|
||||||
|
|
||||||
|
|
||||||
``get_object_trace(obj)`` function:
|
``get_object_trace(obj)`` function:
|
||||||
|
|
||||||
Get the trace of a Python object *obj* as a ``trace`` instance.
|
Get the trace of a Python object *obj* as a ``Trace`` instance.
|
||||||
|
|
||||||
|
The function only returns the trace of the memory block directly
|
||||||
|
holding to object. The ``size`` attribute of the trace is smaller
|
||||||
|
than the total size of the object if the object is composed of more
|
||||||
|
than one memory block.
|
||||||
|
|
||||||
|
Return ``None`` if the ``tracemalloc`` module did not trace the
|
||||||
|
allocation of the object.
|
||||||
|
|
||||||
|
See also ``gc.get_referrers()`` and ``sys.getsizeof()`` functions.
|
||||||
|
|
||||||
Return ``None`` if the tracemalloc module did not save the location
|
|
||||||
when the object was allocated, for example if the module was
|
|
||||||
disabled.
|
|
||||||
|
|
||||||
``get_process_memory()`` function:
|
``get_process_memory()`` function:
|
||||||
|
|
||||||
|
@ -153,334 +186,521 @@ Functions
|
||||||
|
|
||||||
Return ``None`` if the platform is not supported.
|
Return ``None`` if the platform is not supported.
|
||||||
|
|
||||||
Use the ``psutil`` module if available.
|
|
||||||
|
|
||||||
``get_stats()`` function:
|
``get_stats()`` function:
|
||||||
|
|
||||||
Get statistics on Python memory allocations per Python filename and
|
Get statistics on traced Python memory blocks as a dictionary
|
||||||
per Python line number.
|
``{filename (str): {line_number (int): stats}}`` where *stats* in a
|
||||||
|
``TraceStats`` instance, *filename* and *line_number* can be
|
||||||
|
``None``.
|
||||||
|
|
||||||
Return a dictionary
|
Return an empty dictionary if the ``tracemalloc`` module is
|
||||||
``{filename: str -> {line_number: int -> stats: line_stat}}``
|
disabled.
|
||||||
where *stats* in a ``line_stat`` instance. *filename* and
|
|
||||||
*line_number* can be ``None``.
|
|
||||||
|
``get_traced_memory()`` function:
|
||||||
|
|
||||||
|
Get the total size of all traced memory blocks allocated by Python.
|
||||||
|
|
||||||
Return an empty dictionary if the tracemalloc module is disabled.
|
|
||||||
|
|
||||||
``get_tracemalloc_size()`` function:
|
``get_tracemalloc_size()`` function:
|
||||||
|
|
||||||
Get the memory usage in bytes of the ``tracemalloc`` module.
|
Get the memory usage in bytes of the ``tracemalloc`` module.
|
||||||
|
|
||||||
|
|
||||||
``get_traces(obj)`` function:
|
``get_traces(obj)`` function:
|
||||||
|
|
||||||
Get all traces of a Python memory allocations.
|
Get all traces of Python memory allocations as a dictionary
|
||||||
Return a dictionary ``{pointer: int -> trace}`` where *trace*
|
``{address (int): trace}`` where *trace* is a ``Trace`` instance.
|
||||||
is a ``trace`` instance.
|
|
||||||
|
Return an empty dictionary if the ``tracemalloc`` module is
|
||||||
|
disabled.
|
||||||
|
|
||||||
Return an empty dictionary if the ``tracemalloc`` module is disabled.
|
|
||||||
|
|
||||||
``is_enabled()`` function:
|
``is_enabled()`` function:
|
||||||
|
|
||||||
Get the status of the module: ``True`` if it is enabled, ``False``
|
``True`` if the ``tracemalloc`` module is tracing Python memory
|
||||||
otherwise.
|
allocations, ``False`` otherwise.
|
||||||
|
|
||||||
``set_number_frame(nframe: int)`` function:
|
See also ``enable()`` and ``disable()`` functions.
|
||||||
|
|
||||||
Set the maximum number of frames stored in a trace of a memory
|
|
||||||
allocation.
|
|
||||||
|
|
||||||
All traces and statistics of memory allocations are cleared.
|
|
||||||
|
|
||||||
``start_timer(delay: int, func: callable, args: tuple=(), kwargs: dict={})`` function:
|
``start_timer(delay: int, func: callable, args: tuple=(), kwargs: dict={})`` function:
|
||||||
|
|
||||||
Start a timer calling ``func(*args, **kwargs)`` every *delay*
|
Start a timer calling ``func(*args, **kwargs)`` every *delay*
|
||||||
seconds.
|
seconds. Enable the ``tracemalloc`` module if it is disabled. The
|
||||||
|
timer is based on the Python memory allocator, it is not real time.
|
||||||
|
*func* is called after at least *delay* seconds, it is not called
|
||||||
|
exactly after *delay* seconds if no Python memory allocation
|
||||||
|
occurred. The timer has a resolution of 1 second.
|
||||||
|
|
||||||
The timer is based on the Python memory allocator, it is not real
|
If the ``start_timer()`` function is called twice, previous
|
||||||
time. *func* is called after at least *delay* seconds, it is not
|
parameters are replaced. Call the ``stop_timer()`` function to stop
|
||||||
called exactly after *delay* seconds if no Python memory allocation
|
the timer.
|
||||||
occurred.
|
|
||||||
|
|
||||||
If ``start_timer()`` is called twice, previous parameters are
|
The ``DisplayTopTask.start()`` and ``TakeSnapshot.start()`` methods
|
||||||
replaced. The timer has a resolution of 1 second.
|
use the ``start_timer()`` function to run regulary a task.
|
||||||
|
|
||||||
|
|
||||||
|
``set_traceback_limit(limit: int)`` function:
|
||||||
|
|
||||||
|
Set the maximum number of ``Frame`` instances stored in the
|
||||||
|
``traceback`` attribute of a ``Trace`` instance. Clear all traces
|
||||||
|
and statistics on Python memory allocations if the ``tracemalloc``
|
||||||
|
module is enabled,
|
||||||
|
|
||||||
|
Storing the traceback of each memory allocation has an important
|
||||||
|
overhead on the memory usage. Example with the Python test suite:
|
||||||
|
tracing all memory allocations increases the memory usage by
|
||||||
|
``+50%`` when storing only 1 frame and ``+150%`` when storing 10
|
||||||
|
frames. Use ``get_tracemalloc_size()`` to measure the overhead and
|
||||||
|
``add_filter()`` to select which memory allocations are traced.
|
||||||
|
|
||||||
|
Use ``get_traceback_limit()`` to get the current limit.
|
||||||
|
|
||||||
``start_timer()`` is used by ``DisplayTop`` and ``TakeSnapshot`` to
|
|
||||||
run regulary a task.
|
|
||||||
|
|
||||||
``stop_timer()`` function:
|
``stop_timer()`` function:
|
||||||
|
|
||||||
Stop the timer started by ``start_timer()``.
|
Stop the timer started by ``start_timer()``.
|
||||||
|
|
||||||
|
|
||||||
frame class
|
DisplayTop class
|
||||||
|
----------------
|
||||||
|
|
||||||
|
``DisplayTop()`` class:
|
||||||
|
|
||||||
|
Display the top of allocated memory blocks.
|
||||||
|
|
||||||
|
``display_snapshot(snapshot, count=10, group_by="filename_lineno", cumulative=False, file=None)`` method:
|
||||||
|
|
||||||
|
Display a snapshot of memory blocks allocated by Python, *snapshot*
|
||||||
|
is a ``Snapshot`` instance.
|
||||||
|
|
||||||
|
``display_top_diff(top_diff, count=10, file=None)`` method:
|
||||||
|
|
||||||
|
Display differences between two ``GroupedStats`` instances,
|
||||||
|
*top_diff* is a ``StatsDiff`` instance.
|
||||||
|
|
||||||
|
``display_top_stats(top_stats, count=10, file=None)`` method:
|
||||||
|
|
||||||
|
Display the top of allocated memory blocks grouped by the
|
||||||
|
``group_by`` attribute of *top_stats*, *top_stats* is a
|
||||||
|
``GroupedStats`` instance.
|
||||||
|
|
||||||
|
``color`` attribute:
|
||||||
|
|
||||||
|
If ``True``, always use colors. If ``False``, never use colors. The
|
||||||
|
default value is ``None``: use colors if the *file* parameter is a
|
||||||
|
TTY device.
|
||||||
|
|
||||||
|
``compare_with_previous`` attribute:
|
||||||
|
|
||||||
|
If ``True`` (default value), compare with the previous snapshot. If
|
||||||
|
``False``, compare with the first snapshot.
|
||||||
|
|
||||||
|
``filename_parts`` attribute:
|
||||||
|
|
||||||
|
Number of displayed filename parts (int, default: ``3``). Extra
|
||||||
|
parts are replaced with ``'...'``.
|
||||||
|
|
||||||
|
``show_average`` attribute:
|
||||||
|
|
||||||
|
If ``True`` (default value), display the average size of memory blocks.
|
||||||
|
|
||||||
|
``show_count`` attribute:
|
||||||
|
|
||||||
|
If ``True`` (default value), display the number of allocated memory
|
||||||
|
blocks.
|
||||||
|
|
||||||
|
``show_size`` attribute:
|
||||||
|
|
||||||
|
If ``True`` (default value), display the size of memory blocks.
|
||||||
|
|
||||||
|
|
||||||
|
DisplayTopTask class
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
``DisplayTopTask(count=10, group_by="filename_lineno", cumulative=False, file=sys.stdout, user_data_callback=None)`` class:
|
||||||
|
|
||||||
|
Task taking temporary snapshots and displaying the top *count*
|
||||||
|
memory allocations grouped by *group_by*.
|
||||||
|
|
||||||
|
Call the ``start()`` method to start the task.
|
||||||
|
|
||||||
|
``display()`` method:
|
||||||
|
|
||||||
|
Take a snapshot and display the top *count* biggest allocated memory
|
||||||
|
blocks grouped by *group_by* using the ``display_top`` attribute.
|
||||||
|
|
||||||
|
Return the snapshot, a ``Snapshot`` instance.
|
||||||
|
|
||||||
|
``start(delay: int)`` method:
|
||||||
|
|
||||||
|
Start a task using the ``start_timer()`` function calling the
|
||||||
|
``display()`` method every *delay* seconds.
|
||||||
|
|
||||||
|
``stop()`` method:
|
||||||
|
|
||||||
|
Stop the task started by the ``start()`` method using the
|
||||||
|
``stop_timer()`` function.
|
||||||
|
|
||||||
|
``count`` attribute:
|
||||||
|
|
||||||
|
Maximum number of displayed memory blocks.
|
||||||
|
|
||||||
|
``cumulative`` attribute:
|
||||||
|
|
||||||
|
If ``True``, cumulate size and count of memory blocks of all frames
|
||||||
|
of each ``Trace`` instance, not only the most recent frame. The
|
||||||
|
default value is ``False``.
|
||||||
|
|
||||||
|
The option is ignored if the traceback limit is ``1``, see the
|
||||||
|
``get_traceback_limit()`` function.
|
||||||
|
|
||||||
|
``display_top`` attribute:
|
||||||
|
|
||||||
|
Instance of ``DisplayTop``.
|
||||||
|
|
||||||
|
``file`` attribute:
|
||||||
|
|
||||||
|
The top is written into *file*.
|
||||||
|
|
||||||
|
``group_by`` attribute:
|
||||||
|
|
||||||
|
Determine how memory allocations are grouped: see
|
||||||
|
``Snapshot.top_by`` for the available values.
|
||||||
|
|
||||||
|
``user_data_callback`` attribute:
|
||||||
|
|
||||||
|
Optional callback collecting user data (callable, default:
|
||||||
|
``None``). See ``Snapshot.create()``.
|
||||||
|
|
||||||
|
|
||||||
|
Filter class
|
||||||
|
------------
|
||||||
|
|
||||||
|
``Filter(include: bool, pattern: str, lineno: int=None, traceback: bool=False)`` class:
|
||||||
|
|
||||||
|
Filter to select which memory allocations are traced. Filters can be
|
||||||
|
used to reduce the memory usage of the ``tracemalloc`` module, which
|
||||||
|
can be read using ``get_tracemalloc_size()``.
|
||||||
|
|
||||||
|
``match_trace(trace)`` method:
|
||||||
|
|
||||||
|
Return ``True`` if the ``Trace`` instance must be kept according to
|
||||||
|
the filter, ``False`` otherwise.
|
||||||
|
|
||||||
|
``match(filename: str, lineno: int)`` method:
|
||||||
|
|
||||||
|
Return ``True`` if the filename and line number must be kept
|
||||||
|
according to the filter, ``False`` otherwise.
|
||||||
|
|
||||||
|
``match_filename(filename: str)`` method:
|
||||||
|
|
||||||
|
Return ``True`` if the filename must be kept according to the
|
||||||
|
filter, ``False`` otherwise.
|
||||||
|
|
||||||
|
``match_lineno(lineno: int)`` method:
|
||||||
|
|
||||||
|
Return ``True`` if the line number must be kept according to the
|
||||||
|
filter, ``False`` otherwise.
|
||||||
|
|
||||||
|
``include`` attribute:
|
||||||
|
|
||||||
|
If *include* is ``True``, only trace memory blocks allocated in a
|
||||||
|
file with a name matching filename ``pattern`` at line number
|
||||||
|
``lineno``. If *include* is ``False``, ignore memory blocks
|
||||||
|
allocated in a file with a name matching filename :attr`pattern` at
|
||||||
|
line number ``lineno``.
|
||||||
|
|
||||||
|
``pattern`` attribute:
|
||||||
|
|
||||||
|
The filename *pattern* can contain one or many ``*`` joker
|
||||||
|
characters which match any substring, including an empty string. The
|
||||||
|
``.pyc`` and ``.pyo`` suffixes are replaced with ``.py``. On
|
||||||
|
Windows, the comparison is case insensitive and the alternative
|
||||||
|
separator ``/`` is replaced with the standard separator ``\``.
|
||||||
|
|
||||||
|
``lineno`` attribute:
|
||||||
|
|
||||||
|
Line number (``int``). If is is ``None`` or lesser than ``1``, it
|
||||||
|
matches any line number.
|
||||||
|
|
||||||
|
``traceback`` attribute:
|
||||||
|
|
||||||
|
If *traceback* is ``True``, all frames of the ``traceback``
|
||||||
|
attribute of ``Trace`` instances are checked. If *traceback* is
|
||||||
|
``False``, only the most recent frame is checked.
|
||||||
|
|
||||||
|
This attribute only has an effect on the ``match_trace()`` method
|
||||||
|
and only if the traceback limit is greater than ``1``. See the
|
||||||
|
``get_traceback_limit()`` function.
|
||||||
|
|
||||||
|
|
||||||
|
Frame class
|
||||||
-----------
|
-----------
|
||||||
|
|
||||||
``frame`` class:
|
``Frame`` class:
|
||||||
|
|
||||||
Trace of a Python frame.
|
Trace of a Python frame, used by ``Trace.traceback`` attribute.
|
||||||
|
|
||||||
``filename`` attribute (``str``):
|
``filename`` attribute:
|
||||||
|
|
||||||
Python filename, ``None`` if unknown.
|
Python filename, ``None`` if unknown.
|
||||||
|
|
||||||
``lineno`` attribute (``int``):
|
``lineno`` attribute:
|
||||||
|
|
||||||
Python line number, ``None`` if unknown.
|
Python line number, ``None`` if unknown.
|
||||||
|
|
||||||
|
|
||||||
trace class
|
GroupedStats class
|
||||||
-----------
|
------------------
|
||||||
|
|
||||||
``trace`` class:
|
``GroupedStats(stats: dict, group_by: str, cumulative=False, timestamp=None, process_memory=None, tracemalloc_size=None)`` class:
|
||||||
|
|
||||||
This class represents debug information of an allocated memory block.
|
Top of allocated memory blocks grouped by on *group_by* as a
|
||||||
|
dictionary.
|
||||||
|
|
||||||
``size`` attribute (``int``):
|
The ``Snapshot.top_by()`` method creates a ``GroupedStats`` instance.
|
||||||
|
|
||||||
Size in bytes of the memory block.
|
``compare_to(old_stats: GroupedStats=None)`` method:
|
||||||
|
|
||||||
``frames`` attribute (``list``):
|
Compare to an older ``GroupedStats`` instance. Return a
|
||||||
|
``StatsDiff`` instance.
|
||||||
|
|
||||||
Traceback where the memory block was allocated as a list of
|
``cumulative`` attribute:
|
||||||
``frame`` instances (most recent first).
|
|
||||||
|
|
||||||
The list can be empty or incomplete if the tracemalloc module was
|
If ``True``, cumulate size and count of memory blocks of all frames
|
||||||
unable to retrieve the full traceback.
|
of ``Trace``, not only the most recent frame.
|
||||||
|
|
||||||
For efficiency, the traceback is truncated to 10 frames.
|
``group_by`` attribute:
|
||||||
|
|
||||||
|
Determine how memory allocations were grouped. The type of ``stats``
|
||||||
|
keys depends on *group_by*:
|
||||||
|
|
||||||
line_stat class
|
===================== ======================== ==============
|
||||||
----------------
|
group_by description key type
|
||||||
|
===================== ======================== ==============
|
||||||
|
``'filename'`` filename ``str``
|
||||||
|
``'filename_lineno'`` filename and line number ``(str, str)``
|
||||||
|
``'address'`` memory block address ``int``
|
||||||
|
===================== ======================== ==============
|
||||||
|
|
||||||
``line_stat`` class:
|
See the *group_by* parameter of the ``Snapshot.top_by()`` method.
|
||||||
|
|
||||||
Statistics on Python memory allocations of a specific line number.
|
``stats`` attribute:
|
||||||
|
|
||||||
``size`` attribute (``int``):
|
Dictionary ``{key: stats}`` where the *key* type depends on the
|
||||||
|
``group_by`` attribute and *stats* type is ``TraceStats``.
|
||||||
Total size in bytes of all memory blocks allocated on the line.
|
|
||||||
|
|
||||||
``count`` attribute (``int``):
|
|
||||||
|
|
||||||
Number of memory blocks allocated on the line.
|
|
||||||
|
|
||||||
|
|
||||||
DisplayTop class
|
|
||||||
----------------
|
|
||||||
|
|
||||||
``DisplayTop(count: int=10, file=sys.stdout)`` class:
|
|
||||||
|
|
||||||
Display the list of the *count* biggest memory allocations into
|
|
||||||
*file*.
|
|
||||||
|
|
||||||
``display()`` method:
|
|
||||||
|
|
||||||
Display the top once.
|
|
||||||
|
|
||||||
``start(delay: int)`` method:
|
|
||||||
|
|
||||||
Start a task using ``tracemalloc`` timer to display the top every
|
|
||||||
*delay* seconds.
|
|
||||||
|
|
||||||
``stop()`` method:
|
|
||||||
|
|
||||||
Stop the task started by the ``DisplayTop.start()`` method
|
|
||||||
|
|
||||||
``color`` attribute (``bool``, default: ``file.isatty()``):
|
|
||||||
|
|
||||||
If ``True``, ``display()`` uses color.
|
|
||||||
|
|
||||||
``compare_with_previous`` attribute (``bool``, default: ``True``):
|
|
||||||
|
|
||||||
If ``True``, ``display()`` compares with the
|
|
||||||
previous snapshot. If ``False``, compare with the first snapshot.
|
|
||||||
|
|
||||||
``filename_parts`` attribute (``int``, default: ``3``):
|
|
||||||
|
|
||||||
Number of displayed filename parts. Extra parts are replaced
|
|
||||||
with ``"..."``.
|
|
||||||
|
|
||||||
``group_per_file`` attribute (``bool``, default: ``False``):
|
|
||||||
|
|
||||||
If ``True``, group memory allocations per Python filename. If
|
|
||||||
``False``, group allocation per Python line number.
|
|
||||||
|
|
||||||
``show_average`` attribute (``bool``, default: ``True``):
|
|
||||||
|
|
||||||
If ``True``, ``display()`` shows the average size
|
|
||||||
of allocations.
|
|
||||||
|
|
||||||
``show_count`` attribute (``bool``, default: ``True``):
|
|
||||||
|
|
||||||
If ``True``, ``display()`` shows the number of
|
|
||||||
allocations.
|
|
||||||
|
|
||||||
``show_size`` attribute (``bool``, default: ``True``):
|
|
||||||
|
|
||||||
If ``True``, ``display()`` shows the size of
|
|
||||||
allocations.
|
|
||||||
|
|
||||||
``user_data_callback`` attribute (``callable``, default: ``None``):
|
|
||||||
|
|
||||||
Optional callback collecting user data. See ``Snapshot.create()``.
|
|
||||||
|
|
||||||
|
|
||||||
Snapshot class
|
|
||||||
--------------
|
|
||||||
|
|
||||||
``Snapshot()`` class:
|
|
||||||
|
|
||||||
Snapshot of Python memory allocations.
|
|
||||||
|
|
||||||
Use ``TakeSnapshot`` to take regulary snapshots.
|
|
||||||
|
|
||||||
``create(user_data_callback=None)`` method:
|
|
||||||
|
|
||||||
Take a snapshot. If *user_data_callback* is specified, it must be a
|
|
||||||
callable object returning a list of
|
|
||||||
``(title: str, format: str, value: int)``.
|
|
||||||
*format* must be ``'size'``. The list must always have the same
|
|
||||||
length and the same order to be able to compute differences between
|
|
||||||
values.
|
|
||||||
|
|
||||||
Example: ``[('Video memory', 'size', 234902)]``.
|
|
||||||
|
|
||||||
``filter_filenames(patterns: list, include: bool)`` method:
|
|
||||||
|
|
||||||
Remove filenames not matching any pattern of *patterns* if *include*
|
|
||||||
is ``True``, or remove filenames matching a pattern of *patterns* if
|
|
||||||
*include* is ``False`` (exclude).
|
|
||||||
|
|
||||||
See ``fnmatch.fnmatch()`` for the syntax of a pattern.
|
|
||||||
|
|
||||||
``load(filename)`` classmethod:
|
|
||||||
|
|
||||||
Load a snapshot from a file.
|
|
||||||
|
|
||||||
``write(filename)`` method:
|
|
||||||
|
|
||||||
Write the snapshot into a file.
|
|
||||||
|
|
||||||
``pid`` attribute (``int``):
|
|
||||||
|
|
||||||
Identifier of the process which created the snapshot.
|
|
||||||
|
|
||||||
``process_memory`` attribute:
|
``process_memory`` attribute:
|
||||||
|
|
||||||
Result of the ``get_process_memory()`` function, can be ``None``.
|
Result of the ``get_process_memory()`` function, can be ``None``.
|
||||||
|
|
||||||
``stats`` attribute (``dict``):
|
``timestamp`` attribute:
|
||||||
|
|
||||||
Result of the ``get_stats()`` function.
|
Creation date and time of the snapshot, ``datetime.datetime``
|
||||||
|
instance.
|
||||||
|
|
||||||
``tracemalloc_size`` attribute (``int``):
|
``tracemalloc_size`` attribute:
|
||||||
|
|
||||||
The memory usage in bytes of the ``tracemalloc`` module,
|
The memory usage in bytes of the ``tracemalloc`` module, result of
|
||||||
result of the ``get_tracemalloc_size()`` function.
|
the ``get_tracemalloc_size()`` function.
|
||||||
|
|
||||||
``timestamp`` attribute (``datetime.datetime``):
|
|
||||||
|
|
||||||
Creation date and time of the snapshot.
|
|
||||||
|
|
||||||
``user_data`` attribute (``list``, default: ``None``):
|
|
||||||
|
|
||||||
Optional list of user data, result of *user_data_callback* in
|
|
||||||
``Snapshot.create()``.
|
|
||||||
|
|
||||||
|
|
||||||
TakeSnapshot class
|
Snapshot class
|
||||||
------------------
|
--------------
|
||||||
|
|
||||||
``TakeSnapshot`` class:
|
``Snapshot`` class:
|
||||||
|
|
||||||
Task taking snapshots of Python memory allocations: write them into
|
Snapshot of memory blocks allocated by Python.
|
||||||
files. By default, snapshots are written in the current directory.
|
|
||||||
|
|
||||||
``start(delay: int)`` method:
|
Use ``TakeSnapshot`` to take regulary snapshots.
|
||||||
|
|
||||||
Start a task taking a snapshot every delay seconds.
|
``apply_filters(filters)`` method:
|
||||||
|
|
||||||
``stop()`` method:
|
Apply a list filters on the ``traces`` and ``stats`` dictionaries,
|
||||||
|
*filters* is a list of ``Filter`` instances.
|
||||||
|
|
||||||
Stop the task started by the ``TakeSnapshot.start()`` method.
|
``create(\*, with_traces=False, with_stats=True, user_data_callback=None)`` classmethod:
|
||||||
|
|
||||||
``take_snapshot()`` method:
|
Take a snapshot of traces and/or statistics of allocated memory
|
||||||
|
blocks.
|
||||||
|
|
||||||
Take a snapshot.
|
If *with_traces* is ``True``, ``get_traces()`` is called and its
|
||||||
|
result is stored in the ``traces`` attribute. This attribute
|
||||||
|
contains more information than ``stats`` and uses more memory and
|
||||||
|
more disk space. If *with_traces* is ``False``, ``traces`` is set to
|
||||||
|
``None``.
|
||||||
|
|
||||||
``filename_template`` attribute (``str``,
|
If *with_stats* is ``True``, ``get_stats()`` is called and its
|
||||||
default: ``'tracemalloc-$counter.pickle'``):
|
result is stored in the ``Snapshot.stats`` attribute. If
|
||||||
|
*with_stats* is ``False``, ``Snapshot.stats`` is set to ``None``.
|
||||||
|
|
||||||
Template used to create a filename. The following variables can be
|
*with_traces* and *with_stats* cannot be ``False`` at the same time.
|
||||||
used in the template:
|
|
||||||
|
|
||||||
* ``$pid``: identifier of the current process
|
*user_data_callback* is an optional callable object. Its result
|
||||||
* ``$timestamp``: current date and time
|
should be serializable by the ``pickle`` module, or
|
||||||
* ``$counter``: counter starting at 1 and incremented at each snapshot
|
``Snapshot.write()`` would fail. If *user_data_callback* is set, it
|
||||||
|
is called and the result is stored in the ``Snapshot.user_data``
|
||||||
|
attribute. Otherwise, ``Snapshot.user_data`` is set to ``None``.
|
||||||
|
|
||||||
``user_data_callback`` attribute (``callable``, default: ``None``):
|
The ``tracemalloc`` module must be enabled to take a snapshot. See
|
||||||
|
the ``enable()`` function.
|
||||||
|
|
||||||
Optional callback collecting user data. See ``Snapshot.create()``.
|
``load(filename)`` classmethod:
|
||||||
|
|
||||||
|
Load a snapshot from a file.
|
||||||
|
|
||||||
|
``top_by(group_by: str, cumulative: bool=False)`` method:
|
||||||
|
|
||||||
|
Compute top statistics grouped by *group_by* as a ``GroupedStats``
|
||||||
|
instance:
|
||||||
|
|
||||||
|
===================== ======================== ==============
|
||||||
|
group_by description key type
|
||||||
|
===================== ======================== ==============
|
||||||
|
``'filename'`` filename ``str``
|
||||||
|
``'filename_lineno'`` filename and line number ``(str, str)``
|
||||||
|
``'address'`` memory block address ``int``
|
||||||
|
===================== ======================== ==============
|
||||||
|
|
||||||
|
If *cumulative* is ``True``, cumulate size and count of memory
|
||||||
|
blocks of all frames of each ``Trace`` instance, not only the most
|
||||||
|
recent frame. The *cumulative* parameter is ignored if *group_by* is
|
||||||
|
``'address'`` or if the traceback limit is ``1``. See the
|
||||||
|
``traceback_limit`` attribute.
|
||||||
|
|
||||||
|
``write(filename)`` method:
|
||||||
|
|
||||||
|
Write the snapshot into a file.
|
||||||
|
|
||||||
|
``pid`` attribute:
|
||||||
|
|
||||||
|
Identifier of the process which created the snapshot, result of
|
||||||
|
``os.getpid()``.
|
||||||
|
|
||||||
|
``process_memory`` attribute:
|
||||||
|
|
||||||
|
Memory usage of the current process, result of the
|
||||||
|
``get_process_memory()`` function. It can be ``None``.
|
||||||
|
|
||||||
|
``stats`` attribute:
|
||||||
|
|
||||||
|
Statistics on traced Python memory, result of the ``get_stats()``
|
||||||
|
function, if ``create()`` was called with *with_stats* equals to
|
||||||
|
``True``, ``None`` otherwise.
|
||||||
|
|
||||||
|
``tracemalloc_size`` attribute:
|
||||||
|
|
||||||
|
The memory usage in bytes of the ``tracemalloc`` module, result of
|
||||||
|
the ``get_tracemalloc_size()`` function.
|
||||||
|
|
||||||
|
``traceback_limit`` attribute:
|
||||||
|
|
||||||
|
The maximum number of frames stored in the ``traceback`` attribute
|
||||||
|
of a ``Trace``, result of the ``get_traceback_limit()`` function.
|
||||||
|
|
||||||
|
``traces`` attribute:
|
||||||
|
|
||||||
|
Traces of Python memory allocations, result of the ``get_traces()``
|
||||||
|
function, if ``create()`` was called with *with_traces* equals to
|
||||||
|
``True``, ``None`` otherwise.
|
||||||
|
|
||||||
|
The ``traceback`` attribute of each ``Trace`` instance is limited to
|
||||||
|
``traceback_limit`` frames.
|
||||||
|
|
||||||
|
``timestamp`` attribute:
|
||||||
|
|
||||||
|
Creation date and time of the snapshot, ``datetime.datetime``
|
||||||
|
instance.
|
||||||
|
|
||||||
|
``user_data`` attribute:
|
||||||
|
|
||||||
|
Result of *user_data_callback* called in ``Snapshot.create()``
|
||||||
|
(default: ``None``).
|
||||||
|
|
||||||
|
|
||||||
Command line options
|
StatsDiff class
|
||||||
====================
|
---------------
|
||||||
|
|
||||||
The ``python -m tracemalloc`` command can be used to analyze and compare
|
``StatsDiff(differences, old_stats, new_stats)`` class:
|
||||||
snapshots. The command takes a list of snapshot filenames and has the
|
|
||||||
following options.
|
|
||||||
|
|
||||||
``-g``, ``--group-per-file``
|
Differences between two ``GroupedStats`` instances. By default, the
|
||||||
|
``differences`` list is unsorted: call ``sort()`` to sort it.
|
||||||
|
|
||||||
Group allocations per filename, instead of grouping per line number.
|
The ``GroupedStats.compare_to()`` method creates a ``StatsDiff``
|
||||||
|
instance.
|
||||||
|
|
||||||
``-n NTRACES``, ``--number NTRACES``
|
``sort()`` method:
|
||||||
|
|
||||||
Number of traces displayed per top (default: 10).
|
Sort the ``differences`` list from the biggest allocation to the
|
||||||
|
smallest. Sort by *size_diff*, *size*, *count_diff*, *count* and
|
||||||
|
then by *key*.
|
||||||
|
|
||||||
``--first``
|
``differences`` attribute:
|
||||||
|
|
||||||
Compare with the first snapshot, instead of comparing with the
|
Differences between ``old_stats`` and ``new_stats`` as a list of
|
||||||
previous snapshot.
|
``(size_diff, size, count_diff, count, key)`` tuples. *size_diff*,
|
||||||
|
*size*, *count_diff* and *count* are ``int``. The key type depends
|
||||||
|
on the ``group_by`` attribute of ``new_stats``:
|
||||||
|
|
||||||
``--include PATTERN``
|
===================== ======================== ==============
|
||||||
|
group_by description key type
|
||||||
|
===================== ======================== ==============
|
||||||
|
``'filename'`` filename ``str``
|
||||||
|
``'filename_lineno'`` filename and line number ``(str, str)``
|
||||||
|
``'address'`` memory block address ``int``
|
||||||
|
===================== ======================== ==============
|
||||||
|
|
||||||
Only include filenames matching pattern *PATTERN*. The option can be
|
See the ``group_by`` attribute of the ``GroupedStats`` class.
|
||||||
specified multiple times.
|
|
||||||
|
|
||||||
See ``fnmatch.fnmatch()`` for the syntax of patterns.
|
``old_stats`` attribute:
|
||||||
|
|
||||||
``--exclude PATTERN``
|
Old ``GroupedStats`` instance, can be ``None``.
|
||||||
|
|
||||||
Exclude filenames matching pattern *PATTERN*. The option can be
|
``new_stats`` attribute:
|
||||||
specified multiple times.
|
|
||||||
|
|
||||||
See ``fnmatch.fnmatch()`` for the syntax of patterns.
|
New ``GroupedStats`` instance.
|
||||||
|
|
||||||
``-S``, ``--hide-size``
|
|
||||||
|
|
||||||
Hide the size of allocations.
|
Trace class
|
||||||
|
-----------
|
||||||
|
|
||||||
``-C``, ``--hide-count``
|
``Trace`` class:
|
||||||
|
|
||||||
Hide the number of allocations.
|
Debug information of a memory block allocated by Python.
|
||||||
|
|
||||||
``-A``, ``--hide-average``
|
``size`` attribute:
|
||||||
|
|
||||||
Hide the average size of allocations.
|
Size in bytes of the memory block.
|
||||||
|
|
||||||
``-P PARTS``, ``--filename-parts=PARTS``
|
``traceback`` attribute:
|
||||||
|
|
||||||
Number of displayed filename parts (default: 3).
|
Traceback where the memory block was allocated as a list of
|
||||||
|
``Frame`` instances, most recent first.
|
||||||
|
|
||||||
``--color``
|
The list can be empty or incomplete if the ``tracemalloc`` module
|
||||||
|
was unable to retrieve the full traceback.
|
||||||
|
|
||||||
Force usage of colors even if ``sys.stdout`` is not a TTY device.
|
The traceback is limited to ``get_traceback_limit()`` frames. Use
|
||||||
|
``set_traceback_limit()`` to store more frames.
|
||||||
|
|
||||||
``--no-color``
|
|
||||||
|
|
||||||
Disable colors if ``sys.stdout`` is a TTY device.
|
TraceStats class
|
||||||
|
----------------
|
||||||
|
|
||||||
|
``TraceStats`` class:
|
||||||
|
|
||||||
|
Statistics on Python memory allocations.
|
||||||
|
|
||||||
|
``size`` attribute:
|
||||||
|
|
||||||
|
Total size in bytes of allocated memory blocks.
|
||||||
|
|
||||||
|
``count`` attribute:
|
||||||
|
|
||||||
|
Number of allocated memory blocks.
|
||||||
|
|
||||||
|
|
||||||
Links
|
Links
|
||||||
|
|
Loading…
Reference in New Issue