PEP 657: Include fine grained error locations in tracebacks (GH-1950)
This commit is contained in:
Pablo Galindo
GitHub
parent
fae0ce2014
commit
822755724f
|
|
@ -0,0 +1,405 @@
|
||||||
|
PEP: 657
|
||||||
|
Title: Include Fine Grained Error Locations in Tracebacks
|
||||||
|
Version: $Revision$
|
||||||
|
Last-Modified: $Date$
|
||||||
|
Author: Pablo Galindo <pablogsal@python.org>,
|
||||||
|
Batuhan Taskaya <batuhan@python.org>,
|
||||||
|
Ammar Askar <ammar@ammaraskar.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 08-May-2021
|
||||||
|
Python-Version: 3.11
|
||||||
|
Post-History:
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
This PEP proposes adding a mapping from each bytecode instruction to the start
|
||||||
|
and end column offsets of the line that generated them. This data will be used
|
||||||
|
to improve tracebacks displayed by the CPython interpreter in order to improve
|
||||||
|
the debugging experience. The PEP also proposes adding APIs that allow other
|
||||||
|
tools (such as coverage analysis tools, profilers, tracers, debuggers) to
|
||||||
|
consume this information from code objects.
|
||||||
|
|
||||||
|
Motivation
|
||||||
|
==========
|
||||||
|
|
||||||
|
The primary motivation for this PEP is to improve the feedback presented about the location of errors to aid with debugging.
|
||||||
|
|
||||||
|
Python currently keeps a mapping of bytecode to line numbers from compilation.
|
||||||
|
The interpreter uses this mapping to point to the source line associated with
|
||||||
|
an error. While this line-level granularity for instructions is useful, a
|
||||||
|
single line of Python code can compile into dozens of bytecode operations
|
||||||
|
making it hard to track which part of the line caused the error.
|
||||||
|
|
||||||
|
Consider the following line of Python code::
|
||||||
|
|
||||||
|
x['a']['b']['c']['d'] = 1
|
||||||
|
|
||||||
|
If any of the values in the dictionaries are ``None``, the error shown is::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 2, in <module>
|
||||||
|
x['a']['b']['c']['d'] = 1
|
||||||
|
TypeError: 'NoneType' object is not subscriptable
|
||||||
|
|
||||||
|
From the traceback, it is impossible to determine which one of the dictionaries
|
||||||
|
had the ``None`` element that caused the error. Users often have to attach a
|
||||||
|
debugger or split up their expression to track down the problem.
|
||||||
|
|
||||||
|
However, if the interpreter had a mapping of bytecode to column offsets as well
|
||||||
|
as line numbers, it could helpfully display::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 2, in <module>
|
||||||
|
x['a']['b']['c']['d'] = 1
|
||||||
|
^^^^^^^^^^^^^^^^
|
||||||
|
TypeError: 'NoneType' object is not subscriptable
|
||||||
|
|
||||||
|
indicating to the user that the object ``x['a']['b']`` must have been ``None``.
|
||||||
|
This highlighting will occur for every frame in the traceback. For instance, if
|
||||||
|
a similar error is part of a complex function call chain, the traceback would
|
||||||
|
display the code associated to the current instruction in every frame::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 14, in <module>
|
||||||
|
lel3(x)
|
||||||
|
^^^^^^^
|
||||||
|
File "test.py", line 12, in lel3
|
||||||
|
return lel2(x) / 23
|
||||||
|
^^^^^^^
|
||||||
|
File "test.py", line 9, in lel2
|
||||||
|
return 25 + lel(x) + lel(x)
|
||||||
|
^^^^^^
|
||||||
|
File "test.py", line 6, in lel
|
||||||
|
return 1 + foo(a,b,c=x['z']['x']['y']['z']['y'], d=e)
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
TypeError: 'NoneType' object is not subscriptable
|
||||||
|
|
||||||
|
This problem presents itself in the following situations.
|
||||||
|
|
||||||
|
* When passing down multiple objects to function calls while
|
||||||
|
accessing the same attribute in them.
|
||||||
|
For instance, this error::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 19, in <module>
|
||||||
|
foo(a.name, b.name, c.name)
|
||||||
|
AttributeError: 'NoneType' object has no attribute 'name'
|
||||||
|
|
||||||
|
With the improvements in this PEP this would show::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 17, in <module>
|
||||||
|
foo(a.name, b.name, c.name)
|
||||||
|
^^^^^^
|
||||||
|
AttributeError: 'NoneType' object has no attribute 'name'
|
||||||
|
|
||||||
|
* When dealing with lines with complex mathematical expressions,
|
||||||
|
especially with libraries such as numpy where arithmetic
|
||||||
|
operations can fail based on the arguments. For example: ::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 1, in <module>
|
||||||
|
x = (a + b) @ (c + d)
|
||||||
|
ValueError: operands could not be broadcast together with shapes (1,2) (2,3)
|
||||||
|
|
||||||
|
There is no clear indication as to which operation failed, was it the addition
|
||||||
|
on the left, the right or the matrix multiplication in the middle? With this
|
||||||
|
PEP the new error message would look like::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 1, in <module>
|
||||||
|
x = (a + b) @ (c + d)
|
||||||
|
^^^^^
|
||||||
|
ValueError: operands could not be broadcast together with shapes (1,2) (2,3)
|
||||||
|
|
||||||
|
Giving a much clearer and easier to debug error message.
|
||||||
|
|
||||||
|
|
||||||
|
Debugging aside, this extra information would also be useful for code
|
||||||
|
coverage tools, enabling them to measure expression-level coverage instead of
|
||||||
|
just line-level coverage. For instance, given the following line: ::
|
||||||
|
|
||||||
|
x = foo() if bar() else baz()
|
||||||
|
|
||||||
|
coverage, profile or state analysis tools will highlight the full line in both
|
||||||
|
branches, making it impossible to differentiate what branch was taken. This is
|
||||||
|
a known problem in pycoverage_.
|
||||||
|
|
||||||
|
Similar efforts to this PEP have taken place in other languages such as Java in
|
||||||
|
the form of JEP358_. ``NullPointerExceptions`` in Java were similarly nebulous when
|
||||||
|
it came to lines with complicated expressions. A ``NullPointerException`` would
|
||||||
|
provide very little aid in finding the root cause of an error. The
|
||||||
|
implementation for JEP358 is fairly complex, requiring walking back through the
|
||||||
|
bytecode by using a control flow graph analyzer and decompilation techniques to
|
||||||
|
recover the source code that led to the null pointer. Although the complexity
|
||||||
|
of this solution is high and requires maintenance for the decompiler every time
|
||||||
|
Java bytecode is changed, this improvement was deemed to be worth it for the
|
||||||
|
extra information provided for *just one exception type*.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
In order to identify the range of source code being executed when exceptions
|
||||||
|
are raised, this proposal requires adding new data for every bytecode
|
||||||
|
instruction. This will have an impact on the size of ``pyc`` files on disk and
|
||||||
|
the size of code objects in memory. The authors of this proposal have chosen
|
||||||
|
the data types in a way that tries to minimize this impact. The proposed
|
||||||
|
overhead is storing two ``uint8_t`` (one for the start offset and one for the
|
||||||
|
end offset) for every bytecode instruction.
|
||||||
|
|
||||||
|
As an illustrative example to gauge the impact of this change, we have
|
||||||
|
calculated that this change will increase the size of the standard library’s
|
||||||
|
pyc files by 22% (6MB) from 70MB to 76MB. The overhead in memory usage will be
|
||||||
|
the same (assuming the *full standard library* is loaded into the same
|
||||||
|
program). We believe that this is a very acceptable number since the order of
|
||||||
|
magnitude of the overhead is very small, especially considering the storage
|
||||||
|
size and memory capabilities of modern computers. Additionally, in general the
|
||||||
|
memory size of a Python program is not dominated by code objects. To check this
|
||||||
|
assumption we have executed the test suite of several popular PyPI projects
|
||||||
|
(including NumPy, pytest, Django and Cython) as well as several applications
|
||||||
|
(Black, pylint, mypy executed over either mypy or the standard library) and we
|
||||||
|
found that code objects represent normally 3-6% of the average memory size of
|
||||||
|
the program.
|
||||||
|
|
||||||
|
We understand that the extra cost of this information may not be acceptable for
|
||||||
|
some users, so we propose an opt-out mechanism when Python is executed in
|
||||||
|
optimized mode (``python -O``), which will cause pyo files to not include the
|
||||||
|
extra information.
|
||||||
|
|
||||||
|
|
||||||
|
Specification
|
||||||
|
=============
|
||||||
|
|
||||||
|
In order to have enough information to correctly resolve the location within a
|
||||||
|
given line where an error was raised, a map linking bytecode instructions and
|
||||||
|
column offsets (start and end offset) is needed. This is similar in fashion to
|
||||||
|
how line numbers are currently linked to bytecode instructions.
|
||||||
|
|
||||||
|
The following changes will be performed as part of the implementation of this PEP:
|
||||||
|
|
||||||
|
* The offset information will be exposed to Python via a new attribute in the
|
||||||
|
code object class called ``co_col_offsets`` that will return a sequence of
|
||||||
|
two-element tuples (containing the start offsets and end offsets) or None if
|
||||||
|
the code object was created without the offset information.
|
||||||
|
* Two new C-API functions, ``PyCode_Addr2StartOffset`` and
|
||||||
|
``PyCode_Addr2EndOffset`` will be added that can obtain the start and end
|
||||||
|
offsets respectively given the index of a bytecode instruction. These
|
||||||
|
functions will return 0 if the offset information is not available.
|
||||||
|
* A new private (underscore prefixed) C-API constructor for code objects will
|
||||||
|
be added that takes a bytes object containing the start offsets in the even
|
||||||
|
position and the end offsets in the odd positions. Old constructors will be
|
||||||
|
left untouched for backwards compatibility and will create code objects
|
||||||
|
without the new field.
|
||||||
|
|
||||||
|
Offset semantics
|
||||||
|
^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
These offsets are propagated by the compiler from the ones stored currently in
|
||||||
|
all AST nodes. They are 1-indexed and a value of 0 will mean that the
|
||||||
|
information is not available. Although the AST nodes use ``int`` types to store
|
||||||
|
these values, ``uint8_t`` types will be used for storage in the new map to
|
||||||
|
minimize storage impact. This decision allows offsets to go from 0 to 255,
|
||||||
|
while offsets bigger than these values will be treated as missing (value of 0).
|
||||||
|
We believe this is an acceptable compromise as line lengths in Python tend to
|
||||||
|
be much lower than this limit (a query of the top 100 packages in PyPI shows
|
||||||
|
that less than 0.01% of lines were longer than 255 characters).
|
||||||
|
|
||||||
|
Maintaining the current behavior, only a single line will be displayed in
|
||||||
|
tracebacks. For instructions that span multiple lines (the end offset and the
|
||||||
|
start offset belong to different lines), the end offset will be set to 0
|
||||||
|
(meaning it is unavailable). If the start offset is not 0, this will be
|
||||||
|
interpreted by the displaying code as if the range spans from the starting
|
||||||
|
offset to the end of the line. The actual end offset cannot be calculated at
|
||||||
|
compile time since the compiler does not know how many characters “the end of
|
||||||
|
the line” actually represents.
|
||||||
|
|
||||||
|
Displaying tracebacks
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
When displaying tracebacks, the default exception hook will be modified to
|
||||||
|
query this information from the code objects and use it to display a sequence
|
||||||
|
of carets for every displayed line in the traceback if the information is
|
||||||
|
available. For instance::
|
||||||
|
|
||||||
|
File "test.py", line 6, in lel
|
||||||
|
return 1 + foo(a,b,c=x['z']['x']['y']['z']['y'], d=e)
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
TypeError: 'NoneType' object is not subscriptable
|
||||||
|
|
||||||
|
When displaying tracebacks, instruction offsets will be taken from the
|
||||||
|
traceback objects. This makes highlighting exceptions that are re-raised work
|
||||||
|
naturally without the need to store the new information in the stack. For
|
||||||
|
example, for this code::
|
||||||
|
|
||||||
|
def foo(x):
|
||||||
|
1 + 1/0 + 2
|
||||||
|
|
||||||
|
def bar(x):
|
||||||
|
try:
|
||||||
|
1 + foo(x) + foo(x)
|
||||||
|
except Exception as e:
|
||||||
|
raise ValueError("oh no!") from e
|
||||||
|
|
||||||
|
bar(bar(bar(2)))
|
||||||
|
|
||||||
|
The printed traceback would look like this::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 6, in bar
|
||||||
|
1 + foo(x) + foo(x)
|
||||||
|
^^^^^^
|
||||||
|
File "test.py", line 2, in foo
|
||||||
|
1 + 1/0 + 2
|
||||||
|
^^^
|
||||||
|
ZeroDivisionError: division by zero
|
||||||
|
|
||||||
|
The above exception was the direct cause of the following exception:
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 10, in <module>
|
||||||
|
bar(bar(bar(2)))
|
||||||
|
^^^^^^
|
||||||
|
File "test.py", line 8, in bar
|
||||||
|
raise ValueError("oh no!") from e
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
ValueError: oh no
|
||||||
|
|
||||||
|
While this code::
|
||||||
|
|
||||||
|
def foo(x):
|
||||||
|
1 + 1/0 + 2
|
||||||
|
def bar(x):
|
||||||
|
try:
|
||||||
|
1 + foo(x) + foo(x)
|
||||||
|
except Exception:
|
||||||
|
raise
|
||||||
|
bar(bar(bar(2)))
|
||||||
|
|
||||||
|
Will be displayed as::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 10, in <module>
|
||||||
|
bar(bar(bar(2)))
|
||||||
|
^^^^^^
|
||||||
|
File "test.py", line 6, in bar
|
||||||
|
1 + foo(x) + foo(x)
|
||||||
|
^^^^^^
|
||||||
|
File "test.py", line 2, in foo
|
||||||
|
1 + 1/0 + 2
|
||||||
|
^^^
|
||||||
|
ZeroDivisionError: division by zero
|
||||||
|
|
||||||
|
|
||||||
|
Opt-out mechanism
|
||||||
|
^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
To offer an opt-out mechanism for those users that care about the storage and
|
||||||
|
memory overhead, the functionality will be deactivated along with the extra
|
||||||
|
information when Python is executed in optimized mode (``python -O``) resulting
|
||||||
|
in ``pyo`` files not having the overhead associated with the extra required
|
||||||
|
data.
|
||||||
|
|
||||||
|
To allow third party tools and other programs that are currently parsing
|
||||||
|
tracebacks to catch up and to allow users to deactivate the new feature, the
|
||||||
|
following methods will be provided to deactivate displaying the new highlight
|
||||||
|
carets (but not to avoid to storing the data, users will need to use Python in
|
||||||
|
optimized mode for that):
|
||||||
|
|
||||||
|
* A new environment variable: ``PY_DEACTIVATE_TRACEBACK_RANGES``
|
||||||
|
* A new command line option for the dev mode: ``python -Xnotracebackranges``.
|
||||||
|
|
||||||
|
These flags will be removed in the next version of the Python interpreter
|
||||||
|
(counting from the version that releases this feature).
|
||||||
|
|
||||||
|
Backwards Compatibility
|
||||||
|
=======================
|
||||||
|
|
||||||
|
The change is fully backwards compatible.
|
||||||
|
|
||||||
|
|
||||||
|
Reference Implementation
|
||||||
|
========================
|
||||||
|
|
||||||
|
A reference implementation can be found in the implementation_ fork.
|
||||||
|
|
||||||
|
Rejected Ideas
|
||||||
|
==============
|
||||||
|
|
||||||
|
Include end line number
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
Some instructions can span across multiple lines and therefore the end offset
|
||||||
|
and the start offset can be located in two different lines. We have decided to
|
||||||
|
set the value for the start offset to the correct value and set a value of 0 to
|
||||||
|
the end offset. This will result in highlighting the entire line starting from
|
||||||
|
the value of the starting offset. The reason behind this decision is that
|
||||||
|
storing the end line will require us to store another field similar to
|
||||||
|
``co_lnotab``, but our traceback machinery only highlights a single line
|
||||||
|
per frame so this information would only be used to decide to highlight to the
|
||||||
|
end of the line. On the other hand, the end line could be useful for other
|
||||||
|
tools such as coverage-measuring tools and tracers.
|
||||||
|
|
||||||
|
Have a configure flag to opt out
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
Having a configure flag to opt out of the overhead even when executing Python
|
||||||
|
in non-optimized mode may sound desirable, but it may cause problems when
|
||||||
|
reading pyc files that were created with a version of the interpreter that was
|
||||||
|
not compiled with the flag activated. This can lead to crashes that would be
|
||||||
|
very difficult to debug for regular users and will make different pyc files
|
||||||
|
incompatible between each other. As this pyc could be shipped as part of
|
||||||
|
libraries or applications without the original source, it is also not always
|
||||||
|
possible to force recompilation of said pyc files. For these reasons we have
|
||||||
|
decided to use the -O flag to opt-out of this behaviour.
|
||||||
|
|
||||||
|
Lazy loading of column information
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
One potential solution to reduce the memory usage of this feature is to not
|
||||||
|
load the column information from the pyc file when code is imported. Only if an
|
||||||
|
uncaught exception bubbles up or if a call to the C-API functions is made will
|
||||||
|
the column information be loaded from the pyc file. This is similar to how we
|
||||||
|
only read source lines to display them in the traceback when an exception
|
||||||
|
bubbles up. While this would indeed lower memory usage, it also results in a
|
||||||
|
far more complex implementation requiring changes to the importing machinery to
|
||||||
|
selectively ignore a part of the code object. We consider this an interesting
|
||||||
|
avenue to explore but ultimately we think is out of the scope for this particular
|
||||||
|
PEP. It also means that column information will not be available if the user is
|
||||||
|
not using pyc files or for code objects created dynamically at runtime.
|
||||||
|
|
||||||
|
Implement compression
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
Although it would be possible to implement some form of compression over the
|
||||||
|
pyc files and the new data in code objects, we believe that this is out of the
|
||||||
|
scope of this proposal due to its larger impact (in the case of pyc files) and
|
||||||
|
the fact that we expect column offsets to not compress well due to the lack of
|
||||||
|
patterns in them (in case of the new data in code objects).
|
||||||
|
|
||||||
|
Acknowledgments
|
||||||
|
===============
|
||||||
|
Thanks to Carl Friedrich Bolz-Tereick for showing an initial prototype of this
|
||||||
|
idea for the Pypy interpreter and for the helpful discussion.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. _JEP358: https://openjdk.java.net/jeps/358
|
||||||
|
.. _implementation: https://github.com/colnotab/cpython/tree/bpo-43950
|
||||||
|
.. _pycoverage: https://github.com/nedbat/coveragepy/issues/509
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document is placed in the public domain or under the
|
||||||
|
CC0-1.0-Universal license, whichever is more permissive.
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
||||||
Loading…
Reference in New Issue