PEP 657: Add a new rejected idea section for single-caret errors (#1965)

pep-0657.rst

@@ -343,18 +343,46 @@ A reference implementation can be found in the implementation_ fork.
 Rejected Ideas
 ==============
 
-Include end line number
-^^^^^^^^^^^^^^^^^^^^^^^
-Some instructions can span across multiple lines and therefore the end offset
-and the start offset can be located in two different lines. We have decided to
-set the value for the start offset to the correct value and set a value of 0 to
-the end offset. This will result in highlighting the entire line starting from
-the value of the starting offset. The reason behind this decision is that
-storing the end line will require us to store another field similar to
-``co_lnotab``, but our traceback machinery only highlights a single line
-per frame so this information would only be used to decide to highlight to the
-end of the line. On the other hand, the end line could be useful for other
-tools such as coverage-measuring tools and tracers.
+Use a single caret instead of a range
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+It has been proposed to use a single caret instead of highlighting the full
+range when reporting errors as a way to simplify the feature. We have decided
+to not go this route for the following reasons:
+
+* Deriving the location of the caret is not straightforward using the current
+  layout of the AST. This is because the AST nodes only record the start and end
+  line numbers as well as the start and end column offsets. As the AST nodes do
+  not preserve the original tokens (by design) deriving the exact location of some
+  tokens is not possible without extra re-parsing. For instance, currently binary
+  operators have nodes for the operands but the type of the operator is stored
+  in an enumeration so its location cannot be derived from the node (this is just
+  an example of how this problem manifest, and not the only one).
+* Deriving the ranges from AST nodes greatly simplifies the implementation and reduces
+  a lot the maintenance cost and the possibilities of errors. This is because using
+  the ranges is always possible to do generically for any AST node, while any other
+  custom information would need to be extracted differently from different types of
+  nodes. Given how error-prone getting the locations manually was when this used to
+  be a manual process when generating the AST, we believe that a generic solution is
+  a very important property to pursue.
+* Storing the information to highlight a single caret will be very limiting for tools
+  such as coverage tools and profilers as well as for tools like IPython and IDEs that
+  want to make use of this new feature. As `this message <https://discuss.python.org/t/pep-657-include-fine-grained-error-locations-in-tracebacks/8629/2?u=pablogsal>`_ from the author of "friendly-traceback"
+  mentions, the reason is that without the full range (including end lines) these tools
+  will find very difficult to highlight correctly the relevant source code. For instance,
+  for this code::
+
+    something = foo(a,b,c) if bar(a,b,c) else other(b,c,d)
+
+  tools (such as coverage reporters) want to be able to highlight the totality of the call
+  that is covered by the executed bytecode (let's say ``foo(a,b,c)``) and not just a single
+  character.  Even if is technically possible to re-parse and re-tokenize the source code
+  to re-construct the information, it is not possible to do this reliably and would
+  result in a much worse user experience.
+* Many users have reported that a single caret is much harder to read than a full range,
+  and this motivated using ranges to highlight syntax errors, which was very well received.
+  Additionally, it has been noted that users with vision problems can identify the ranges
+  much easily than a single caret character, which we believe is a great advantage of
+  using ranges.
 
 Have a configure flag to opt out
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^