Merge
This commit is contained in:
Brett Cannon
commit
18608c96cb
|
|
@ -0,0 +1,313 @@
|
||||||
|
PEP: 490
|
||||||
|
Title: Chain exceptions at C level
|
||||||
|
Version: $Revision$
|
||||||
|
Last-Modified: $Date$
|
||||||
|
Author: Victor Stinner <victor.stinner@gmail.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 25-March-2015
|
||||||
|
Python-Version: 3.5
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
Chain exceptions at C level, as already done at Python level.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
Python 3 introduced a new killer feature: exceptions are chained by
|
||||||
|
default, PEP 3134.
|
||||||
|
|
||||||
|
Example::
|
||||||
|
|
||||||
|
try:
|
||||||
|
raise TypeError("err1")
|
||||||
|
except TypeError:
|
||||||
|
raise ValueError("err2")
|
||||||
|
|
||||||
|
Output::
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 2, in <module>
|
||||||
|
raise TypeError("err1")
|
||||||
|
TypeError: err1
|
||||||
|
|
||||||
|
During handling of the above exception, another exception occurred:
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "test.py", line 4, in <module>
|
||||||
|
raise ValueError("err2")
|
||||||
|
ValueError: err2
|
||||||
|
|
||||||
|
Exceptions are chained by default in Python code, but not in
|
||||||
|
extensions written in C.
|
||||||
|
|
||||||
|
A new private ``_PyErr_ChainExceptions()`` function was introduced in
|
||||||
|
Python 3.4.3 and 3.5 to chain exceptions. Currently, it must be called
|
||||||
|
explicitly to chain exceptions and its usage is not trivial.
|
||||||
|
|
||||||
|
Example of ``_PyErr_ChainExceptions()`` usage from the ``zipimport``
|
||||||
|
module to chain the previous ``OSError`` to a new ``ZipImportError``
|
||||||
|
exception::
|
||||||
|
|
||||||
|
PyObject *exc, *val, *tb;
|
||||||
|
PyErr_Fetch(&exc, &val, &tb);
|
||||||
|
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
|
||||||
|
_PyErr_ChainExceptions(exc, val, tb);
|
||||||
|
|
||||||
|
This PEP proposes to also chain exceptions automatically at C level to
|
||||||
|
stay consistent and give more information on failures to help
|
||||||
|
debugging. The previous example becomes simply::
|
||||||
|
|
||||||
|
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
|
||||||
|
|
||||||
|
|
||||||
|
Proposal
|
||||||
|
========
|
||||||
|
|
||||||
|
Modify PyErr_*() functions to chain exceptions
|
||||||
|
----------------------------------------------
|
||||||
|
|
||||||
|
Modify C functions raising exceptions of the Python C API to
|
||||||
|
automatically chain exceptions: modify ``PyErr_SetString()``,
|
||||||
|
``PyErr_Format()``, ``PyErr_SetNone()``, etc.
|
||||||
|
|
||||||
|
|
||||||
|
Modify functions to not chain exceptions
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
Keeping the previous exception is not always interesting when the new
|
||||||
|
exception contains information of the previous exception or even more
|
||||||
|
information, especially when the two exceptions have the same type.
|
||||||
|
|
||||||
|
Example of an useless exception chain with ``int(str)``::
|
||||||
|
|
||||||
|
TypeError: a bytes-like object is required, not 'type'
|
||||||
|
|
||||||
|
During handling of the above exception, another exception occurred:
|
||||||
|
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "<stdin>", line 1, in <module>
|
||||||
|
TypeError: int() argument must be a string, a bytes-like object or a number, not 'type'
|
||||||
|
|
||||||
|
The new ``TypeError`` exception contains more information than the
|
||||||
|
previous exception. The previous exception should be hidden.
|
||||||
|
|
||||||
|
The ``PyErr_Clear()`` function can be called to clear the current
|
||||||
|
exception before raising a new exception, to not chain the current
|
||||||
|
exception with a new exception.
|
||||||
|
|
||||||
|
|
||||||
|
Modify functions to chain exceptions
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
Some functions save and then restore the current exception. If a new
|
||||||
|
exception is raised, the exception is currently displayed into
|
||||||
|
sys.stderr or ignored depending on the function. Some of these
|
||||||
|
functions should be modified to chain exceptions instead.
|
||||||
|
|
||||||
|
Examples of function ignoring the new exception(s):
|
||||||
|
|
||||||
|
* ``ptrace_enter_call()``: ignore exception
|
||||||
|
* ``subprocess_fork_exec()``: ignore exception raised by enable_gc()
|
||||||
|
* ``t_bootstrap()`` of the ``_thread`` module: ignore exception raised
|
||||||
|
by trying to display the bootstrap function to ``sys.stderr``
|
||||||
|
* ``PyDict_GetItem()``, ``_PyDict_GetItem_KnownHash()``: ignore
|
||||||
|
exception raised by looking for a key in the dictionary
|
||||||
|
* ``_PyErr_TrySetFromCause()``: ignore exception
|
||||||
|
* ``PyFrame_LocalsToFast()``: ignore exception raised by
|
||||||
|
``dict_to_map()``
|
||||||
|
* ``_PyObject_Dump()``: ignore exception. ``_PyObject_Dump()`` is used
|
||||||
|
to debug, to inspect a running process, it should not modify the
|
||||||
|
Python state.
|
||||||
|
* ``Py_ReprLeave()``: ignore exception "because there is no way to
|
||||||
|
report them"
|
||||||
|
* ``type_dealloc()``: ignore exception raised by
|
||||||
|
``remove_all_subclasses()``
|
||||||
|
* ``PyObject_ClearWeakRefs()``: ignore exception?
|
||||||
|
* ``call_exc_trace()``, ``call_trace_protected()``: ignore exception
|
||||||
|
* ``remove_importlib_frames()``: ignore exception
|
||||||
|
* ``do_mktuple()``, helper used by ``Py_BuildValue()`` for example:
|
||||||
|
ignore exception?
|
||||||
|
* ``flush_io()``: ignore exception
|
||||||
|
* ``sys_write()``, ``sys_format()``: ignore exception
|
||||||
|
* ``_PyTraceback_Add()``: ignore exception
|
||||||
|
* ``PyTraceBack_Print()``: ignore exception
|
||||||
|
|
||||||
|
Examples of function displaying the new exception to ``sys.stderr``:
|
||||||
|
|
||||||
|
* ``atexit_callfuncs()``: display exceptions with
|
||||||
|
``PyErr_Display()`` and return the latest exception, the function
|
||||||
|
calls multiple callbacks and only returns the latest exception
|
||||||
|
* ``sock_dealloc()``: log the ``ResourceWarning`` exception with
|
||||||
|
``PyErr_WriteUnraisable()``
|
||||||
|
* ``slot_tp_del()``: display exception with
|
||||||
|
``PyErr_WriteUnraisable()``
|
||||||
|
* ``_PyGen_Finalize()``: display ``gen_close()`` exception with
|
||||||
|
``PyErr_WriteUnraisable()``
|
||||||
|
* ``slot_tp_finalize()``: display exception raised by the
|
||||||
|
``__del__()`` method with ``PyErr_WriteUnraisable()``
|
||||||
|
* ``PyErr_GivenExceptionMatches()``: display exception raised by
|
||||||
|
``PyType_IsSubtype()`` with ``PyErr_WriteUnraisable()``
|
||||||
|
|
||||||
|
|
||||||
|
Backward compatibility
|
||||||
|
======================
|
||||||
|
|
||||||
|
A side effect of chaining exceptions is that exceptions store
|
||||||
|
traceback objects which store frame objects which store local
|
||||||
|
variables. Local variables are kept alive by exceptions. A common
|
||||||
|
issue is a reference cycle between local variables and exceptions: an
|
||||||
|
exception is stored in a local variable and the frame indirectly
|
||||||
|
stored in the exception. The cycle only impacts applications storing
|
||||||
|
exceptions.
|
||||||
|
|
||||||
|
The reference cycle can now be fixed with the new
|
||||||
|
``traceback.TracebackException`` object introduced in Python 3.5. It
|
||||||
|
stores informations required to format a full textual traceback without
|
||||||
|
storing local variables.
|
||||||
|
|
||||||
|
The ``asyncio`` is impacted by the reference cycle issue. This module
|
||||||
|
is also maintained outside Python standard library to release a
|
||||||
|
version for Python 3.3. ``traceback.TracebackException`` will maybe
|
||||||
|
be backported in a private ``asyncio`` module to fix reference cycle
|
||||||
|
issues.
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
============
|
||||||
|
|
||||||
|
No change
|
||||||
|
---------
|
||||||
|
|
||||||
|
A new private ``_PyErr_ChainExceptions()`` function is enough to chain
|
||||||
|
manually exceptions.
|
||||||
|
|
||||||
|
Exceptions will only be chained explicitly where it makes sense.
|
||||||
|
|
||||||
|
|
||||||
|
New helpers to chain exceptions
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
Functions like ``PyErr_SetString()`` don't chain automatically
|
||||||
|
exceptions. To make the usage of ``_PyErr_ChainExceptions()`` easier,
|
||||||
|
new private functions are added:
|
||||||
|
|
||||||
|
* ``_PyErr_SetStringChain(exc_type, message)``
|
||||||
|
* ``_PyErr_FormatChain(exc_type, format, ...)``
|
||||||
|
* ``_PyErr_SetNoneChain(exc_type)``
|
||||||
|
* ``_PyErr_SetObjectChain(exc_type, exc_value)``
|
||||||
|
|
||||||
|
Helper functions to raise specific exceptions like
|
||||||
|
``_PyErr_SetKeyError(key)`` or ``PyErr_SetImportError(message, name,
|
||||||
|
path)`` don't chain exceptions. The generic
|
||||||
|
``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)`` should be used
|
||||||
|
to chain exceptions with these helper functions.
|
||||||
|
|
||||||
|
|
||||||
|
Appendix
|
||||||
|
========
|
||||||
|
|
||||||
|
PEPs
|
||||||
|
----
|
||||||
|
|
||||||
|
* `PEP 3134 -- Exception Chaining and Embedded Tracebacks
|
||||||
|
<https://www.python.org/dev/peps/pep-3134/>`_ (Python 3.0):
|
||||||
|
new ``__context__`` and ``__cause__`` attributes for exceptions
|
||||||
|
* `PEP 415 - Implement context suppression with exception attributes
|
||||||
|
<https://www.python.org/dev/peps/pep-0415/>`_ (Python 3.3):
|
||||||
|
``raise exc from None``
|
||||||
|
* `PEP 409 - Suppressing exception context
|
||||||
|
<https://www.python.org/dev/peps/pep-0409/>`_
|
||||||
|
(superseded by the PEP 415)
|
||||||
|
|
||||||
|
|
||||||
|
Python C API
|
||||||
|
------------
|
||||||
|
|
||||||
|
The header file ``Include/pyerror.h`` declares functions related to
|
||||||
|
exceptions.
|
||||||
|
|
||||||
|
Functions raising exceptions:
|
||||||
|
|
||||||
|
* ``PyErr_SetNone(exc_type)``
|
||||||
|
* ``PyErr_SetObject(exc_type, exc_value)``
|
||||||
|
* ``PyErr_SetString(exc_type, message)``
|
||||||
|
* ``PyErr_Format(exc, format, ...)``
|
||||||
|
|
||||||
|
Helpers to raise specific exceptions:
|
||||||
|
|
||||||
|
* ``PyErr_BadArgument()``
|
||||||
|
* ``PyErr_BadInternalCall()``
|
||||||
|
* ``PyErr_NoMemory()``
|
||||||
|
* ``PyErr_SetFromErrno(exc)``
|
||||||
|
* ``PyErr_SetFromWindowsErr(err)``
|
||||||
|
* ``PyErr_SetImportError(message, name, path)``
|
||||||
|
* ``_PyErr_SetKeyError(key)``
|
||||||
|
* ``_PyErr_TrySetFromCause(prefix_format, ...)``
|
||||||
|
|
||||||
|
Manage the current exception:
|
||||||
|
|
||||||
|
* ``PyErr_Clear()``: clear the current exception,
|
||||||
|
like ``except: pass``
|
||||||
|
* ``PyErr_Fetch(exc_type, exc_value, exc_tb)``
|
||||||
|
* ``PyErr_Restore(exc_type, exc_value, exc_tb)``
|
||||||
|
* ``PyErr_GetExcInfo(exc_type, exc_value, exc_tb)``
|
||||||
|
* ``PyErr_SetExcInfo(exc_type, exc_value, exc_tb)``
|
||||||
|
|
||||||
|
Others function to handle exceptions:
|
||||||
|
|
||||||
|
* ``PyErr_ExceptionMatches(exc)``: check to implement
|
||||||
|
``except exc: ...``
|
||||||
|
* ``PyErr_GivenExceptionMatches(exc1, exc2)``
|
||||||
|
* ``PyErr_NormalizeException(exc_type, exc_value, exc_tb)``
|
||||||
|
* ``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)``
|
||||||
|
|
||||||
|
|
||||||
|
Python Issues
|
||||||
|
-------------
|
||||||
|
|
||||||
|
Chain exceptions:
|
||||||
|
|
||||||
|
* `Issue #23763: Chain exceptions in C
|
||||||
|
<http://bugs.python.org/issue23763>`_
|
||||||
|
* `Issue #23696: zipimport: chain ImportError to OSError
|
||||||
|
<http://bugs.python.org/issue23696>`_
|
||||||
|
* `Issue #21715: Chaining exceptions at C level
|
||||||
|
<http://bugs.python.org/issue21715>`_: added
|
||||||
|
``_PyErr_ChainExceptions()``
|
||||||
|
* `Issue #18488: sqlite: finalize() method of user function may be
|
||||||
|
called with an exception set if a call to step() method failed
|
||||||
|
<http://bugs.python.org/issue18488>`_
|
||||||
|
* `Issue #23781: Add private _PyErr_ReplaceException() in 2.7
|
||||||
|
<http://bugs.python.org/issue23781>`_
|
||||||
|
* `Issue #23782: Leak in _PyTraceback_Add
|
||||||
|
<http://bugs.python.org/issue23782>`_
|
||||||
|
|
||||||
|
Changes preventing to loose exceptions:
|
||||||
|
|
||||||
|
* `Issue #23571: Raise SystemError if a function returns a result with an
|
||||||
|
exception set <http://bugs.python.org/issue23571>`_
|
||||||
|
* `Issue #18408: Fixes crashes found by pyfailmalloc
|
||||||
|
<http://bugs.python.org/issue18408>`_
|
||||||
|
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
||||||
Loading…
Reference in New Issue