PEP 490
This commit is contained in:
Victor Stinner
parent
d1d9f7a914
commit
32ad4c5b64
334
pep-0490.txt
334
pep-0490.txt
|
|
@ -19,8 +19,8 @@ Chain exceptions at C level, as already done at Python level.
|
|||
Rationale
|
||||
=========
|
||||
|
||||
Python 3 introduced a new killer feature: exceptions are chained by default,
|
||||
PEP 3134.
|
||||
Python 3 introduced a new killer feature: exceptions are chained by
|
||||
default, PEP 3134.
|
||||
|
||||
Example::
|
||||
|
||||
|
|
@ -43,9 +43,27 @@ Output::
|
|||
raise ValueError("err2")
|
||||
ValueError: err2
|
||||
|
||||
Exceptions are chained by default in Python code, but not in extensions written
|
||||
in C. This PEP proposes to also chain exceptions automatically at C level to
|
||||
stay consistent.
|
||||
Exceptions are chained by default in Python code, but not in
|
||||
extensions written in C.
|
||||
|
||||
A new private ``_PyErr_ChainExceptions()`` function was introduced in
|
||||
Python 3.4.3 and 3.5 to chain exceptions. Currently, it must be called
|
||||
explicitly to chain exceptions and its usage is not trivial.
|
||||
|
||||
Example of ``_PyErr_ChainExceptions()`` usage from the ``zipimport``
|
||||
module to chain the previous ``OSError`` to a new ``ZipImportError``
|
||||
exception::
|
||||
|
||||
PyObject *exc, *val, *tb;
|
||||
PyErr_Fetch(&exc, &val, &tb);
|
||||
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
|
||||
_PyErr_ChainExceptions(exc, val, tb);
|
||||
|
||||
This PEP proposes to also chain exceptions automatically at C level to
|
||||
stay consistent and give more information on failures to help
|
||||
debugging. The previous example becomes simply::
|
||||
|
||||
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
|
||||
|
||||
|
||||
Proposal
|
||||
|
|
@ -54,141 +72,110 @@ Proposal
|
|||
Modify PyErr_*() functions to chain exceptions
|
||||
----------------------------------------------
|
||||
|
||||
Modify C functions raising exceptions of the Python C API to automatically
|
||||
chain exceptions.
|
||||
Modify C functions raising exceptions of the Python C API to
|
||||
automatically chain exceptions: modify ``PyErr_SetString()``,
|
||||
``PyErr_Format()``, ``PyErr_SetNone()``, etc.
|
||||
|
||||
|
||||
Modify functions to not chain exceptions
|
||||
----------------------------------------
|
||||
|
||||
To remove the previous exception, PyErr_Clear() can be called before
|
||||
PyErr_SetString() (or other functions raising exceptions).
|
||||
Keeping the previous exception is not always interesting when the new
|
||||
exception contains information of the previous exception or even more
|
||||
information, especially when the two exceptions have the same type.
|
||||
|
||||
Keeping the original exception is not always interesting.
|
||||
Example of an useless exception chain with ``int(str)``::
|
||||
|
||||
Example of Python code (``os._Environ.__getitem__``) suppressing the context::
|
||||
|
||||
try:
|
||||
value = self._data[self.encodekey(key)]
|
||||
except KeyError:
|
||||
# raise KeyError with the original key value
|
||||
raise KeyError(key) from None
|
||||
|
||||
Example of an exception chain raised by the struct module with exceptions
|
||||
chained by default::
|
||||
|
||||
$ python3.5 -c 'import struct; struct.pack("b", 2**100)'
|
||||
OverflowError: Python int too large to convert to C long
|
||||
TypeError: a bytes-like object is required, not 'type'
|
||||
|
||||
During handling of the above exception, another exception occurred:
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "<string>", line 1, in <module>
|
||||
struct.error: argument out of range
|
||||
File "<stdin>", line 1, in <module>
|
||||
TypeError: int() argument must be a string, a bytes-like object or a number, not 'type'
|
||||
|
||||
The ``OverflowError`` exception is not very useful, it can be hidden.
|
||||
The new ``TypeError`` exception contains more information than the
|
||||
previous exception. The previous exception should be hidden.
|
||||
|
||||
The ``PyErr_Clear()`` function can be called to clear the current
|
||||
exception before raising a new exception, to not chain the current
|
||||
exception with a new exception.
|
||||
|
||||
|
||||
Modify functions to chain exceptions
|
||||
------------------------------------
|
||||
|
||||
Some functions save and then restore the current exception. If a new exception
|
||||
is raised, the exception is displayed or ignored depending on the function.
|
||||
|
||||
Maybe some of these functions should be modified to chain exceptions instead?
|
||||
|
||||
Examples of function displaying the new exception:
|
||||
|
||||
* atexit_callfuncs(): display exceptions with PyErr_Display() and return the
|
||||
latest exception
|
||||
* sock_dealloc(): log warning exception with PyErr_WriteUnraisable()
|
||||
* slot_tp_del(): display exception with PyErr_WriteUnraisable()
|
||||
* _PyGen_Finalize(): display gen_close() exception with PyErr_WriteUnraisable()
|
||||
* slot_tp_finalize(): display exception raised by __del__() with
|
||||
PyErr_WriteUnraisable()
|
||||
* PyErr_GivenExceptionMatches(): display exception raised by PyType_IsSubtype()
|
||||
with PyErr_WriteUnraisable()
|
||||
Some functions save and then restore the current exception. If a new
|
||||
exception is raised, the exception is currently displayed into
|
||||
sys.stderr or ignored depending on the function. Some of these
|
||||
functions should be modified to chain exceptions instead.
|
||||
|
||||
Examples of function ignoring the new exception(s):
|
||||
|
||||
* ptrace_enter_call(): ignore exception
|
||||
* subprocess_fork_exec(): ignore exception raised by enable_gc()
|
||||
* t_bootstrap() of the _thread module: ignore raised exception when trying to
|
||||
display the bootstrap function to sys.stderr
|
||||
* PyDict_GetItem(), _PyDict_GetItem_KnownHash(): ignore exception raiesd when
|
||||
looking for a key
|
||||
* _PyErr_TrySetFromCause(): ignore ecxeption
|
||||
* PyFrame_LocalsToFast(): ignore exception raised by dict_to_map()
|
||||
* _PyObject_Dump(): ignore exception. _PyObject_Dump() is used to debug, to
|
||||
inspect a running process, it should not modify the Python state.
|
||||
* Py_ReprLeave(): ignore exception "because there is no way to report them"
|
||||
* type_dealloc(): ignore exception raised by remove_all_subclasses()
|
||||
* PyObject_ClearWeakRefs(): ignore exceptions?
|
||||
* call_exc_trace(), call_trace_protected(): ignore exception
|
||||
* remove_importlib_frames(): ignore exception
|
||||
* do_mktuple(), helped used by Py_BuildValue() for example: ignore exception?
|
||||
* flush_io(): ignore exception
|
||||
* sys_write(), sys_format(): ignore exception
|
||||
* _PyTraceback_Add(): ignore exception
|
||||
* PyTraceBack_Print(): ignore exception
|
||||
* ``ptrace_enter_call()``: ignore exception
|
||||
* ``subprocess_fork_exec()``: ignore exception raised by enable_gc()
|
||||
* ``t_bootstrap()`` of the ``_thread`` module: ignore exception raised
|
||||
by trying to display the bootstrap function to ``sys.stderr``
|
||||
* ``PyDict_GetItem()``, ``_PyDict_GetItem_KnownHash()``: ignore
|
||||
exception raised by looking for a key in the dictionary
|
||||
* ``_PyErr_TrySetFromCause()``: ignore exception
|
||||
* ``PyFrame_LocalsToFast()``: ignore exception raised by
|
||||
``dict_to_map()``
|
||||
* ``_PyObject_Dump()``: ignore exception. ``_PyObject_Dump()`` is used
|
||||
to debug, to inspect a running process, it should not modify the
|
||||
Python state.
|
||||
* ``Py_ReprLeave()``: ignore exception "because there is no way to
|
||||
report them"
|
||||
* ``type_dealloc()``: ignore exception raised by
|
||||
``remove_all_subclasses()``
|
||||
* ``PyObject_ClearWeakRefs()``: ignore exception?
|
||||
* ``call_exc_trace()``, ``call_trace_protected()``: ignore exception
|
||||
* ``remove_importlib_frames()``: ignore exception
|
||||
* ``do_mktuple()``, helper used by ``Py_BuildValue()`` for example:
|
||||
ignore exception?
|
||||
* ``flush_io()``: ignore exception
|
||||
* ``sys_write()``, ``sys_format()``: ignore exception
|
||||
* ``_PyTraceback_Add()``: ignore exception
|
||||
* ``PyTraceBack_Print()``: ignore exception
|
||||
|
||||
Examples of function displaying the new exception to ``sys.stderr``:
|
||||
|
||||
Python C API
|
||||
------------
|
||||
|
||||
Include/pyerror.h declares functions related to exceptions.
|
||||
|
||||
Functions raising exceptions:
|
||||
|
||||
* PyErr_SetNone(exc_type)
|
||||
* PyErr_SetObject(exc_type, exc_value)
|
||||
* PyErr_SetString(exc_type, message)
|
||||
* PyErr_Format(exc, format, ...)
|
||||
|
||||
Helpers to raise specific exceptions:
|
||||
|
||||
* PyErr_BadArgument()
|
||||
* PyErr_BadInternalCall()
|
||||
* PyErr_NoMemory()
|
||||
* PyErr_SetFromErrno(exc)
|
||||
* PyErr_SetFromWindowsErr(err)
|
||||
* PyErr_SetImportError(message, name, path)
|
||||
* _PyErr_SetKeyError(key)
|
||||
* _PyErr_TrySetFromCause(prefix_format, ...)
|
||||
|
||||
Manage the current exception:
|
||||
|
||||
* PyErr_Clear(): clear the current exception, like "except: pass"
|
||||
* PyErr_Fetch(exc_type, exc_value, exc_tb)
|
||||
* PyErr_Restore(exc_type, exc_value, exc_tb)
|
||||
* PyErr_GetExcInfo(exc_type, exc_value, exc_tb)
|
||||
* PyErr_SetExcInfo(exc_type, exc_value, exc_tb)
|
||||
|
||||
Others function to handle exceptions:
|
||||
|
||||
* PyErr_ExceptionMatches(exc): check to implement "except exc: ..."
|
||||
* PyErr_GivenExceptionMatches(exc1, exc2)
|
||||
* PyErr_NormalizeException(exc_type, exc_value, exc_tb)
|
||||
* _PyErr_ChainExceptions(exc_type, exc_value, exc_tb)
|
||||
* ``atexit_callfuncs()``: display exceptions with
|
||||
``PyErr_Display()`` and return the latest exception, the function
|
||||
calls multiple callbacks and only returns the latest exception
|
||||
* ``sock_dealloc()``: log the ``ResourceWarning`` exception with
|
||||
``PyErr_WriteUnraisable()``
|
||||
* ``slot_tp_del()``: display exception with
|
||||
``PyErr_WriteUnraisable()``
|
||||
* ``_PyGen_Finalize()``: display ``gen_close()`` exception with
|
||||
``PyErr_WriteUnraisable()``
|
||||
* ``slot_tp_finalize()``: display exception raised by the
|
||||
``__del__()`` method with ``PyErr_WriteUnraisable()``
|
||||
* ``PyErr_GivenExceptionMatches()``: display exception raised by
|
||||
``PyType_IsSubtype()`` with ``PyErr_WriteUnraisable()``
|
||||
|
||||
|
||||
Backward compatibility
|
||||
======================
|
||||
|
||||
A side effect of chaining exceptions is that exceptions store traceback
|
||||
objects. Traceback objects store frame objects which store local variables.
|
||||
Local variables are kept alive by exceptions. A common issue is a reference
|
||||
cycle between local variables and exceptions. The cycle only impacts
|
||||
applications storing exceptions.
|
||||
A side effect of chaining exceptions is that exceptions store
|
||||
traceback objects which store frame objects which store local
|
||||
variables. Local variables are kept alive by exceptions. A common
|
||||
issue is a reference cycle between local variables and exceptions: an
|
||||
exception is stored in a local variable and the frame indirectly
|
||||
stored in the exception. The cycle only impacts applications storing
|
||||
exceptions.
|
||||
|
||||
The reference cycle can now be fixed with the new traceback.TracebackException
|
||||
object introduced in Python 3.5 which store informations required to format a
|
||||
full textual traceback without storing local variables.
|
||||
The reference cycle can now be fixed with the new
|
||||
``traceback.TracebackException`` object introduced in Python 3.5. It
|
||||
stores informations required to format a full textual traceback without
|
||||
storing local variables.
|
||||
|
||||
The asyncio is impacted by the reference cycle issue, but this module is also
|
||||
maintained outside Python standard library to release a version for Python 3.3.
|
||||
traceback.TracebackException will probably be backward in a private asyncio
|
||||
module to fix the issue.
|
||||
The ``asyncio`` is impacted by the reference cycle issue. This module
|
||||
is also maintained outside Python standard library to release a
|
||||
version for Python 3.3. ``traceback.TracebackException`` will maybe
|
||||
be backported in a private ``asyncio`` module to fix reference cycle
|
||||
issues.
|
||||
|
||||
|
||||
Alternatives
|
||||
|
|
@ -197,54 +184,34 @@ Alternatives
|
|||
No change
|
||||
---------
|
||||
|
||||
Python 3.5 introduces a new private ``_PyErr_ChainExceptions()`` function which
|
||||
is enough to chain manually exceptions.
|
||||
A new private ``_PyErr_ChainExceptions()`` function is enough to chain
|
||||
manually exceptions.
|
||||
|
||||
Exceptions will only be chained explicitly where it makes sense.
|
||||
|
||||
|
||||
New helpers to chain exceptions
|
||||
-------------------------------
|
||||
|
||||
Functions like ``PyErr_SetString()`` don't chain automatically exceptions. To
|
||||
make usage of ``_PyErr_ChainExceptions()`` easier, new functions are added:
|
||||
Functions like ``PyErr_SetString()`` don't chain automatically
|
||||
exceptions. To make the usage of ``_PyErr_ChainExceptions()`` easier,
|
||||
new private functions are added:
|
||||
|
||||
* PyErr_SetStringChain(exc_type, message)
|
||||
* PyErr_FormatChaine(exc_type, format, ...)
|
||||
* PyErr_SetNoneChain(exc_type)
|
||||
* PyErr_SetObjectChain(exc_type, exc_value)
|
||||
* ``_PyErr_SetStringChain(exc_type, message)``
|
||||
* ``_PyErr_FormatChain(exc_type, format, ...)``
|
||||
* ``_PyErr_SetNoneChain(exc_type)``
|
||||
* ``_PyErr_SetObjectChain(exc_type, exc_value)``
|
||||
|
||||
Helper functions like _PyErr_SetKeyError(key) or PyErr_SetImportError(message,
|
||||
name, path) don't chain exceptions. The generic
|
||||
_PyErr_ChainExceptions(exc_type, exc_value, exc_tb) should be used.
|
||||
Helper functions to raise specific exceptions like
|
||||
``_PyErr_SetKeyError(key)`` or ``PyErr_SetImportError(message, name,
|
||||
path)`` don't chain exceptions. The generic
|
||||
``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)`` should be used
|
||||
to chain exceptions with these helper functions.
|
||||
|
||||
|
||||
Appendix
|
||||
========
|
||||
|
||||
Python Issues
|
||||
-------------
|
||||
|
||||
Chain exceptions:
|
||||
|
||||
* `Issue #23763: Chain exceptions in C
|
||||
<http://bugs.python.org/issue23763>`_
|
||||
* `Issue #23696: zipimport: chain ImportError to OSError
|
||||
<http://bugs.python.org/issue23696>`_
|
||||
* `Issue #21715: Chaining exceptions at C level
|
||||
<http://bugs.python.org/issue21715>`_: added _PyErr_ChainExceptions()
|
||||
* `Issue #18488: sqlite: finalize() method of user function may be called with
|
||||
an exception set if a call to step() method failed
|
||||
<http://bugs.python.org/issue18488>`_
|
||||
* `Issue #23781: Add private _PyErr_ReplaceException() in 2.7
|
||||
<http://bugs.python.org/issue23781>`_
|
||||
|
||||
Changes preventing loosing exceptions:
|
||||
|
||||
* `Issue #23571: Raise SystemError if a function returns a result with an
|
||||
exception set <http://bugs.python.org/issue23571>`_
|
||||
* `Issue #18408: Fixes crashes found by pyfailmalloc
|
||||
<http://bugs.python.org/issue18408>`_
|
||||
|
||||
|
||||
PEPs
|
||||
----
|
||||
|
||||
|
|
@ -255,7 +222,78 @@ PEPs
|
|||
<https://www.python.org/dev/peps/pep-0415/>`_ (Python 3.3):
|
||||
``raise exc from None``
|
||||
* `PEP 409 - Suppressing exception context
|
||||
<https://www.python.org/dev/peps/pep-0409/>`_ (superseded by the PEP 415)
|
||||
<https://www.python.org/dev/peps/pep-0409/>`_
|
||||
(superseded by the PEP 415)
|
||||
|
||||
|
||||
Python C API
|
||||
------------
|
||||
|
||||
The header file ``Include/pyerror.h`` declares functions related to
|
||||
exceptions.
|
||||
|
||||
Functions raising exceptions:
|
||||
|
||||
* ``PyErr_SetNone(exc_type)``
|
||||
* ``PyErr_SetObject(exc_type, exc_value)``
|
||||
* ``PyErr_SetString(exc_type, message)``
|
||||
* ``PyErr_Format(exc, format, ...)``
|
||||
|
||||
Helpers to raise specific exceptions:
|
||||
|
||||
* ``PyErr_BadArgument()``
|
||||
* ``PyErr_BadInternalCall()``
|
||||
* ``PyErr_NoMemory()``
|
||||
* ``PyErr_SetFromErrno(exc)``
|
||||
* ``PyErr_SetFromWindowsErr(err)``
|
||||
* ``PyErr_SetImportError(message, name, path)``
|
||||
* ``_PyErr_SetKeyError(key)``
|
||||
* ``_PyErr_TrySetFromCause(prefix_format, ...)``
|
||||
|
||||
Manage the current exception:
|
||||
|
||||
* ``PyErr_Clear()``: clear the current exception,
|
||||
like ``except: pass``
|
||||
* ``PyErr_Fetch(exc_type, exc_value, exc_tb)``
|
||||
* ``PyErr_Restore(exc_type, exc_value, exc_tb)``
|
||||
* ``PyErr_GetExcInfo(exc_type, exc_value, exc_tb)``
|
||||
* ``PyErr_SetExcInfo(exc_type, exc_value, exc_tb)``
|
||||
|
||||
Others function to handle exceptions:
|
||||
|
||||
* ``PyErr_ExceptionMatches(exc)``: check to implement
|
||||
``except exc: ...``
|
||||
* ``PyErr_GivenExceptionMatches(exc1, exc2)``
|
||||
* ``PyErr_NormalizeException(exc_type, exc_value, exc_tb)``
|
||||
* ``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)``
|
||||
|
||||
|
||||
Python Issues
|
||||
-------------
|
||||
|
||||
Chain exceptions:
|
||||
|
||||
* `Issue #23763: Chain exceptions in C
|
||||
<http://bugs.python.org/issue23763>`_
|
||||
* `Issue #23696: zipimport: chain ImportError to OSError
|
||||
<http://bugs.python.org/issue23696>`_
|
||||
* `Issue #21715: Chaining exceptions at C level
|
||||
<http://bugs.python.org/issue21715>`_: added
|
||||
``_PyErr_ChainExceptions()``
|
||||
* `Issue #18488: sqlite: finalize() method of user function may be
|
||||
called with an exception set if a call to step() method failed
|
||||
<http://bugs.python.org/issue18488>`_
|
||||
* `Issue #23781: Add private _PyErr_ReplaceException() in 2.7
|
||||
<http://bugs.python.org/issue23781>`_
|
||||
* `Issue #23782: Leak in _PyTraceback_Add
|
||||
<http://bugs.python.org/issue23782>`_
|
||||
|
||||
Changes preventing to loose exceptions:
|
||||
|
||||
* `Issue #23571: Raise SystemError if a function returns a result with an
|
||||
exception set <http://bugs.python.org/issue23571>`_
|
||||
* `Issue #18408: Fixes crashes found by pyfailmalloc
|
||||
<http://bugs.python.org/issue18408>`_
|
||||
|
||||
|
||||
Copyright
|
||||
|
|
|
|||
Loading…
Reference in New Issue