python-peps/pep-0348.txt

PEP: 348
Title: Exception Reorganization for Python 3.0
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Jul-2005
Post-History: 03-Aug-2005


Abstract
========

Python, as 0of version 2.4, has 38 exceptions (including warnings) in
the built-in namespace in a rather shallow hierarchy.  This list of
classes has grown over the years without a chance to learn from
experience.  This PEP proposes doing a reorganization of the hierarchy
for Python 3.0 when backwards-compatibility is not as much of an
issue.  Along with this reorganization, adding a requirement that all
objects passed to a ``raise`` statement must inherit from a specific
superclass is proposed.  Lastly, bare ``except`` clauses will catch
only exceptions inheriting from Exception.


Rationale
=========

Exceptions are a critical part of Python.  While exceptions are
traditionally used to signal errors in a program, they have also grown
to be used for flow control for things such as iterators.

While their importance is great, there is lack of structure to them.
This stems from the fact that any object can be raised as an
exception.  Because of this you have no guarantee in terms of what
kind of object will be raised, destroying any possible hierarchy 
raised objects might adhere to.

But exceptions do have a hierarchy, showing the severity of the
exception.  The hierarchy also groups related exceptions together to
simplify catching them in ``except`` clauses.  To allow peopele to
be able to rely on this hierarchy, a common superclasse that all
raise objects must inherit from is being proposed.  It also allows
guarantees about the interface to raised objects to be made (see
PEP 344 [#PEP344]_).  A discussion about all of this has occurred
before on python-dev [#Summary2004-08-01]_.

But allowing a guarantee about the hierarchy is not the only place
where exceptions can stand improvement.  Bare ``except`` clauses are
often used in an inappropriate manner.  Since they catch *all* raised
objects, they can catch exceptions that should have been allowed to
propagate to the top level of the execution stack, leading to the
interpreter terminating execution.  Once again, this has been
discussed on python-dev [#python-dev3]_.

To fix this over-reaching of bare ``except`` clauses, it is being
proposed that only objects inheriting from Exception be caught by
bare ``except`` clauses.  This will allow the exception hierarchy
to be organized in such a way that bare ``except`` clauses can be
more useful by allowing exceptions that should not normally be caught
to lead to the termination of the interpreter.


Philosophy of Reorganization
============================

For the reorganization of the hierarchy, there was a general
philosophy followed that developed from discussion of earlier drafts
of this PEP [#python-dev-thread1]_, [#python-dev-thread2]_,
[#python-dev-thread3]_.  First and foremost was to not break anything
that works.  This meant that renaming exceptions was out of the
question unless the name was deemed severely bad.  This
also meant no removal of exceptions unless they were viewed as
truly misplaced. The introduction of new exceptions were only done in
situations where there might be a use for catching a superclass of a
category of exceptions.  Lastly, existing exceptions would have their
inheritance tree changed only if it was felt they were truly
misplaced to begin with.

For all new exceptions, the proper suffix had to be chosen.  For
those that signal an error, "Error" is to be used.  If the exception
is a warning, then "Warning".  "Exception" is to be used when none
of the other suffixes are proper to use and no specific suffix is
a better fit.

After that it came down to choosing which exceptions should and
should not inherit from Exception.  This was for the purpose of
making bare ``except`` clauses more useful.

Lastly, the entire existing hierarchy had to inherit from the new
exception meant to act as the required superclass for all exceptions
to inherit from.


New Hierarchy
=============

.. Note:: Exceptions flagged with "stricter inheritance" will no
   longer inherit from a certain class.  A "broader inheritance" flag
   means a class has been added to the exception's inheritance tree.
   All comparisons are against the Python 2.4 exception hierarchy.

.. parsed-literal::

   +-- BaseException (new; broader inheritance for subclasses)
       +-- TerminalException (new; stricter inheritance for subclasses)
	   +-- KeyboardInterrupt
	   +-- SystemExit
       +-- Exception
	   +-- GeneratorExit (defined in PEP 342 [#PEP342]_)
	   +-- StandardError
	       +-- ArithmeticError
		   +-- DivideByZeroError
		   +-- FloatingPointError
		   +-- OverflowError
	       +-- AssertionError
	       +-- AttributeError
	       +-- EnvironmentError
		   +-- IOError
		       +-- EOFError (broader inheritance)
		   +-- OSError
	       +-- ImportError
	       +-- LookupError
		   +-- IndexError
		   +-- KeyError
	       +-- NameError
		   +-- UnboundLocalError
	       +-- NotImplementedError (stricter inheritance)
	       +-- SyntaxError
		   +-- IndentationError
		       +-- TabError
	       +-- TypeError
	       +-- RuntimeError
	       +-- UnicodeError
		   +-- UnicodeDecodeError
		   +-- UnicodeEncodeError
		   +-- UnicodeTranslateError
	       +-- ValueError
	       +-- VMError (new; broader inheritance for subclasses)
		   +-- MemoryError
		   +-- SystemError
	       +-- ReferenceError
	   +-- StopIteration
	   +-- Warning
	       +-- AnyDeprecationWarning (new; broader inheritance for subclasses)
		   +-- PendingDeprecationWarning
		   +-- DeprecationWarning
	       +-- FutureWarning
	       +-- SyntaxWarning
	       +-- RuntimeWarning
	       +-- UserWarning


Differences Compared to Python 2.4
==================================

A more thorough explanation of terms is needed when discussing
inheritance changes.  Inheritance changes result in either broader or
more restrictive inheritance.  "Broader" is when a class has an
inheritance tree like ``cls, A`` and then becomes ``cls, B, A``.
"Stricter is the reverse.


New Exceptions
--------------

BaseException
'''''''''''''

The superclass that all exceptions must inherit from.


TerminalException
'''''''''''''''''

Superclass for exceptions that are meant to the termination of the
interpreter.  It does not inherit from Exception so that
subclasses are not caught by bare ``except`` clauses.


VMError
'''''''

Superclass for exceptions that deal directly with the virtual
machine.


AnyDeprecationWarning
'''''''''''''''''''''

A common superclass for all deprecation-related exceptions.  While
having DeprecationWarning inherit from PendingDeprecationWarning was
suggested (a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is happening now), the logic was not
agreed upon by a majority.  But since the exceptions are related,
creating a common superclass is warranted.


Removed Exceptions
------------------

WindowsError
''''''''''''

Too OS-specific to be kept in the built-in exception hierarchy.


Change of Position in the Exception Hierarchy
---------------------------------------------

NotImplementedError
'''''''''''''''''''

Inherits from Exception instead of from RuntimeError.


Originally inheriting from RuntimeError, NotImplementedError does not
have any direct relation to the exception meant for use in user code
as a quick-and-dirty exception.  Thus it now directly inherits from
Exception.


EOFError
''''''''

Subclasses IOError.

Since an EOF comes from I/O it only makes sense that it be considered
an I/O error.


Required Superclass for ``raise``
=================================

By requiring all objects passed to a ``raise`` statement to inherit
from a specific superclass, all exceptions are guaranteed to have
certain attributes.  If PEP 344 [#PEP344]_ is accepted, the attributes
outlined there will be guaranteed to be on all exceptions raised.
This should help facilitate debugging by making the querying of
information from exceptions much easier.

The proposed hierarchy has BaseException as the required base class.


Implementation
--------------

Enforcement is straightforward.  Modifying ``RAISE_VARARGS`` to do an
inheritance check first before raising an exception should be enough.
For the C API, all functions that set an exception will have the same
inheritance check applied.


Bare ``except`` Clauses Catching ``Exception`` Only
===================================================

While Python does have its "explicit is better than implicit" tenant,
it is not necessary if there is a reasonable default behavior.
Changing the behavior of a bare ``except`` clause makes its existance
quite reasonable.

In Python 2.4, a bare ``except`` clause will catch any and all
exceptions.  Typically, though, this is not what is truly desired.
More often than not one wants to catch all error exceptions that do
not signify a "bad" interpreter state.  In the new exception hierarchy
this is condition is embodied by Exception.  Thus bare ``except``
clauses will catch only exceptions inheriting from Exception.


Implementation
--------------

In the compiler, when a bare ``except`` clause is reached, the code
for ``except Exception`` will be emitted.


Transition Plan
===============

Exception Hierarchy Changes
---------------------------

New Exceptions
''''''''''''''

New exceptions can simply be added to the built-in namespace.  Any
pre-existing objects with the same name will mask the new exceptions,
preserving backwards-compatibility.


New Inheritance for Old Exceptions
''''''''''''''''''''''''''''''''''

Using multiple inheritance to our advantage, exceptions whose
inheritance is now more resrictive can be made backwards-compatible.
By inheriting from both the new superclasses as well as the original
superclasses, existing ``except`` clauses will continue to work as
before while allowing the new inheritance to be used for new code.

A PendingDeprecationWarning will be raised based on whether the
bytecode ``COMPARE_OP(10)`` results in an exception being caught that
would not have under the new hierarchy.  This will require hard-coding
in the implementation of the bytecode.


Removed Exceptions
''''''''''''''''''

Exceptions scheduled for removal will be transitioned much like the
old names of renamed exceptions.  Upon instantiation a
PendingDeprecationWarning will be raised stating the the exception is
due for removal in Python 3.0.


Required Superclass for ``raise``
---------------------------------

A DeprecationWarning will be raised when an object is passed to
``raise`` that does not have the proper inheritance.


Removal of Bare ``except`` Clauses
----------------------------------

A RuntimeWarning will be raised for all bare ``except`` clauses that
catch an exception that does not inherit from Exception.


Rejected Ideas
==============

Multiple threads on python-dev discussing this PEP have lead to
various ideas being rejected [#python-dev-thread1]_,
[#python-dev-thread2]_, [#python-dev-thread3]_.


KeyboardInterrupt inheriting from ControlFlowException
------------------------------------------------------

KeyboardInterrupt has been a contentious point within this hierarchy.
Some view the exception more as control flow being caused by the user.
But with its asynchronous cause (the user is able to trigger the
exception at any point in code) its proper place is inheriting from
CriticalError.


Other Names for BaseException and Exception
-------------------------------------------

Alternative names for BaseException and Exception have been
Raisable/Exception and Exception/StandardError.  The former
alternatives were rejected because "Raisable" does not reflect its
exception nature well enough.  The latter alternatives were rejected
because they do not reflect current use.


DeprecationWarning Inheriting From PendingDeprecationWarning
------------------------------------------------------------

This was originally proposed because a DeprecationWarning can be
viewed as a PendingDeprecationWarning that is being removed in the
next version.  But since enough people thought the inheritance could
logically work the other way around, the idea was dropped.


AttributeError Inheriting From TypeError or NameError
-----------------------------------------------------

Viewing attributes as part of the interface of a type caused the idea
of inheriting from TypeError.  But that partially defeats the thinking
of duck typing and thus the idea was dropped.

Inheriting from NameError was suggested because objects can be viewed
as having their own namespace where the attributes live and when an
attribute is not found it is a namespace failure.  This was also
dropped as a possibility since not everyone shared this view.


Removal of EnvironmentError
---------------------------

Originally proposed based on the idea that EnvironmentError was an
unneeded distinction, the BDFL overruled this idea [#python-dev4]_.


Introduction of MacError and UnixError
--------------------------------------

Proposed to add symmetry to WindowsError, the BDFL said they won't be
used enough [#python-dev4]_.  The idea of then removing WindowsError
was proposed and accepted as reasonable, thus completely negating the
idea of adding these exceptions.


SystemError Subclassing SystemExit
----------------------------------

Proposed because a SystemError is meant to lead to a system exit, the
idea was removed since CriticalError indicates this better.


ControlFlowException Under Exception
------------------------------------

It has been suggested that ControlFlowException should inherit from
Exception.  This idea has been rejected based on the thinking that
control flow exceptions typically should not be caught by bare
``except`` clauses, whereas Exception subclasses should be.


Removal of Bare ``except`` Clauses
----------------------------------

The suggestion has been made to remove bare ``except`` clauses
altogether, in the name of "explicit is better than implicit".  But
Guido has said this is too weak of an argument since other areas of
Python have default behavior [#python-dev3]_.


Rename NameError to NamespaceError
----------------------------------

NameError is considered more succinct and leaves open no possible mistyping of
the capitalization of "Namespace" [#python-dev5]_.


Renaming RuntimeError or Introducing SimpleError
''''''''''''''''''''''''''''''''''''''''''''''''

The thinking was that RuntimeError was in no way an obvious name for
an exception meant to be used when a situation did not call for the
creation of a new exception.  The renaming was rejected on the basis
that the exception is already used throughout the interpreter [#python-dev6]_.
Rejection of SimpleError was founded on the thought that people
should be free to use whatever exception they choose and not have one
so blatently suggested [#python-dev7]_.

Renaming Existing Exceptions
----------------------------

Various renamings were suggested but non garnered more than a +0 vote
(renaming ReferenceError to WeakReferenceError).  The thinking was
that the existing names were fine and no one had actively complained
about them ever.  To minimize backwards-compatibility issues and
causing existing Python programmers extra pain, the renamings were
removed.


Acknowledgements
================

Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy
Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing,
James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull,
Raymond Hettinger, and everyone else I missed for participating in the
discussion.


References
==========

.. [#PEP342] PEP 342 (Coroutines via Enhanced Generators)
   http://www.python.org/peps/pep-0342.html

.. [#PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks)
   http://www.python.org/peps/pep-0344.html

.. [#Summary2004-08-01] python-dev Summary (An exception is an
   exception, unless it doesn't inherit from Exception)
   http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception

.. [#python-dev3] python-dev email (PEP, take 2: Exception
   Reorganization for Python 3.0)
   http://mail.python.org/pipermail/python-dev/2005-August/055116.html

.. [#exceptions-stdlib] exceptions module
   http://docs.python.org/lib/module-exceptions.html

.. [#python-dev-thread1] python-dev thread (Pre-PEP: Exception
   Reorganization for Python 3.0)
   http://mail.python.org/pipermail/python-dev/2005-July/055020.html,
   http://mail.python.org/pipermail/python-dev/2005-August/055065.html

.. [#python-dev-thread2] python-dev thread (PEP, take 2: Exception
   Reorganization for Python 3.0)
   http://mail.python.org/pipermail/python-dev/2005-August/055103.html

.. [#python-dev-thread3] python-dev thread (Reorg PEP checked in)
    http://mail.python.org/pipermail/python-dev/2005-August/055138.html

.. [#python-dev4] python-dev email (Pre-PEP: Exception Reorganization
   for Python 3.0)
   http://mail.python.org/pipermail/python-dev/2005-July/055019.html

.. [#python-dev5] python-dev email (PEP, take 2: Exception Reorganization for
    Python 3.0)
    http://mail.python.org/pipermail/python-dev/2005-August/055159.html

.. [#python-dev6] python-dev email (Exception Reorg PEP checked in)
    http://mail.python.org/pipermail/python-dev/2005-August/055149.html

.. [#python-dev7] python-dev email (Exception Reorg PEP checked in)
    http://mail.python.org/pipermail/python-dev/2005-August/055175.html


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: