python-peps/pep-0348.txt

PEP: 348
Title: Exception Reorganization for Python 3.0
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Jul-2005
Post-History: 03-Aug-2005


Abstract
========

Python, as of version 2.4, has 38 exceptions (including warnings) in
the built-in namespace in a rather shallow hierarchy.  This list of
classes has grown over the years without a chance to learn from
mistakes and clean up the hierarchy.  This PEP proposes doing a
reorganization for Python 3.0 when backwards-compatibility is not an
issue.  Along with this reorganization, adding a requirement that all
objects passed to a ``raise`` statement must inherit from a specific
superclass is proposed.  Lastly, bare ``except`` clauses will catch
only exceptions inheriting from Exception.


Rationale
=========

Exceptions are a critical part of Python.  While exceptions are
traditionally used to signal errors in a program, they have also grown
to be used for flow control for things such as iterators.  Their
importance is great.

But the organization of the exception hierarchy is suboptimal to serve
the multiple uses of exceptions.  Mostly for backwards-compatibility
reasons, the hierarchy has stayed very flat and old exceptions whose
usefulness has not been proven have been left in.  Making exceptions
more hierarchical would help facilitate exception handling by making
exception catching using inheritance much more logical.  This should
also help lead to fewer errors from overly broad exception catching in
``except`` clauses.

A mandatory superclass for all exceptions is also being proposed
[#Summary2004-08-01]_.  By requiring any object that is used in a
``raise`` statement to inherit from a specific superclass, certain
attributes (such as those laid out in PEP 344 [#PEP344]_) can be
guaranteed to exist.  This also will lead to the planned removal of
string exceptions.

Lastly, bare ``except`` clauses are to catch only exceptions that
inherit from ``Exception`` [#python-dev3]_.  While currently used to
catch all exceptions, that use is too far-reaching and typically not
desired.  Catching only exceptions that inherit from ``Exception``
allows other exceptions (those that should not be caught unless
explicitly desired) to continue to propagate up the execution stack.


Philosophy of Reorganization
============================

There are several goals in this reorganization that defined the
philosophy used to guide the work.  One goal was to prune out unneeded
exceptions.  Extraneous exceptions should not be left in since they
just serve to clutter the built-in namespace.  Unneeded exceptions
also dilute the importance of other exceptions by splitting uses
between several exceptions when all uses should have been under a
single exception.

Another goal was to introduce exceptions that were deemed necessary to
fill holes in the hierarchy.  Most new exceptions were added to flesh
out the inheritance hierarchy to make it easier to catch a category of
exceptions with a simpler ``except`` clause.

Changing inheritance to make the hierarchy more reasonable was a goal.
As stated above, having proper inheritance allows for more accurate
``except`` statements when catching exceptions based on the
inheritance tree.

Lastly, some renaming was done to make the usage of certain exceptions
more obvious.  Having to look up an exception due to the name not
accurately reflecting its intended use is annoying and slows down
debugging.  Having accurate names also makes debugging easier for new
programmers.  But for simplicity, for the convenience of existing
users, and for the sake of transitioning to Python 3.0, only
exceptions whose names were significantly out of alignment with their
stated purpose have been renamed.  All exceptions dealing with errors
will be named with an "Error" suffix.


New Hierarchy
=============

.. Note:: Exceptions flagged with "stricter inheritance" will no
   longer inherit from a certain class.  A "broader inheritance" flag
   means a class has been added to the exception's inheritance tree.

.. parsed-literal::

   BaseException
   +-- CriticalError (new)
       +-- KeyboardInterrupt (stricter inheritance)
       +-- MemoryError (stricter inheritance)
       +-- SystemError (stricter inheritance)
   +-- ControlFlowException (new)
       +-- GeneratorExit (defined in PEP 342 [#PEP342]_)
       +-- StopIteration (stricter inheritance)
       +-- SystemExit (stricter inheritance)
   +-- Exception
       +-- StandardError
           +-- ArithmeticError
               +-- DivideByZeroError
               +-- FloatingPointError
               +-- OverflowError
           +-- AssertionError
           +-- AttributeError
           +-- EnvironmentError
               +-- IOError
                   +-- EOFError (broader inheritance)
               +-- OSError
           +-- ImportError
           +-- LookupError
               +-- IndexError
               +-- KeyError
           +-- NamespaceError (renamed from NameError)
               +-- UnboundFreeError (new)
               +-- UnboundGlobalError (new)
               +-- UnboundLocalError
           +-- NotImplementedError (stricter inheritance)
           +-- SyntaxError
               +-- IndentationError
                   +-- TabError
           +-- TypeError
           +-- UserError (renamed from RuntimeError)
           +-- UnicodeError
               +-- UnicodeDecodeError
               +-- UnicodeEncodeError
               +-- UnicodeTranslateError
           +-- ValueError
       +-- Warning
           +-- AnyDeprecationWarning (new; broader inheritance for subclasses)
               +-- PendingDeprecationWarning
               +-- DeprecationWarning
           +-- FutureWarning
           +-- SyntaxWarning
           +-- SemanticsWarning (renamed from RuntimeWarning)
           +-- UserWarning
       +-- WeakReferenceError (renamed from ReferenceError)


Differences Compared to Python 2.4
==================================

Changes to exceptions from Python 2.4 can take shape in three forms:
removal, renaming, or change of position in the hierarchy.  There are
also new exceptions introduced in the proposed hierarchy.

In terms of new exceptions, almost all are added to flesh out the
inheritance tree.  Those that are leaf classes are added to alleviate
the overloading of another exception.

Positional changes result in either broader or more restrictive
inheritance.  The broader inheritance typically occurred to allow for
a more reasonable superclass to group related exceptions together.
Stricter inheritance happened when the pre-existing inheritance was
deemed incorrect and needed correction.


New Exceptions
--------------

BaseException
'''''''''''''

The superclass that all exceptions must inherit from.


CriticalError
'''''''''''''

The superclass for severe error exceptions; typically, one would not
want to recover from such an exception.  The name is meant to reflect
that these exceptions are raised asynchronously by the interpreter
when a critical event has occured.


ControlFlowException
''''''''''''''''''''

This exception exists as a superclass for all exceptions that directly
deal with control flow.  Inheriting from BaseException instead of
Exception prevents them from being caught accidently when one wants to
catch errors.  The name, by not mentioning "Error", does not lead to
one to confuse the subclasses as errors.


UnboundGlobalError
''''''''''''''''''

Raised when a global variable was not found.


UnboundFreeError
''''''''''''''''

Raised when a free variable is not found.


AnyDeprecationWarning
'''''''''''''''''''''

A common superclass for all deprecation-related exceptions.  While
having DeprecationWarning inherit from PendingDeprecationWarning was
suggested (a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is happening now), the logic was not
agreed upon by a majority.  But since the exceptions are related,
creating a common superclass is warranted.


Removed Exceptions
------------------

WindowsError
''''''''''''

Too OS-specific to be kept in the built-in exception hierarchy.


Renamed Exceptions
------------------

RuntimeError
''''''''''''

Renamed to UserError.

Meant as a generic exception for use when neither a new exception
class nor inheritance-based exception catching is desired,
RuntimeError is poorly named.  Its name in Python 2.4 seems to suggest
an error that occurred at runtime, possibly an error in the VM.
Renaming the exception to UserError more clearly states the purpose of
the exception as a quick-and-dirty error exception.  The name also
keeps it in line with UserWarning.

If a user wants an non-error exception, raising BaseException directly
should be sufficient since Exception, which UserError inherits from,
is only used for errors.


ReferenceError
''''''''''''''

Renamed to WeakReferenceError.

ReferenceError was added to the built-in exception hierarchy in Python
2.2 [#exceptions-stdlib]_.  Its name comes directly from the time when
it resided in the ``weakref`` module.  Unfortunately its name does not
suggest its connection to weak references and thus deserves a
renaming.


NameError
'''''''''

Renamed to NamespaceError.

While NameError suggests its common use, it is not entirely apparent.
Making it a superclass for namespace-related exceptions warrants a
renaming to make its use abundantly clear.  Plus the documentation of
the exception module [#exceptions-stdlib]_ states that it was actually
meant for global names and not for just any exception.


RuntimeWarning
''''''''''''''

Renamed to SemanticsWarning.

RuntimeWarning is supposed to represent semantic changes coming in the
future.  But while saying that it affects the "runtime" is true,
flat-out stating that it is a semantic change is much clearer,
eliminating any possible association of the term "runtime" with the
virtual machine.


Change of Position in the Exception Hierarchy
---------------------------------------------

KeyboardInterrupt, MemoryError, and SystemError
'''''''''''''''''''''''''''''''''''''''''''''''

Inherit from CriticalError instead of from Exception.

These three exceptions are not standard errors by any means.  They are
raised asynchronously by the interpreter when something specific has
occurred.  Thus they warrant not inheriting from Exception but from an
entirely separate exception that will not be caught by a bare
``except`` clause.


StopIteration and SystemExit
''''''''''''''''''''''''''''

Inherit from ControlFlowException instead of from Exception.

By having these exceptions no longer inherit from Exception they will
not be accidentally caught by a bare ``except`` clause.


NotImplementedError
'''''''''''''''''''

Inherits from Exception instead of from RuntimeError (renamed to
UserError).

Originally inheriting from RuntimeError, NotImplementedError does not
have any direct relation to the exception meant for use in user code
as a quick-and-dirty exception.  Thus it now directly inherits from
Exception.


EOFError
''''''''

Subclasses IOError.

Since an EOF comes from I/O it only makes sense that it be considered
an I/O error.


Required Superclass for ``raise``
=================================

By requiring all objects passed to a ``raise`` statement to inherit
from a specific superclass, all exceptions are guaranteed to have
certain attributes.  If PEP 344 [#PEP344]_ is accepted, the attributes
outlined there will be guaranteed to be on all exceptions raised.
This should help facilitate debugging by making the querying of
information from exceptions much easier.

The proposed hierarchy has BaseException as the required base class.


Implementation
--------------

Enforcement is straightforward.  Modifying ``RAISE_VARARGS`` to do an
inheritance check first before raising an exception should be enough.
For the C API, all functions that set an exception will have the same
inheritance check applied.


Bare ``except`` Clauses Catching ``Exception`` Only
===================================================

While Python does have its "explicit is better than implicit" tenant,
it is not necessary if there is a reasonable default behavior.
Changing the behavior of a bare ``except`` clause makes its existance
quite reasonable.

In Python 2.4, a bare ``except`` clause will catch any and all
exceptions.  Typically, though, this is not what is truly desired.
More often than not one wants to catch all error exceptions that do
not signify a "bad" interpreter state.  In the new exception hierarchy
this is condition is embodied by Exception.  Thus bare ``except``
clauses will catch only exceptions inheriting from Exception.


Implementation
--------------

In the compiler, when a bare ``except`` clause is reached, the code
for ``except Exception`` will be emitted.


Transition Plan
===============

Exception Hierarchy Changes
---------------------------

New Exceptions
''''''''''''''

New exceptions can simply be added to the built-in namespace.  Any
pre-existing objects with the same name will mask the new exceptions,
preserving backwards-compatibility.


Renamed Exceptions
''''''''''''''''''

Renamed exceptions will directly subclass the new names.  When the old
exceptions are instantiated (which occurs when an exception is caught,
either by a ``try`` statement or by propagating to the top of the
execution stack), a PendingDeprecationWarning will be raised.

This should properly preserve backwards-compatibility as old usage
won't change and the new names can also be used to catch exceptions
using the old names.  The warning of the deprecation is also kept
simple.


New Inheritance for Old Exceptions
''''''''''''''''''''''''''''''''''

Using multiple inheritance to our advantage, exceptions whose
inheritance is now more resrictive can be made backwards-compatible.
By inheriting from both the new superclasses as well as the original
superclasses, existing ``except`` clauses will continue to work as
before while allowing the new inheritance to be used for new code.

A PendingDeprecationWarning will be raised based on whether the
bytecode ``COMPARE_OP(10)`` results in an exception being caught that
would not have under the new hierarchy.  This will require hard-coding
in the implementation of the bytecode.


Removed Exceptions
''''''''''''''''''

Exceptions scheduled for removal will be transitioned much like the
old names of renamed exceptions.  Upon instantiation a
PendingDeprecationWarning will be raised stating the the exception is
due for removal in Python 3.0.


Required Superclass for ``raise``
---------------------------------

A SemanticsWarning will be raised when an object is passed to
``raise`` that does not have the proper inheritance.


Removal of Bare ``except`` Clauses
----------------------------------

A SemanticsWarning will be raised for all bare ``except`` clauses.


Rejected Ideas
==============

Threads on python-dev discussing this PEP can be found at
[#python-dev-thread1]_ and [#python-dev-thread2]_


KeyboardInterrupt inheriting from ControlFlowException
------------------------------------------------------

KeyboardInterrupt has been a contentious point within this hierarchy.
Some view the exception more as control flow being caused by the user.
But with its asynchronous cause (the user is able to trigger the
exception at any point in code) its proper place is inheriting from
CriticalException.  It also keeps the name of the exception from being
"CriticalError".


Other Names for BaseException and Exception
-------------------------------------------

Alternative names for BaseException and Exception have been
Raisable/Exception and Exception/StandardError.  The former
alternatives were rejected because "Raisable" does not reflect its
exception nature well enough.  The latter alternatives were rejected
because they do not reflect current use.


DeprecationWarning Inheriting From PendingDeprecationWarning
------------------------------------------------------------

This was originally proposed because a DeprecationWarning can be
viewed as a PendingDeprecationWarning that is being removed in the
next version.  But since enough people thought the inheritance could
logically work the other way around, the idea was dropped.


AttributeError Inheriting From TypeError or NameError
-----------------------------------------------------

Viewing attributes as part of the interface of a type caused the idea
of inheriting from TypeError.  But that partially defeats the thinking
of duck typing and thus the idea was dropped.

Inheriting from NameError was suggested because objects can be viewed
as having their own namespace where the attributes live and when an
attribute is not found it is a namespace failure.  This was also
dropped as a possibility since not everyone shared this view.


Removal of EnvironmentError
---------------------------

Originally proposed based on the idea that EnvironmentError was an
unneeded distinction, the BDFL overruled this idea [#python-dev4]_.


Introduction of MacError and UnixError
--------------------------------------

Proposed to add symmetry to WindowsError, the BDFL said they won't be
used enough [#python-dev4]_.  The idea of then removing WindowsError
was proposed and accepted as reasonable, thus completely negating the
idea of adding these exceptions.


SystemError Subclassing SystemExit
----------------------------------

Proposed because a SystemError is meant to lead to a system exit, the
idea was removed since CriticalException indicates this better.


ControlFlowException Under Exception
------------------------------------

It has been suggested that ControlFlowException should inherit from
Exception.  This idea has been rejected based on the thinking that
control flow exceptions typically should not be caught by bare
``except`` clauses, whereas Exception subclasses should be.


Removal of Bare ``except`` Clauses
----------------------------------

The suggestion has been made to remove bare ``except`` clauses
altogether, in the name of "explicit is better than implicit".  But
Guido has said this is too weak of an argument since other areas of
Python have default behavior [#python-dev3]_.


Open Issues
===========

Remove ControlFlowException?
----------------------------

It has been suggested that ControlFlowException is not needed.  Since
the desire to catch any control flow exception will be atypical, the
suggestion is to just remove the exception and let the exceptions that
inherited from it inherit directly from BaseException.  This still
preserves the seperation from Exception which is one of the driving
factors behind the introduction of ControlFlowException.


Acknowledgements
================

Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy
Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing,
James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull and
everyone else I missed for participating in the discussion.


References
==========

.. [#PEP342] PEP 342 (Coroutines via Enhanced Generators)
   (http://www.python.org/peps/pep-0342.html)

.. [#PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks)
   (http://www.python.org/peps/pep-0344.html)

.. [#Summary2004-08-01] python-dev Summary (An exception is an
   exception, unless it doesn't inherit from Exception)
   (http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception)

.. [#Summary2004-09-01] python-dev Summary (Cleaning the Exception House)
   (http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-exception-house)

.. [#python-dev1] python-dev email (Exception hierarchy)
   (http://mail.python.org/pipermail/python-dev/2004-August/047908.html)

.. [#python-dev2] python-dev email (Dangerous exceptions)
   (http://mail.python.org/pipermail/python-dev/2004-September/048681.html)

.. [#python-dev3] python-dev email (PEP, take 2: Exception
   Reorganization for Python 3.0)
   (http://mail.python.org/pipermail/python-dev/2005-August/055116.html)

.. [#exceptions-stdlib] exceptions module
   (http://docs.python.org/lib/module-exceptions.html)

.. [#python-dev-thread1] python-dev thread (Pre-PEP: Exception
   Reorganization for Python 3.0)
   (http://mail.python.org/pipermail/python-dev/2005-July/055020.html,
   http://mail.python.org/pipermail/python-dev/2005-August/055065.html)

.. [#python-dev-thread2] python-dev thread (PEP, take 2: Exception
   Reorganization for Python 3.0)
   (http://mail.python.org/pipermail/python-dev/2005-August/055103.html)

.. [#python-dev4] python-dev email (Pre-PEP: Exception Reorganization
   for Python 3.0)
   (http://mail.python.org/pipermail/python-dev/2005-July/055019.html)


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: