python-peps/pep-0348.txt

PEP: 348
Title: Exception Reorganization for Python 3.0
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Jul-2005
Post-History: 03-Aug-2005


Abstract
========

Python, as of version 2.4, has 38 exceptions (including warnings) in the built-in namespace in a rather shallow hierarchy.
This list of classes has grown over the years without a chance to learn from mistakes and cleaning up the hierarchy.
This PEP proposes doing a reorganization for Python 3.0 when backwards-compatibility is not an issue.
Along with this reorganization, adding a requirement that all objects passed to
a ``raise`` statement must inherit from a specific superclass is proposed.
Lastly, bare ``except`` clauses will catch only exceptions inheriting from
Exception.


Rationale
=========

Exceptions are a critical part of Python.
While exceptions are traditionally used to signal errors in a program, they have also grown to be used for flow control for things such as iterators.
There importance is great.

But the organization of the exception hierarchy is suboptimal to serve the
multiple uses of exceptions.
Mostly for backwards-compatibility reasons, the hierarchy has stayed very flat and old exceptions who usefulness have not been proven have been left in.
Making exceptions more hierarchical would help facilitate exception handling by making catching exceptions using inheritance much more logical.
This should also help lead to less errors from being too broad in what exceptions are caught in an ``except`` clause.

A required superclass for all exceptions is also being proposed [Summary2004-08-01]_.
By requiring any object that is used in a ``raise`` statement to inherit from a specific superclass, certain attributes (such as those laid out in PEP 344 [PEP344]_) can be guaranteed to exist.
This also will lead to the planned removal of string exceptions.

Lastly, bare ``except`` clauses are to catch only exceptions that inherit from
Exception [python-dev3]_.
While currently used to catch all exceptions, that use is rather far reaching
and typically not desired.
Catching only exceptions inheriting from Exception allows exceptions that
should not be caught unless explicitly desired to continue to propagate up the
execution stack.


Philosophy of Reorganization
============================

There are several goals in this reorganization that defined the philosophy used to guide the work.
One goal was to prune out unneeded exceptions.
Extraneous exceptions should not be left in since it just serves to clutter the built-in namespace.
Unneeded exceptions also dilute the importance of other exceptions by splitting uses between several exceptions when all uses should have been under a single exception.

Another goal was to introduce any exceptions that were deemed needed to fill any holes in the hierarchy.
Most new exceptions were done to flesh out the inheritance hierarchy to make it easier to catch a category of exceptions with a simpler ``except`` clause.

Changing inheritance to make it more reasonable was a goal.
As stated above, having proper inheritance allows for more accurate ``except`` statements when catching exceptions based on the inheritance tree.

Lastly, any renaming to make an exception's use more obvious from its name was done.
Having to look up what an exception is meant to be used for because the name does not proper reflect its usage is annoying and slows down debugging.
Having a proper name also makes debugging easier on new programmers.
But for simplicity of existing user's and for transitioning to Python 3.0, only exceptions whose names were fairly out of alignment with their stated purpose have been renamed.
It was also made sure the exceptions dealing with errors had the "Error"
suffix.

New Hierarchy
=============

.. note:: exceptions flagged as "stricter inheritance" means that the class no
    longer inherits from a certain class; "broader inheritance" means a class has
    been added to the exception's inheritance tree

::

 BaseException
 +-- CriticalError (new)
    +-- KeyboardInterrupt (stricter inheritance)
    +-- MemoryError (stricter inheritance)
    +-- SystemError (stricter inheritance)
 +-- ControlFlowException (new)
    +-- GeneratorExit (defined in PEP 342 [PEP342]_)
    +-- StopIteration (stricter inheritance)
    +-- SystemExit (stricter inheritance)
 +-- Exception
    +-- StandardError
	+-- ArithmeticError
	    +-- DivideByZeroError
	    +-- FloatingPointError
	    +-- OverflowError
	+-- AssertionError
	+-- AttributeError
	+-- EnvironmentError
	    +-- IOError
		+-- EOFError (broader inheritance)
	    +-- OSError
	+-- ImportError
	+-- LookupError
	    +-- IndexError
	    +-- KeyError
	+-- NamespaceError (rename of NameError)
	    +-- UnboundFreeError (new)
	    +-- UnboundGlobalError (new)
	    +-- UnboundLocalError
	+-- NotImplementedError (stricter inheritance)
	+-- SyntaxError
	    +-- IndentationError
		+-- TabError
	+-- TypeError
	+-- UserError (rename of RuntimeError)
	+-- UnicodeError
	    +-- UnicodeDecodeError
	    +-- UnicodeEncodeError
	    +-- UnicodeTranslateError
	+-- ValueError
    +-- Warning
	+-- AnyDeprecationWarning (new; broader inheritance for subclasses)
	    +-- PendingDeprecationWarning
	    +-- DeprecationWarning
	+-- FutureWarning
	+-- SyntaxWarning
	+-- SemanticsWarning (rename of RuntimeWarning)
	+-- UserWarning
    +-- WeakReferenceError (rename of ReferenceError)


Differences Compared to Python 2.4
==================================

Changes to exceptions from Python 2.4 can take shape in three forms: removal, renaming, or change in the inheritance tree.
There are also new exceptions introduced in the proposed hierarchy.

In terms of new exceptions, almost all are to flesh out the inheritance tree.
Those that are leaf classes are to alleaviate overloading the use of another exception.

Inheritance change can be broader or more restrictive.  The broader inheritance
typically occurred to allow for a more reasonable superclass to group related
exceptions together.  Stricter inheritance happened when the pre-existing
inheritance was deemed incorrect and needed correction.


New Exceptions
--------------

BaseException
'''''''''''''

The superclass that all exceptions must inherit from.


CriticalError
'''''''''''''

The superclass for exceptions for which a severe error has occurred that one
would not want to recover from.
The name is meant to reflect that these exceptions are raised asynchronously by
the interpreter when a critical event has occured that one would most likely
want the interpreter to halt over.


ControlFlowException
''''''''''''''''''''

This exception exists as a superclass for all exceptions that directly deal
with control flow.
Inheriting from BaseException instead of Exception prevents them from being caught accidently when one wants to catch errors.
The name, by not mentioning "Error", does not lead to one to confuse the subclasses as errors.


UnboundGlobalError
''''''''''''''''''

Raised when a global variable was not found.


UnboundFreeError
''''''''''''''''

Raised when a free variable is not found.


AnyDeprecationWarning
'''''''''''''''''''''

A common superclass for all deprecation-related exceptions.
While having DeprecationWarning inherit from PendingDeprecationWarning was
suggested because a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is happening now, the logic was not agreed upon
by a majority.
But since the exceptions are related, creating a common superclass is
warranted.


Removed Exceptions
------------------

WindowsError
''''''''''''

Too OS-specific to be kept in the built-in exception hierarchy.


Renamed Exceptions
------------------

RuntimeError
''''''''''''

Renamed UserError.

Meant for use as a generic exception to be used when one does not want to
create a new exception class but do not want to raise an exception that might
be caught based on inheritance, RuntimeError is poorly named.
It's name in Python 2.4 seems to suggest an error that occurred at runtime,
possibly an error in the VM.
Renaming the exception to UserError more clearly states the purpose for
the exception as quick-and-dirty error exception for the user to use.
The name also keeps it in line with UserWarning.

If a user wants an exception that is not to be used as an error, raising
BaseException directly should be sufficient as Exception, as UserError inherits
from, is only used for errors.


ReferenceError
''''''''''''''

Renamed WeakReferenceError.

ReferenceError was added to the built-in exception hierarchy in Python 2.2
[exceptions-stdlib]_.
Taken directly from the ``weakref`` module, its name comes directly from its original name when it resided in the module.
Unfortunately its name does not suggest its connection to weak references and thus deserves a renaming.


NameError
'''''''''

Renamed NamespaceError.

While NameError suggests its common use, it is not entirely apparent.
Making it more of a superclass for namespace-related exceptions warrants a
renaming to make it abundantly clear its use.
Plus the documentation of the exception module[exceptions-stdlib]_ states that it is actually meant for global names and not for just any exception.


RuntimeWarning
''''''''''''''

Renamed SemanticsWarning.

RuntimeWarning is to represent semantic changes coming in the future.
But while saying that affects "runtime" is true, flat-out stating it is a semantic change is much clearer, eliminating any possible association of "runtime" with the virtual machine specifically.


Changed Inheritance
-------------------

KeyboardInterrupt, MemoryError, and SystemError
'''''''''''''''''''''''''''''''''''''''''''''''

Inherit from CriticalError instead of Exception.

The three above-mentioned exceptions are not standard errors by any means.
They are raised asynchronously by the interpreter when something specific has
occurred.  Thus they warrant not inheriting from Exception but from an
entirely separate exception that will not be caught by a bare ``except``
clause.


StopIteration and SystemExit
''''''''''''''''''''''''''''

Inherit from ControlFlowException instead of Exception.

By having these exceptions no longer inherit from Exception they will not be
accidentally caught by a bare ``except`` clause.


NotImplementedError
'''''''''''''''''''

Inherits from Exception instead of RuntimeError (renamed UserError).

Originally inheriting from RuntimeError, NotImplementedError does not have any
direct relation to the exception meant for use in user code as a
quick-and-dirty exception.  Thus it now directly inherits from Exception.


EOFError
''''''''

Subclasses IOError.

Since an EOF comes from I/O it only makes sense that it be considered an I/O error.


Required Superclass for ``raise``
=================================

By requiring all objects passed to a ``raise`` statement inherit from a specific superclass, one is guaranteed that all exceptions will have certain attributes.
If PEP 342 [PEP344]_ is accepted, the attributes outlined there will be guaranteed to be on all exceptions raised.
This should help facilitate debugging by making the querying of information from exceptions much easier.

The proposed hierarchy has BaseException as the required class that one must inherit from.


Implementation
--------------

Enforcement is straight-forward.
Modifying ``RAISE_VARARGS`` to do an inheritance check first before raising
an exception should be enough.  For the C API, all functions that set an
exception will have the same inheritance check.


Bare ``except`` Clauses Catching Exception Only
===============================================

While Python does have its "explicit is better than implicit" tenant, it is not
necessary if a default behavior is reasonable.  In the case of a bare
``except`` clause, changing the behavior makes it quite reasonable to have
around.

In Python 2.4, a bare ``except`` clause will catch all exceptions.  Typically,
though, this is not what is truly desired.
More often than not one wants to catch all error exceptions that do not signify
a bad state of the interpreter.  In the new exception hierarchy this is
embodied by Exception.  Thus bare ``except`` clauses will catch only
exceptions inheriting from Exception.


Implementation
--------------

In the compiler, when a bare ``except`` clause is reached, the code for
``except Exception`` will be emitted.


Transition Plan
===============

Exception Hierarchy Changes
---------------------------

New Exceptions
''''''''''''''

New exceptions can simply be added to the built-in namespace.
Any pre-existing objects with the same name will mask the new exceptions,
preserving backwards-compatibility.


Renamed Exceptions
''''''''''''''''''

Renamed exceptions will directly subclass the new names.
When the old exceptions are instantiated (which occurs when an exception is
caught, either by a ``try`` statement or by propagating to the top of the
execution stack), a PendingDeprecationWarning will be raised.

This should properly preserve backwards-compatibility as old usage won't change
and the new names can be used to also catch exceptions using the old name.
The warning of the deprecation is also kept simple.


New Inheritance for Old Exceptions
''''''''''''''''''''''''''''''''''

Using multiple inheritance to our advantage, exceptions whose inheritance is
now more resrictive can be made backwards-compatible.
By inheriting from both the new superclasses as well as the original
superclasses existing ``except`` clauses will continue to work as before while
allowing the new inheritance to be used for new clauses.

A PendingDeprecationWarning will be raised based on whether the bytecode
``COMPARE_OP(10)`` results in an exception being caught that would not have
under the new hierarchy.  This will require hard-coding in the implementation
of the bytecode.


Removed Exceptions
''''''''''''''''''

Exceptions scheduled for removal will be transitioned much like the old names
of renamed exceptions.
Upon instantiation a PendingDeprecationWarning will be raised stating the the
exception is due to be removed by Python 3.0 .


Required Superclass for ``raise``
---------------------------------

A SemanticsWarning will be raised when an object is passed to ``raise`` that
does not have the proper inheritance.


Removal of Bare ``except`` Clauses
----------------------------------

A SemanticsWarning will be raised for all bare ``except`` clauses.


Rejected Ideas
==============

Threads on python-dev discussing this PEP can be found at
[python-dev-thread1]_, [python-dev-thread2]_


KeyboardInterrupt inheriting from ControlFlowException
------------------------------------------------------

KeyboardInterrupt has been a contentious point within this hierarchy.
Some view the exception as more control flow being caused by the user.
But with its asynchronous cause thanks to the user being able to trigger the
exception at any point in code it has a more proper place inheriting from
CriticalException.  It also keeps the name of the exception from being "CriticalError".


Other Names for BaseException and Exception
-------------------------------------------

Alternative names for BaseException/Exception have been Raisable/Exception and
Exception/StandardError.  The former has been rejected on the basis that
Raisable does not reflect how it is an exception well enough.  The latter was
rejected based on the fact that it did not reflect current use as the chosen
names do.


DeprecationWarning Inheriting From PendingDeprecationWarning
------------------------------------------------------------

Originally proposed because a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is being removed in the next version.
But enough people thought the inheritance could logically work the other way
the idea was dropped.


AttributeError Inheriting From TypeError or NameError
-----------------------------------------------------

Viewing attributes as part of the interface of a type caused the idea of
inheriting from TypeError.
But that partially defeats the thinking of duck typing and thus was dropped.

Inheriting from NameError was suggested because objects can be viewed as having
their own namespace that the attributes lived in and when they are not found it
is a namespace failure.  This was also dropped as a possibility since not
everyone shared this view.


Removal of EnvironmentError
---------------------------

Originally proposed based on the idea that EnvironmentError was an unneeded
distinction, the BDFL overruled this idea [python-dev4]_.


Introduction of MacError and UnixError
--------------------------------------

Proposed to add symmetry to WindowsError, the BDFL said they won't be used
enough [python-dev4]_.  The idea of then removing WindowsError was proposed and
accepted as reasonable, thus completely negating the idea of adding these
exceptions.


SystemError Subclassing SystemExit
----------------------------------

Proposed because a SystemError is meant to lead to a system exit, the idea was
removed since CriticalException signifies this better.


ControlFlowException Under Exception
------------------------------------

It has been suggested that ControlFlowException inherit from Exception.
This idea has been rejected based on the thinking that control flow exceptions
are typically not desired to be caught in a generic fashion as Exception 
will usually be used.


Removal of Bare ``except`` Clauses
----------------------------------

The suggestion has been made to remove bare ``except`` clauses in the name of
"explicit is better than implicit".  But Guido has said this is too weak of an
argument since other things in Python has default behavior [python-dev3]_.


Open Issues
===========

Remove ControlFlowException?
----------------------------

It has been suggested that ControlFlowException is not needed.
Since the desire to catch any control flow exception will be atypical, the
suggestion is to just remove the exception and let the exceptions that
inherited from it inherit directly from BaseException.  This still preserves the
seperation from Exception which is one of the driving factors behind the
introduction of the exception.


Acknowledgements
================

Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing, James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull and everyone else I missed for participating in the discussion.


References
==========

.. [PEP342] PEP 342 (Coroutines via Enhanced Generators)
            (http://www.python.org/peps/pep-0342.html)

.. [PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks)
            (http://www.python.org/peps/pep-0344.html)

.. [exceptionsmodules] 'exceptions' module
            (http://docs.python.org/lib/module-exceptions.html)

.. [Summary2004-08-01] python-dev Summary (An exception is an exception, unless it doesn't inherit from Exception)
            (http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception)

.. [Summary2004-09-01] python-dev Summary (Cleaning the Exception House)
            (http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-exception-house)

.. [python-dev1] python-dev email (Exception hierarchy)
            (http://mail.python.org/pipermail/python-dev/2004-August/047908.html)

.. [python-dev2] python-dev email (Dangerous exceptions)
            (http://mail.python.org/pipermail/python-dev/2004-September/048681.html)

.. [python-dev3] python-dev email (PEP, take 2: Exception Reorganization for Python 3.0)
	    (http://mail.python.org/pipermail/python-dev/2005-August/055116.html)

.. [exceptions-stdlib] exceptions module
	    (http://www.python.org/doc/2.4.1/lib/module-exceptions.html)

.. [python-dev-thread1] python-dev thread (Pre-PEP: Exception Reorganization for Python 3.0)
	    (http://mail.python.org/pipermail/python-dev/2005-July/055020.html
	    ,
	    http://mail.python.org/pipermail/python-dev/2005-August/055065.html)

.. [python-dev-thread2] python-dev thread (PEP, take 2: Exception Reorganization for Python 3.0)
	    (http://mail.python.org/pipermail/python-dev/2005-August/055103.html)

.. [python-dev4] python-dev email (Pre-PEP: Exception Reorganization for Python 3.0)
	    (http://mail.python.org/pipermail/python-dev/2005-July/055019.html)


Copyright
=========

This document has been placed in the public domain.