PEP: 348 Title: Exception Reorganization for Python 3.0 Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Jul-2005 Post-History: 03-Aug-2005 Abstract ======== Python, as of version 2.4, has 38 exceptions (including warnings) in the built-in namespace in a rather shallow hierarchy. This list of classes has grown over the years without a chance to learn from mistakes and clean up the hierarchy. This PEP proposes doing a reorganization for Python 3.0 when backwards-compatibility is not an issue. Along with this reorganization, adding a requirement that all objects passed to a ``raise`` statement must inherit from a specific superclass is proposed. Lastly, bare ``except`` clauses will catch only exceptions inheriting from Exception. Rationale ========= Exceptions are a critical part of Python. While exceptions are traditionally used to signal errors in a program, they have also grown to be used for flow control for things such as iterators. Their importance is great. But the organization of the exception hierarchy is suboptimal to serve the multiple uses of exceptions. Mostly for backwards-compatibility reasons, the hierarchy has stayed very flat and old exceptions whose usefulness has not been proven have been left in. Making exceptions more hierarchical would help facilitate exception handling by making exception catching using inheritance much more logical. This should also help lead to fewer errors from overly broad exception catching in ``except`` clauses. A mandatory superclass for all exceptions is also being proposed [#Summary2004-08-01]_. By requiring any object that is used in a ``raise`` statement to inherit from a specific superclass, certain attributes (such as those laid out in PEP 344 [#PEP344]_) can be guaranteed to exist. This also will lead to the planned removal of string exceptions. Lastly, bare ``except`` clauses are to catch only exceptions that inherit from ``Exception`` [#python-dev3]_. While currently used to catch all exceptions, that use is too far-reaching and typically not desired. Catching only exceptions that inherit from ``Exception`` allows other exceptions (those that should not be caught unless explicitly desired) to continue to propagate up the execution stack. Philosophy of Reorganization ============================ There are several goals in this reorganization that defined the philosophy used to guide the work. One goal was to prune out unneeded exceptions. Extraneous exceptions should not be left in since they just serve to clutter the built-in namespace. Unneeded exceptions also dilute the importance of other exceptions by splitting uses between several exceptions when all uses should have been under a single exception. Another goal was to introduce exceptions that were deemed necessary to fill holes in the hierarchy. Most new exceptions were added to flesh out the inheritance hierarchy to make it easier to catch a category of exceptions with a simpler ``except`` clause. Changing inheritance to make the hierarchy more reasonable was a goal. As stated above, having proper inheritance allows for more accurate ``except`` statements when catching exceptions based on the inheritance tree. Lastly, some renaming was done to make the usage of certain exceptions more obvious. Having to look up an exception due to the name not accurately reflecting its intended use is annoying and slows down debugging. Having accurate names also makes debugging easier for new programmers. But for simplicity, for the convenience of existing users, and for the sake of transitioning to Python 3.0, only exceptions whose names were significantly out of alignment with their stated purpose have been renamed. All exceptions dealing with errors will be named with an "Error" suffix. New Hierarchy ============= .. Note:: Exceptions flagged with "stricter inheritance" will no longer inherit from a certain class. A "broader inheritance" flag means a class has been added to the exception's inheritance tree. .. parsed-literal:: BaseException +-- CriticalError (new) +-- KeyboardInterrupt (stricter inheritance) +-- MemoryError (stricter inheritance) +-- SystemError (stricter inheritance) +-- ControlFlowException (new) +-- GeneratorExit (defined in PEP 342 [#PEP342]_) +-- StopIteration (stricter inheritance) +-- SystemExit (stricter inheritance) +-- Exception +-- StandardError +-- ArithmeticError +-- DivideByZeroError +-- FloatingPointError +-- OverflowError +-- AssertionError +-- AttributeError +-- EnvironmentError +-- IOError +-- EOFError (broader inheritance) +-- OSError +-- ImportError +-- LookupError +-- IndexError +-- KeyError +-- NamespaceError (renamed from NameError) +-- UnboundFreeError (new) +-- UnboundGlobalError (new) +-- UnboundLocalError +-- NotImplementedError (stricter inheritance) +-- SyntaxError +-- IndentationError +-- TabError +-- TypeError +-- UserError (renamed from RuntimeError) +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- ValueError +-- Warning +-- AnyDeprecationWarning (new; broader inheritance for subclasses) +-- PendingDeprecationWarning +-- DeprecationWarning +-- FutureWarning +-- SyntaxWarning +-- SemanticsWarning (renamed from RuntimeWarning) +-- UserWarning +-- WeakReferenceError (renamed from ReferenceError) Differences Compared to Python 2.4 ================================== Changes to exceptions from Python 2.4 can take shape in three forms: removal, renaming, or change of position in the hierarchy. There are also new exceptions introduced in the proposed hierarchy. In terms of new exceptions, almost all are added to flesh out the inheritance tree. Those that are leaf classes are added to alleviate the overloading of another exception. Positional changes result in either broader or more restrictive inheritance. The broader inheritance typically occurred to allow for a more reasonable superclass to group related exceptions together. Stricter inheritance happened when the pre-existing inheritance was deemed incorrect and needed correction. New Exceptions -------------- BaseException ''''''''''''' The superclass that all exceptions must inherit from. CriticalError ''''''''''''' The superclass for severe error exceptions; typically, one would not want to recover from such an exception. The name is meant to reflect that these exceptions are raised asynchronously by the interpreter when a critical event has occured. ControlFlowException '''''''''''''''''''' This exception exists as a superclass for all exceptions that directly deal with control flow. Inheriting from BaseException instead of Exception prevents them from being caught accidently when one wants to catch errors. The name, by not mentioning "Error", does not lead to one to confuse the subclasses as errors. UnboundGlobalError '''''''''''''''''' Raised when a global variable was not found. UnboundFreeError '''''''''''''''' Raised when a free variable is not found. AnyDeprecationWarning ''''''''''''''''''''' A common superclass for all deprecation-related exceptions. While having DeprecationWarning inherit from PendingDeprecationWarning was suggested (a DeprecationWarning can be viewed as a PendingDeprecationWarning that is happening now), the logic was not agreed upon by a majority. But since the exceptions are related, creating a common superclass is warranted. Removed Exceptions ------------------ WindowsError '''''''''''' Too OS-specific to be kept in the built-in exception hierarchy. Renamed Exceptions ------------------ RuntimeError '''''''''''' Renamed to UserError. Meant as a generic exception for use when neither a new exception class nor inheritance-based exception catching is desired, RuntimeError is poorly named. Its name in Python 2.4 seems to suggest an error that occurred at runtime, possibly an error in the VM. Renaming the exception to UserError more clearly states the purpose of the exception as a quick-and-dirty error exception. The name also keeps it in line with UserWarning. If a user wants an non-error exception, raising BaseException directly should be sufficient since Exception, which UserError inherits from, is only used for errors. ReferenceError '''''''''''''' Renamed to WeakReferenceError. ReferenceError was added to the built-in exception hierarchy in Python 2.2 [#exceptions-stdlib]_. Its name comes directly from the time when it resided in the ``weakref`` module. Unfortunately its name does not suggest its connection to weak references and thus deserves a renaming. NameError ''''''''' Renamed to NamespaceError. While NameError suggests its common use, it is not entirely apparent. Making it a superclass for namespace-related exceptions warrants a renaming to make its use abundantly clear. Plus the documentation of the exception module [#exceptions-stdlib]_ states that it was actually meant for global names and not for just any exception. RuntimeWarning '''''''''''''' Renamed to SemanticsWarning. RuntimeWarning is supposed to represent semantic changes coming in the future. But while saying that it affects the "runtime" is true, flat-out stating that it is a semantic change is much clearer, eliminating any possible association of the term "runtime" with the virtual machine. Change of Position in the Exception Hierarchy --------------------------------------------- KeyboardInterrupt, MemoryError, and SystemError ''''''''''''''''''''''''''''''''''''''''''''''' Inherit from CriticalError instead of from Exception. These three exceptions are not standard errors by any means. They are raised asynchronously by the interpreter when something specific has occurred. Thus they warrant not inheriting from Exception but from an entirely separate exception that will not be caught by a bare ``except`` clause. StopIteration and SystemExit '''''''''''''''''''''''''''' Inherit from ControlFlowException instead of from Exception. By having these exceptions no longer inherit from Exception they will not be accidentally caught by a bare ``except`` clause. NotImplementedError ''''''''''''''''''' Inherits from Exception instead of from RuntimeError (renamed to UserError). Originally inheriting from RuntimeError, NotImplementedError does not have any direct relation to the exception meant for use in user code as a quick-and-dirty exception. Thus it now directly inherits from Exception. EOFError '''''''' Subclasses IOError. Since an EOF comes from I/O it only makes sense that it be considered an I/O error. Required Superclass for ``raise`` ================================= By requiring all objects passed to a ``raise`` statement to inherit from a specific superclass, all exceptions are guaranteed to have certain attributes. If PEP 344 [#PEP344]_ is accepted, the attributes outlined there will be guaranteed to be on all exceptions raised. This should help facilitate debugging by making the querying of information from exceptions much easier. The proposed hierarchy has BaseException as the required base class. Implementation -------------- Enforcement is straightforward. Modifying ``RAISE_VARARGS`` to do an inheritance check first before raising an exception should be enough. For the C API, all functions that set an exception will have the same inheritance check applied. Bare ``except`` Clauses Catching ``Exception`` Only =================================================== While Python does have its "explicit is better than implicit" tenant, it is not necessary if there is a reasonable default behavior. Changing the behavior of a bare ``except`` clause makes its existance quite reasonable. In Python 2.4, a bare ``except`` clause will catch any and all exceptions. Typically, though, this is not what is truly desired. More often than not one wants to catch all error exceptions that do not signify a "bad" interpreter state. In the new exception hierarchy this is condition is embodied by Exception. Thus bare ``except`` clauses will catch only exceptions inheriting from Exception. Implementation -------------- In the compiler, when a bare ``except`` clause is reached, the code for ``except Exception`` will be emitted. Transition Plan =============== Exception Hierarchy Changes --------------------------- New Exceptions '''''''''''''' New exceptions can simply be added to the built-in namespace. Any pre-existing objects with the same name will mask the new exceptions, preserving backwards-compatibility. Renamed Exceptions '''''''''''''''''' Renamed exceptions will directly subclass the new names. When the old exceptions are instantiated (which occurs when an exception is caught, either by a ``try`` statement or by propagating to the top of the execution stack), a PendingDeprecationWarning will be raised. This should properly preserve backwards-compatibility as old usage won't change and the new names can also be used to catch exceptions using the old names. The warning of the deprecation is also kept simple. New Inheritance for Old Exceptions '''''''''''''''''''''''''''''''''' Using multiple inheritance to our advantage, exceptions whose inheritance is now more resrictive can be made backwards-compatible. By inheriting from both the new superclasses as well as the original superclasses, existing ``except`` clauses will continue to work as before while allowing the new inheritance to be used for new code. A PendingDeprecationWarning will be raised based on whether the bytecode ``COMPARE_OP(10)`` results in an exception being caught that would not have under the new hierarchy. This will require hard-coding in the implementation of the bytecode. Removed Exceptions '''''''''''''''''' Exceptions scheduled for removal will be transitioned much like the old names of renamed exceptions. Upon instantiation a PendingDeprecationWarning will be raised stating the the exception is due for removal in Python 3.0. Required Superclass for ``raise`` --------------------------------- A SemanticsWarning will be raised when an object is passed to ``raise`` that does not have the proper inheritance. Removal of Bare ``except`` Clauses ---------------------------------- A SemanticsWarning will be raised for all bare ``except`` clauses. Rejected Ideas ============== Threads on python-dev discussing this PEP can be found at [#python-dev-thread1]_ and [#python-dev-thread2]_ KeyboardInterrupt inheriting from ControlFlowException ------------------------------------------------------ KeyboardInterrupt has been a contentious point within this hierarchy. Some view the exception more as control flow being caused by the user. But with its asynchronous cause (the user is able to trigger the exception at any point in code) its proper place is inheriting from CriticalException. It also keeps the name of the exception from being "CriticalError". Other Names for BaseException and Exception ------------------------------------------- Alternative names for BaseException and Exception have been Raisable/Exception and Exception/StandardError. The former alternatives were rejected because "Raisable" does not reflect its exception nature well enough. The latter alternatives were rejected because they do not reflect current use. DeprecationWarning Inheriting From PendingDeprecationWarning ------------------------------------------------------------ This was originally proposed because a DeprecationWarning can be viewed as a PendingDeprecationWarning that is being removed in the next version. But since enough people thought the inheritance could logically work the other way around, the idea was dropped. AttributeError Inheriting From TypeError or NameError ----------------------------------------------------- Viewing attributes as part of the interface of a type caused the idea of inheriting from TypeError. But that partially defeats the thinking of duck typing and thus the idea was dropped. Inheriting from NameError was suggested because objects can be viewed as having their own namespace where the attributes live and when an attribute is not found it is a namespace failure. This was also dropped as a possibility since not everyone shared this view. Removal of EnvironmentError --------------------------- Originally proposed based on the idea that EnvironmentError was an unneeded distinction, the BDFL overruled this idea [#python-dev4]_. Introduction of MacError and UnixError -------------------------------------- Proposed to add symmetry to WindowsError, the BDFL said they won't be used enough [#python-dev4]_. The idea of then removing WindowsError was proposed and accepted as reasonable, thus completely negating the idea of adding these exceptions. SystemError Subclassing SystemExit ---------------------------------- Proposed because a SystemError is meant to lead to a system exit, the idea was removed since CriticalException indicates this better. ControlFlowException Under Exception ------------------------------------ It has been suggested that ControlFlowException should inherit from Exception. This idea has been rejected based on the thinking that control flow exceptions typically should not be caught by bare ``except`` clauses, whereas Exception subclasses should be. Removal of Bare ``except`` Clauses ---------------------------------- The suggestion has been made to remove bare ``except`` clauses altogether, in the name of "explicit is better than implicit". But Guido has said this is too weak of an argument since other areas of Python have default behavior [#python-dev3]_. Open Issues =========== Remove ControlFlowException? ---------------------------- It has been suggested that ControlFlowException is not needed. Since the desire to catch any control flow exception will be atypical, the suggestion is to just remove the exception and let the exceptions that inherited from it inherit directly from BaseException. This still preserves the seperation from Exception which is one of the driving factors behind the introduction of ControlFlowException. Acknowledgements ================ Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing, James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull and everyone else I missed for participating in the discussion. References ========== .. [#PEP342] PEP 342 (Coroutines via Enhanced Generators) (http://www.python.org/peps/pep-0342.html) .. [#PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks) (http://www.python.org/peps/pep-0344.html) .. [#Summary2004-08-01] python-dev Summary (An exception is an exception, unless it doesn't inherit from Exception) (http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception) .. [#Summary2004-09-01] python-dev Summary (Cleaning the Exception House) (http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-exception-house) .. [#python-dev1] python-dev email (Exception hierarchy) (http://mail.python.org/pipermail/python-dev/2004-August/047908.html) .. [#python-dev2] python-dev email (Dangerous exceptions) (http://mail.python.org/pipermail/python-dev/2004-September/048681.html) .. [#python-dev3] python-dev email (PEP, take 2: Exception Reorganization for Python 3.0) (http://mail.python.org/pipermail/python-dev/2005-August/055116.html) .. [#exceptions-stdlib] exceptions module (http://docs.python.org/lib/module-exceptions.html) .. [#python-dev-thread1] python-dev thread (Pre-PEP: Exception Reorganization for Python 3.0) (http://mail.python.org/pipermail/python-dev/2005-July/055020.html, http://mail.python.org/pipermail/python-dev/2005-August/055065.html) .. [#python-dev-thread2] python-dev thread (PEP, take 2: Exception Reorganization for Python 3.0) (http://mail.python.org/pipermail/python-dev/2005-August/055103.html) .. [#python-dev4] python-dev email (Pre-PEP: Exception Reorganization for Python 3.0) (http://mail.python.org/pipermail/python-dev/2005-July/055019.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: