PEP: 348 Title: Exception Reorganization for Python 3.0 Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Jul-2005 Post-History: Abstract ======== Python, as of version 2.4, has 38 exceptions (including warnings) in the built-in namespace in a rather shallow hierarchy. These classes have come about over the years without a chance to learn from experience. This PEP proposes doing a reorganization of the hierarchy for Python 3.0 when backwards-compatibility is not as much of an issue. Along with this reorganization, adding a requirement that all objects passed to a ``raise`` statement must inherit from a specific superclass is proposed. This is to have guarantees about the basic interface of exceptions and to further enhance the natural hierarchy of exceptions. Lastly, bare ``except`` clauses will be removed. While they had their usefulness when exceptions could be any object, with the above proposal of a required superclass for exceptions bare ``except`` clauses lose their purpose. Rationale For Wanting Change ============================ Exceptions are a critical part of Python. While exceptions are traditionally used to signal errors in a program, they have also grown to be used for flow control for things such as iterators. While their importance is great, there is a lack of structure to them. This stems from the fact that any object can be raised as an exception. Because of this you have no guarantee in terms of what kind of object will be raised, destroying any possible hierarchy raised objects might adhere to. But exceptions do have a hierarchy, showing the severity of the exception. The hierarchy also groups related exceptions together to simplify catching them in ``except`` clauses. To allow peopele to be able to rely on this hierarchy, a common superclass that all raise objects must inherit from is being proposed. It also allows guarantees about the interface to raised objects to be made (see PEP 344 [#PEP344]_). A discussion about all of this has occurred before on python-dev [#Summary2004-08-01]_. With the requirement of a common superclass for all exceptions, bare ``except`` clauses are impacted. Currently used as a catch-all for raised exceptions, its usefulness is terminally weakened. Now, the same functionality is possible by catching the required superclass. Once again, this has been discussed on python-dev [#python-dev3]_. Finally, slight changes to the exception hierarchy will make it much more reasonable in terms of structure. By minor rearranging exceptions that should not typically be caught can be allowed to propagate to the top of the execution stack, terminating the interpreter as intended. Philosophy of Reorganization ============================ For the reorganization of the hierarchy, there was a general philosophy followed that developed from discussion of earlier drafts of this PEP [#python-dev-thread1]_, [#python-dev-thread2]_, [#python-dev-thread3]_. First and foremost was to not break anything that works. This meant that renaming exceptions was out of the question unless the name was deemed severely bad. This also meant no removal of exceptions unless they were viewed as truly misplaced. The introduction of new exceptions were only done in situations where there might be a use for catching a superclass of a category of exceptions. Lastly, existing exceptions would have their inheritance tree changed only if it was felt they were truly misplaced to begin with. For all new exceptions, the proper suffix had to be chosen. For those that signal an error, "Error" is to be used. If the exception is a warning, then "Warning". "Exception" is to be used when none of the other suffixes are proper to use and no specific suffix is a better fit. After that it came down to choosing which exceptions should and should not inherit from Exception. This was for the purpose of making bare ``except`` clauses more useful. Lastly, the entire existing hierarchy had to inherit from the new exception meant to act as the required superclass for all exceptions to inherit from. New Hierarchy ============= .. Note:: Exceptions flagged with "stricter inheritance" will no longer inherit from a certain class. A "broader inheritance" flag means a class has been added to the exception's inheritance tree. All comparisons are against the Python 2.4 exception hierarchy. .. parsed-literal:: +-- BaseException (new; broader inheritance for subclasses) +-- Exception +-- GeneratorExit (defined in PEP 342 [#PEP342]_) +-- StandardError +-- ArithmeticError +-- DivideByZeroError +-- FloatingPointError +-- OverflowError +-- AssertionError +-- AttributeError +-- EnvironmentError +-- IOError +-- EOFError +-- OSError +-- ImportError +-- LookupError +-- IndexError +-- KeyError +-- MemoryError +-- NameError +-- UnboundLocalError +-- NotImplementedError (stricter inheritance) +-- SyntaxError +-- IndentationError +-- TabError +-- TypeError +-- RuntimeError +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- ValueError +-- ReferenceError +-- StopIteration +-- SystemError +-- Warning +-- DeprecationWarning +-- FutureWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning + -- WindowsError +-- TerminatingException (new; stricter inheritance for subclasses) +-- KeyboardInterrupt +-- SystemExit Differences Compared to Python 2.4 ================================== A more thorough explanation of terms is needed when discussing inheritance changes. Inheritance changes result in either broader or more restrictive inheritance. "Broader" is when a class has an inheritance tree like ``cls, A`` and then becomes ``cls, B, A``. "Stricter" is the reverse. BaseException ------------- The superclass that all exceptions must inherit from. It's name was chosen to reflect that it is at the base of the exception hierarchy while being an exception itself. "Raisable" was considered as a name, it was passed on because its name did not properly reflect the fact that it is an exception itself. Direct inheritance of BaseException is not expected, and will be discouraged for the general case. Most user-defined exceptions should inherit from Exception instead. This allows catching Exception to continue to work in the common case of catching all exceptions that should be caught. Direct inheritance of BaseException should only be done in cases where an entirely new category of exception is desired. But, for cases where all exceptions should be caught blindly, ``except BaseException`` will work. TerminatingException -------------------- Superclass for exceptions that are meant to symbolize the termination of the interpreter. It does not inherit from Exception so that the common ``except Exception`` statement does not catch the exceptions. This for ``try`` statements that want to catch all exceptions that are signals of errors and handle them separately from exceptions that signify that the interpreter should be terminated:: try: ... # Catch all exceptions that are expected to be caught except Exception: ... # Catch exceptions expected to terminate the interpreter except TerminatingException: ... Compare this to:: try: ... except Exception: ... except (KeyboardInterrupt, SystemExit): ... While more explicit, it is not necessarily obvious why the two exceptions are being caught directly. By providing a common superclass with a name that explicitly states the exceptions' typical usage the reasoning behind the catch becomes more apparent. Comparing it to an even more general ``except`` clause:: try: ... except Exception: ... except BaseException: ... While this will perform the same action and catch all exceptions, guaranteed, it is once again not obvious why Exception and BaseException are being caught separately and not just BaseException based purely on the code. TerminatingException will also help with transitioning from Python 2.x to 3.0 . With TerminatingException being a new exception, when it is used the new inheritance for KeyboardInterrupt and SystemExit becomes more obvious. It has been argued that TerminatingException should not be added because it goes against Flat Is Better Than Nested (FIBTN). While this new exception will not make the hierarchy as flat is it could be, the overall depth of the tree is not changed; Exception has a much deeper inheritance tree below it. It has also been argued that since KeyboardInterrupt and SystemExit are not caught together that often in 2.4, there might not be much of a need for TerminatingException. But with their new position in the hierarchy catching them separately from other exceptions should become more prevalent. Naming is based on the idea that the interpreter is trying to terminate when KeyboardInterrupt and SystemExit are raised. An earlier proposal suggested "TerminalException" but avoidance of any confusion with an actual terminal along with "terminal" being more fatalistic than "terminating" led to the current name choice. NotImplementedError ------------------- Inherits from Exception instead of from RuntimeError. Originally inheriting from RuntimeError, NotImplementedError does not have any direct relation to the exception meant for use in user code as a quick-and-dirty exception. Thus it now directly inherits from Exception. Required Superclass for ``raise`` ================================= By requiring all objects passed to a ``raise`` statement to inherit from a specific superclass, all exceptions are guaranteed to have certain attributes. If PEP 344 [#PEP344]_ is accepted, the attributes outlined there will be guaranteed to be on all exceptions raised. This should help facilitate debugging by making the querying of information from exceptions much easier. The proposed hierarchy has BaseException as the required base class. Implementation -------------- Enforcement is straightforward. Modifying ``RAISE_VARARGS`` to do an inheritance check first before raising an exception should be enough. For the C API, all functions that set an exception will have the same inheritance check applied. Removal of Bare ``except`` Clauses ================================== Bare ``except`` clauses serve the purpose of catching all exceptions in Python 2.x . This is needed thanks to the fact that in 2.x any exception can be raised as an exception. But if this PEP is accepted this will no longer be the case because of a required superclass for all raised objects. This goes against a part of the Zen of Python; One Way To Do It (OWTDI) [#zen]_. By having bare ``except`` clauses keep their semantic meaning, there would be two ways of doing the same things. It also goes against Exlpicit Is Better Than Implicit (EIBTI) by implicitly doing something that can easily be covered by a more explicit statement. It has also been proposed that bare ``except`` clauses be changed to semantically be equivalent to ``except Exception``. This has been proposed since this is what most bare ``except`` clauses are meant to do. This would make ``except`` clauses have a more reasonable default behavior. But this line of reasoning has some issues. First is that it also goes against the Zen of Python; both OWTDI and EIBTI by providing an implicit alternative to ``except Exception``. Secondly, backwards-compatibility becomes more difficult. While removal will break code, it can mechanically be changed to have the same semantic meaning as it currently has by finding all occurances of ``except:`` and replacing them with ``except BaseException:``. But going with this semantic change makes compatibility more subtle. Was a bare ``except`` clause meant to catch all exceptions, or actually meant to catch all reasonable exceptions (i.e., everything but TerminatingException)? Every case will need to be carefully examined, and could easily be incorrectly examined, leading to subtle bugs. The shorter typing afforded by bare ``except`` statements also does not justify its existence. "Exception", the typical exception that\ will be caught, is only nine characters. Add in the needed space to separate "Exception" from "except" you get only 10 more characters to type. While this might be a nuisance in quick-and-dirty scripts, the impact is minimal. Implementation -------------- Changing Grammar/Grammar is all that is needed to remove bare ``except`` clauses. Transition Plan =============== Exception Hierarchy Changes --------------------------- New Exceptions '''''''''''''' New exceptions can simply be added to the built-in namespace. Any pre-existing objects with the same name will mask the new exceptions, preserving backwards-compatibility. New Inheritance for Old Exceptions '''''''''''''''''''''''''''''''''' Using multiple inheritance to our advantage, exceptions whose inheritance is now more resrictive can be made backwards-compatible. By inheriting from both the new superclasses as well as the original superclasses, existing ``except`` clauses will continue to work as before while allowing the new inheritance to be used for new code. A PendingDeprecationWarning will be raised based on whether the bytecode ``COMPARE_OP(10)`` results in an exception being caught that would not have under the new hierarchy. This will require hard-coding in the implementation of the bytecode. Required Superclass for ``raise`` --------------------------------- A DeprecationWarning will be raised when an object is passed to ``raise`` that does not have the proper inheritance. Removal of Bare ``except`` Clauses ---------------------------------- A PendingDeprecationWarning will be raised when a bare ``except`` clause is found in code. One release before they are removed the warning will be changed to DeprecationWarning. Doing ``from __future__ import exceptions`` will cause bare ``except`` clauses to be considered syntax errors. Roadmap ------- Python 2.x is the first version that contains changes. Python 3.0 will be when transition will be complete. Version 3.0-1 represents one version before 3.0 is released, 3.0-2 two versions before, etc. * 2.x - Add BaseException, TerminatingException - Have KeyboardInterrupt and SystemExit inherit from both Exception and TerminatingException - Introduce ``from __future__ import exceptions`` - Provide a script that mechanically changes all bare ``except`` clauses to catch BaseException in a .py file - PendingDeprecationWarning for bare ``except`` clauses - PendingDeprecationWarning for all objects raised that do not inherit from BaseException - PendingDeprecationWarning raised when KeyboardInterrupt or SystemExit are caught because of their inheritance of Exception * 3.0-1 - Turn all introduced PendingDeprecationWarnings into DeprecationWarning * 3.0 - Remove DeprecationWarnings - Have KeyboardInterrupt and SystemExit only inherit from TerminatingException - Remove ``exceptions`` __future__ import support Rejected Ideas ============== Multiple threads on python-dev discussing this PEP have lead to various ideas being rejected [#python-dev-thread1]_, [#python-dev-thread2]_, [#python-dev-thread3]_. DeprecationWarning Inheriting From PendingDeprecationWarning ------------------------------------------------------------ This was originally proposed because a DeprecationWarning can be viewed as a PendingDeprecationWarning that is being removed in the next version. But since enough people thought the inheritance could logically work the other way around, the idea was dropped. AttributeError Inheriting From TypeError or NameError ----------------------------------------------------- Viewing attributes as part of the interface of a type caused the idea of inheriting from TypeError. But that partially defeats the thinking of duck typing and thus the idea was dropped. Inheriting from NameError was suggested because objects can be viewed as having their own namespace where the attributes live and when an attribute is not found it is a namespace failure. This was also dropped as a possibility since not everyone shared this view. Removal of EnvironmentError --------------------------- Originally proposed based on the idea that EnvironmentError was an unneeded distinction, the BDFL overruled this idea [#python-dev4]_. Introduction of MacError and UnixError -------------------------------------- Proposed to add symmetry to WindowsError, the BDFL said they won't be used enough [#python-dev4]_. The idea of then removing WindowsError was proposed and accepted as reasonable, thus completely negating the idea of adding these exceptions. SystemError Subclassing SystemExit ---------------------------------- Proposed because a SystemError is meant to lead to a system exit, the idea was removed since CriticalError indicates this better. ControlFlowException Under Exception ------------------------------------ It has been suggested that ControlFlowException should inherit from Exception. This idea has been rejected based on the thinking that control flow exceptions typically do not all need to be caught by a single ``except`` clause. Rename NameError to NamespaceError ---------------------------------- NameError is considered more succinct and leaves open no possible mistyping of the capitalization of "Namespace" [#python-dev5]_. Renaming RuntimeError or Introducing SimpleError '''''''''''''''''''''''''''''''''''''''''''''''' The thinking was that RuntimeError was in no way an obvious name for an exception meant to be used when a situation did not call for the creation of a new exception. The renaming was rejected on the basis that the exception is already used throughout the interpreter [#python-dev6]_. Rejection of SimpleError was founded on the thought that people should be free to use whatever exception they choose and not have one so blatently suggested [#python-dev7]_. Renaming Existing Exceptions ---------------------------- Various renamings were suggested but non garnered more than a +0 vote (renaming ReferenceError to WeakReferenceError). The thinking was that the existing names were fine and no one had actively complained about them ever. To minimize backwards-compatibility issues and causing existing Python programmers extra pain, the renamings were removed. Have EOFError Subclass IOError ------------------------------ The original thought was that since EOFError deals directly with I/O, it should subclass IOError. But since EOFError is used more as a signal that an event has occurred (the exhaustion of an I/O port), it should not subclass such a specific error exception. Have MemoryError and SystemError Have a Common Superclass --------------------------------------------------------- Both classes deal with the interpreter, so why not have them have a common superclass? Because one of them means that the interpreter is in a state that it should not recover from while the other does not. Common Superclass for PendingDeprecationWarning and DeprecationWarning ---------------------------------------------------------------------- Grouping the deprecation warning exceptions together makes intuitive sense. But this sensical idea does not extend well when one considers how rarely either warning is used, let along at the same time. Removing WindowsError --------------------- Originally proposed based on the idea that having such a platform-specific exception should not be in the built-in namespace. It turns out, though, enough code exists that uses the exception to warrant it staying. Acknowledgements ================ Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing, James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull, Raymond Hettinger, and everyone else I missed for participating in the discussion. References ========== .. [#PEP342] PEP 342 (Coroutines via Enhanced Generators) http://www.python.org/peps/pep-0342.html .. [#PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks) http://www.python.org/peps/pep-0344.html .. [#Summary2004-08-01] python-dev Summary (An exception is an exception, unless it doesn't inherit from Exception) http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception .. [#python-dev3] python-dev email (PEP, take 2: Exception Reorganization for Python 3.0) http://mail.python.org/pipermail/python-dev/2005-August/055116.html .. [#exceptions-stdlib] exceptions module http://docs.python.org/lib/module-exceptions.html .. [#python-dev-thread1] python-dev thread (Pre-PEP: Exception Reorganization for Python 3.0) http://mail.python.org/pipermail/python-dev/2005-July/055020.html, http://mail.python.org/pipermail/python-dev/2005-August/055065.html .. [#python-dev-thread2] python-dev thread (PEP, take 2: Exception Reorganization for Python 3.0) http://mail.python.org/pipermail/python-dev/2005-August/055103.html .. [#python-dev-thread3] python-dev thread (Reorg PEP checked in) http://mail.python.org/pipermail/python-dev/2005-August/055138.html .. [#python-dev4] python-dev email (Pre-PEP: Exception Reorganization for Python 3.0) http://mail.python.org/pipermail/python-dev/2005-July/055019.html .. [#python-dev5] python-dev email (PEP, take 2: Exception Reorganization for Python 3.0) http://mail.python.org/pipermail/python-dev/2005-August/055159.html .. [#python-dev6] python-dev email (Exception Reorg PEP checked in) http://mail.python.org/pipermail/python-dev/2005-August/055149.html .. [#python-dev7] python-dev email (Exception Reorg PEP checked in) http://mail.python.org/pipermail/python-dev/2005-August/055175.html .. [#zen] PEP 20 (The Zen of Python) http://www.python.org/peps/pep-0020.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: