python-peps/pep-0463.txt

PEP: 463
Title: Exception-catching expressions
Version: $Revision$
Last-Modified: $Date$
Author: Chris Angelico <rosuav@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 15-Feb-2014
Python-Version: 3.5
Post-History: 16-Feb-2014


Abstract
========

Just as PEP 308 introduced a means of value-based conditions in an
expression, this system allows exception-based conditions to be used
as part of an expression.


Motivation
==========

A number of functions and methods have parameters which will cause
them to return a specified value instead of raising an exception.  The
current system is ad-hoc and inconsistent, and requires that each
function be individually written to have this functionality; not all
support this.

* dict.get(key, default) - second positional argument in place of
  KeyError

* next(iter, default) - second positional argument in place of
  StopIteration

* list.pop() - no way to return a default

* seq[index] - no way to handle a bounds error

* min(sequence, default=default) - keyword argument in place of
  ValueError

* sum(sequence, start=default) - slightly different but can do the
  same job

* statistics.mean(data) - no way to handle an empty iterator

Additionally, this syntax would allow a convenient way to capture
exceptions in interactive Python; returned values are captured by "_",
but exceptions currently are not. This could be spelled:

>>> expr except Exception as e: e


Rationale
=========

The current system requires that a function author predict the need
for a default, and implement support for it.  If this is not done, a
full try/except block is needed.

Since try/except is a statement, it is impossible to catch exceptions
in the middle of an expression.  Just as if/else does for conditionals
and lambda does for function definitions, so does this allow exception
catching in an expression context.

This provides a clean and consistent way for a function to provide a
default: it simply raises an appropriate exception, and the caller
catches it.

There's no convenient way to write a helper function to do this; the
nearest is something ugly using either lambda::

    def except_(expression, exception_list, default):
        try:
            return expression()
        except exception_list as e:
            return default(e)
    value = except_(lambda: 1/x, ZeroDivisionError, lambda e: float("nan"))

which is clunky, and unable to handle multiple exception clauses; or
eval::

    def except_(expression, exception_list, default):
        try:
            return eval(expression, globals_of_caller(), locals_of_caller())
        except exception_list as exc:
            l = locals_of_caller().copy()
            l['exc'] = exc
            return eval(default, globals_of_caller(), l)

    def globals_of_caller():
        return sys._getframe(2).f_globals

    def locals_of_caller():
        return sys._getframe(2).f_locals

    value = except_("""1/x""",ZeroDivisionError,""" "Can't divide by zero" """)

which is even clunkier, and relies on implementation-dependent hacks.
(Writing globals_of_caller() and locals_of_caller() for interpreters
other than CPython is left as an exercise for the reader.)

Raymond Hettinger `expresses`__ a desire for such a consistent
API. Something similar has been `requested`__ `multiple`__ `times`__
in the past.

__ https://mail.python.org/pipermail/python-ideas/2014-February/025443.html
__ https://mail.python.org/pipermail/python-ideas/2013-March/019760.html
__ https://mail.python.org/pipermail/python-ideas/2009-August/005441.html
__ https://mail.python.org/pipermail/python-ideas/2008-August/001801.html


Proposal
========

Just as the 'or' operator and the three part 'if-else' expression give
short circuiting methods of catching a falsy value and replacing it,
this syntax gives a short-circuiting method of catching an exception
and replacing it.

This currently works::

    lst = [1, 2, None, 3]
    value = lst[2] or "No value"

The proposal adds this::

    lst = [1, 2]
    value = lst[2] except IndexError: "No value"

Specifically, the syntax proposed is::

    expr except exc [as e]: default [except exc2 [as e]: default2] ...

where expr, exc, and default are all expressions.  First, expr is
evaluated.  If no exception is raised, its value is the value of the
overall expression.  If any exception is raised, exc is evaluated, and
should result in either a type or a tuple, just as with the statement
form of try/except.  Any matching exception will result in the
corresponding default expression being evaluated and becoming the
value of the expression.  As with the statement form of try/ except,
subsequent except clauses will be checked if the first one does not
match, and if none match, the raised exception will propagate upward.
Also as with the try/except statement, the keyword 'as' can be used to
bind the exception object to a local name.

Omitting the exception list should be legal, just as with the
statement form of try/except, though this should of course be
discouraged by PEP 8.

The exception object can be captured just as in a normal try/except
block::

    # Return the next yielded or returned value from a generator
    value = next(it) except StopIteration as e: e.args[0]

This is effectively equivalent to::

    try:
        _ = next(it)
    except StopIteration as e:
        _ = e.args[0]
    value = _

This ternary operator would be between lambda and if/else in
precedence.


Chaining
--------

Multiple 'except' keywords can be used, and they will all catch
exceptions raised in the original expression (only)::

    # Will catch any of the listed exceptions thrown by expr;
    # any exception thrown by a default expression will propagate.
    value = (expr
        except Exception1 [as e]: default1
        except Exception2 [as e]: default2
        # ... except ExceptionN [as e]: defaultN
    )

Using parentheses to force an alternative interpretation works as
expected::

    # Will catch an Exception2 thrown by either expr or default1
    value = (
        (expr except Exception1: default1)
        except Exception2: default2
    )
    # Will catch an Exception2 thrown by default1 only
    value = (expr except Exception1:
        (default1 except Exception2: default2)
    )

This last form is confusing and should be discouraged by PEP 8, but it
is syntactically legal: you can put any sort of expression inside a
ternary-except; ternary-except is an expression; therefore you can put
a ternary-except inside a ternary-except.


Alternative Proposals
=====================

Discussion on python-ideas brought up the following syntax suggestions::

    value = expr except default if Exception [as e]
    value = expr except default for Exception [as e]
    value = expr except default from Exception [as e]
    value = expr except Exception [as e] return default
    value = expr except (Exception [as e]: default)
    value = expr except Exception [as e] try default
    value = expr except Exception [as e] continue with default
    value = default except Exception [as e] else expr
    value = try expr except Exception [as e]: default
    value = expr except Exception [as e] pass default

It has also been suggested that a new keyword be created, rather than
reusing an existing one.  Such proposals fall into the same structure
as the last form, but with a different keyword in place of 'pass'.
Suggestions include 'then', 'when', and 'use'.

Additionally, the following has been suggested as a similar
short-hand, though not technically an expression::

    statement except Exception: pass

    try:
        statement
    except Exception:
        pass


Open Issues
===========

Commas between multiple except clauses
--------------------------------------

Where there are multiple except clauses, should they be separated by
commas?  It may be easier for the parser, that way::

    value = (expr
        except Exception1 [as e]: default1,
        except Exception2 [as e]: default2,
        # ... except ExceptionN [as e]: defaultN,
    )

with an optional comma after the last, as per tuple rules.  Downside:
Omitting the comma would be syntactically valid, and would have almost
identical semantics, but would nest the entire preceding expression in
its exception catching rig - a matching exception raised in the
default clause would be caught by the subsequent except clause.  As
this difference is so subtle, it runs the risk of being a major bug
magnet.

As a mitigation of this risk, this form::

    value = expr except Exception1: default1 except Exception2: default2

could be syntactically forbidden, and parentheses required if the
programmer actually wants that behaviour::

    value = (expr except Exception1: default1) except Exception2: default2

This would prevent the accidental omission of a comma from changing
the expression's meaning.


Parentheses around the entire expression
----------------------------------------

Generator expressions require parentheses, unless they would be
strictly redundant.  Ambiguities with except expressions could be
resolved in the same way, forcing nested except-in-except trees to be
correctly parenthesized and requiring that the outer expression be
clearly delineated.  `Steven D'Aprano elaborates on the issue.`__

__ https://mail.python.org/pipermail/python-ideas/2014-February/025647.html


Parentheses around the except clauses
-------------------------------------

Should it be legal to parenthesize the except clauses, separately from
the expression that could raise? Example::

    value = expr (
        except Exception1 [as e]: default1
        except Exception2 [as e]: default2
        # ... except ExceptionN [as e]: defaultN
    )


Scope of default expressions and 'as'
-------------------------------------

In a try/except block, the use of 'as' to capture the exception object
creates a local name binding, and implicitly deletes that binding in a
finally clause.  As 'finally' is not a part of this proposal (see
below), this makes it tricky to describe; also, this use of 'as' gives
a way to create a name binding in an expression context.  Should the
default clause have an inner scope in which the name exists, shadowing
anything of the same name elsewhere? Should it behave the same way the
statement try/except does, and unbind the name?  Should it bind the
name and leave it bound? (Almost certainly not; this behaviour was
changed in Python 3 for good reason.)

(The inner scope idea is tempting, but currently CPython handles list
comprehensions with a nested function call, as this is considered
easier.  It may be of value to simplify both comprehensions and except
expressions, but that is a completely separate proposal to this PEP;
alternatively, it may be better to stick with what's known to
work. `Nick Coghlan elaborates.`__)

__ https://mail.python.org/pipermail/python-ideas/2014-February/025702.html


Example usage
=============

For each example, an approximately-equivalent statement form is given,
to show how the expression will be parsed.  These are not always
strictly equivalent, but will accomplish the same purpose.  It is NOT
safe for the interpreter to translate one into the other.

Perform some lengthy calculations in EAFP mode, handling division by
zero as a sort of sticky NaN::

    value = calculate(x) except ZeroDivisionError: float("nan")

    try:
        value = calculate(x)
    except ZeroDivisionError:
        value = float("nan")

Retrieving from a generator, either the next yielded value or the
returned, and coping with the absence of such a return value::

    value = (next(it)
        except StopIteration as e:
            (e.args[0] except IndexError: None)
    )

    try:
        value = next(it)
    except StopIteration as e:
        try:
            value = e.args[0]
        except IndexError:
            value = None

Calculate the mean of a series of numbers, falling back on zero::

    value = statistics.mean(lst) except statistics.StatisticsError: 0

    try:
        value = statistics.mean(lst)
    except statistics.StatisticsError:
        value = 0

Set a PyGTK label to a human-readable result from fetching a URL::

    display.set_text(
        urllib.request.urlopen(url)
        except urllib.error.HTTPError as e: "Error %d: %s"%(x.getcode(), x.msg)
        except (ValueError, urllib.error.URLError) as e: "Invalid URL: "+str(e)
    )

    try:
        display.set_text(urllib.request.urlopen(url))
    except urllib.error.HTTPError as e:
        display.set_text("Error %d: %s"%(x.getcode(), x.msg))
    except (ValueError, urllib.error.URLError) as e:
        display.set_text("Invalid URL: "+str(e))

Retrieving a message from either a cache or the internet,with auth
check::

    logging.info("Message shown to user: %s",((cache[k]
        except LookupError:
            (backend.read(k) except OSError: 'Resource not available')
        )
        if check_permission(k) else 'Access denied'
    ) except: "I'm an idiot and using a bare except clause")

    try:
        if check_permission(k):
            try:
                _ = cache[k]
            except LookupError:
                try:
                    _ = backend.read(k)
                except OSError:
                    _ = 'Resource not available'
        else:
            _ = 'Access denied'
    except:
        _ = "I'm an idiot and using a bare except clause"
    logging.info("Message shown to user: %s", _)

Looking up objects in a sparse list of overrides::

    (overrides[x] or default except IndexError: default).ping()

    try:
        (overrides[x] or default).ping()
    except IndexError:
        default.ping()


Rejected sub-proposals
======================

The statement form try... finally or try... except... finally has no
logical corresponding expression form.  Therefore the finally keyword
is not a part of this proposal, in any way.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: