iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

reSTify 5 PEPs (#276)

This commit is contained in:
csabella 2017-06-02 14:46:13 -04:00 committed by Brett Cannon
parent 02969ebbd2
commit 329ed7e935
5 changed files with 1597 additions and 1535 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

412
pep-0236.txt
View File

@ -5,80 +5,84 @@ Last-Modified: $Date$
Author: Tim Peters <tim.peters@gmail.com>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 26-Feb-2001
Python-Version: 2.1
Post-History: 26-Feb-2001
Motivation
==========
From time to time, Python makes an incompatible change to the
advertised semantics of core language constructs, or changes their
accidental (implementation-dependent) behavior in some way. While this
is never done capriciously, and is always done with the aim of
improving the language over the long term, over the short term it's
contentious and disrupting.
From time to time, Python makes an incompatible change to the advertised
semantics of core language constructs, or changes their accidental
(implementation-dependent) behavior in some way. While this is never done
capriciously, and is always done with the aim of improving the language over
the long term, over the short term it's contentious and disrupting.
PEP 5, Guidelines for Language Evolution[1] suggests ways to ease
the pain, and this PEP introduces some machinery in support of that.
PEP 5, Guidelines for Language Evolution [1]_ suggests ways to ease the pain,
and this PEP introduces some machinery in support of that.
PEP 227, Statically Nested Scopes[2] is the first application, and
will be used as an example here.
PEP 227, Statically Nested Scopes [2]_ is the first application, and will be
used as an example here.
Intent
======
[Note: This is policy, and so should eventually move into PEP 5 [1]]
[Note: This is policy, and so should eventually move into PEP 5 [1]_]
When an incompatible change to core language syntax or semantics is
being made:
When an incompatible change to core language syntax or semantics is being
made:
1. The release C that introduces the change does not change the
syntax or semantics by default.
1. The release C that introduces the change does not change the syntax or
semantics by default.
2. A future release R is identified in which the new syntax or semantics
will be enforced.
2. A future release R is identified in which the new syntax or semantics will
be enforced.
3. The mechanisms described in PEP 3, Warning Framework[3] are
used to generate warnings, whenever possible, about constructs
or operations whose meaning may[4] change in release R.
3. The mechanisms described in PEP 3, Warning Framework [3]_ are used to
generate warnings, whenever possible, about constructs or operations whose
meaning may [4]_ change in release R.
4. The new future_statement (see below) can be explicitly included in a
module M to request that the code in module M use the new syntax or
semantics in the current release C.
4. The new future_statement (see below) can be explicitly included in a module
M to request that the code in module M use the new syntax or semantics in
the current release C.
So old code continues to work by default, for at least one release,
although it may start to generate new warning messages. Migration to
the new syntax or semantics can proceed during that time, using the
future_statement to make modules containing it act as if the new syntax
or semantics were already being enforced.
So old code continues to work by default, for at least one release, although
it may start to generate new warning messages. Migration to the new syntax or
semantics can proceed during that time, using the future_statement to make
modules containing it act as if the new syntax or semantics were already being
enforced.
Note that there is no need to involve the future_statement machinery
in new features unless they can break existing code; fully backward-
compatible additions can-- and should --be introduced without a
corresponding future_statement.
Note that there is no need to involve the future_statement machinery in new
features unless they can break existing code; fully backward- compatible
additions can-- and should --be introduced without a corresponding
future_statement.
Syntax
======
A future_statement is simply a from/import statement using the reserved
module name __future__:
A future_statement is simply a from/import statement using the reserved module
name ``__future__``::
future_statement: "from" "__future__" "import" feature ["as" name]
("," feature ["as" name])*
(","feature ["as" name])*
feature: identifier
name: identifier
In addition, all future_statments must appear near the top of the
module. The only lines that can appear before a future_statement are:
In addition, all future_statments must appear near the top of the module. The
only lines that can appear before a future_statement are:
+ The module docstring (if any).
+ Comments.
+ Blank lines.
+ Other future_statements.
+ The module docstring (if any).
+ Comments.
+ Blank lines.
+ Other future_statements.
Example::
Example:
"""This is a module docstring."""
# This is a comment, preceded by a blank line and followed by
@ -91,37 +95,38 @@ Syntax
Semantics
=========
A future_statement is recognized and treated specially at compile time:
changes to the semantics of core constructs are often implemented by
generating different code. It may even be the case that a new feature
introduces new incompatible syntax (such as a new reserved word), in
which case the compiler may need to parse the module differently. Such
decisions cannot be pushed off until runtime.
A future_statement is recognized and treated specially at compile time:
changes to the semantics of core constructs are often implemented by
generating different code. It may even be the case that a new feature
introduces new incompatible syntax (such as a new reserved word), in which
case the compiler may need to parse the module differently. Such decisions
cannot be pushed off until runtime.
For any given release, the compiler knows which feature names have been
defined, and raises a compile-time error if a future_statement contains
a feature not known to it[5].
For any given release, the compiler knows which feature names have been
defined, and raises a compile-time error if a future_statement contains a
feature not known to it [5]_.
The direct runtime semantics are the same as for any import statement:
there is a standard module __future__.py, described later, and it will
be imported in the usual way at the time the future_statement is
executed.
The direct runtime semantics are the same as for any ``import`` statement:
there is a standard module ``__future__.py``, described later, and it will be
imported in the usual way at the time the future_statement is executed.
The *interesting* runtime semantics depend on the specific feature(s)
"imported" by the future_statement(s) appearing in the module.
The *interesting* runtime semantics depend on the specific feature(s)
"imported" by the future_statement(s) appearing in the module.
Note that there is nothing special about the statement:
Note that there is nothing special about the statement::
import __future__ [as name]
That is not a future_statement; it's an ordinary import statement, with
no special semantics or syntax restrictions.
That is not a future_statement; it's an ordinary import statement, with no
special semantics or syntax restrictions.
Example
=======
Consider this code, in file scope.py:
Consider this code, in file scope.py::
x = 42
def f():
@ -131,243 +136,244 @@ Example
g()
f()
Under 2.0, it prints:
Under 2.0, it prints::
x is 42
Nested scopes[2] are being introduced in 2.1. But under 2.1, it still
prints
Nested scopes [2]_ are being introduced in 2.1. But under 2.1, it still
prints::
x is 42
and also generates a warning.
and also generates a warning.
In 2.2, and also in 2.1 *if* "from __future__ import nested_scopes" is
included at the top of scope.py, it prints
In 2.2, and also in 2.1 *if* ``from __future__ import nested_scopes`` is
included at the top of ``scope.py``, it prints::
x is 666
Standard Module __future__.py
=============================
Lib/__future__.py is a real module, and serves three purposes:
``Lib/__future__.py`` is a real module, and serves three purposes:
1. To avoid confusing existing tools that analyze import statements and
expect to find the modules they're importing.
1. To avoid confusing existing tools that analyze import statements and expect
to find the modules they're importing.
2. To ensure that future_statements run under releases prior to 2.1
at least yield runtime exceptions (the import of __future__ will
fail, because there was no module of that name prior to 2.1).
2. To ensure that future_statements run under releases prior to 2.1 at least
yield runtime exceptions (the import of ``__future__`` will fail, because
there was no module of that name prior to 2.1).
3. To document when incompatible changes were introduced, and when they
will be-- or were --made mandatory. This is a form of executable
documentation, and can be inspected programatically via importing
__future__ and examining its contents.
3. To document when incompatible changes were introduced, and when they will
be-- or were --made mandatory. This is a form of executable documentation,
and can be inspected programatically via importing ``__future__`` and
examining its contents.
Each statement in __future__.py is of the form:
Each statement in ``__future__.py`` is of the form::
FeatureName = "_Feature(" OptionalRelease "," MandatoryRelease ")"
where, normally, OptionalRelease < MandatoryRelease, and both are
5-tuples of the same form as sys.version_info:
where, normally, *OptionalRelease* < *MandatoryRelease*, and both are
5-tuples of the same form as ``sys.version_info``::
(PY_MAJOR_VERSION, # the 2 in 2.1.0a3; an int
PY_MINOR_VERSION, # the 1; an int
PY_MICRO_VERSION, # the 0; an int
PY_RELEASE_LEVEL, # "alpha", "beta", "candidate" or "final"; string
PY_RELEASE_SERIAL # the 3; an int
)
PY_RELEASE_SERIAL # the 3; an int )
OptionalRelease records the first release in which
*OptionalRelease* records the first release in which::
from __future__ import FeatureName
was accepted.
was accepted.
In the case of MandatoryReleases that have not yet occurred,
MandatoryRelease predicts the release in which the feature will become
part of the language.
In the case of *MandatoryReleases* that have not yet occurred,
*MandatoryRelease* predicts the release in which the feature will become part
of the language.
Else MandatoryRelease records when the feature became part of the
language; in releases at or after that, modules no longer need
Else *MandatoryRelease* records when the feature became part of the language;
in releases at or after that, modules no longer need::
from __future__ import FeatureName
to use the feature in question, but may continue to use such imports.
to use the feature in question, but may continue to use such imports.
MandatoryRelease may also be None, meaning that a planned feature got
dropped.
*MandatoryRelease* may also be ``None``, meaning that a planned feature got
dropped.
Instances of class _Feature have two corresponding methods,
.getOptionalRelease() and .getMandatoryRelease().
Instances of ``class _Feature`` have two corresponding methods,
``.getOptionalRelease()`` and ``.getMandatoryRelease()``.
No feature line will ever be deleted from __future__.py.
No feature line will ever be deleted from ``__future__.py``.
Example line:
Example line::
nested_scopes = _Feature((2, 1, 0, "beta", 1), (2, 2, 0, "final", 0))
This means that
This means that::
from __future__ import nested_scopes
will work in all releases at or after 2.1b1, and that nested_scopes are
intended to be enforced starting in release 2.2.
will work in all releases at or after 2.1b1, and that nested_scopes are
intended to be enforced starting in release 2.2.
Resolved Problem: Runtime Compilation
======================================
Several Python features can compile code during a module's runtime:
Several Python features can compile code during a module's runtime:
1. The exec statement.
2. The execfile() function.
3. The compile() function.
4. The eval() function.
5. The input() function.
1. The ``exec`` statement.
2. The ``execfile()`` function.
3. The ``compile()`` function.
4. The ``eval()`` function.
5. The ``input()`` function.
Since a module M containing a future_statement naming feature F
explicitly requests that the current release act like a future release
with respect to F, any code compiled dynamically from text passed to
one of these from within M should probably also use the new syntax or
semantics associated with F. The 2.1 release does behave this way.
Since a module M containing a future_statement naming feature F explicitly
requests that the current release act like a future release with respect to F,
any code compiled dynamically from text passed to one of these from within M
should probably also use the new syntax or semantics associated with F. The
2.1 release does behave this way.
This isn't always desired, though. For example, doctest.testmod(M)
compiles examples taken from strings in M, and those examples should
use M's choices, not necessarily the doctest module's choices. In the
2.1 release, this isn't possible, and no scheme has yet been suggested
for working around this. NOTE: PEP 264 later addressed this in a
flexible way, by adding optional arguments to compile().
This isn't always desired, though. For example, ``doctest.testmod(M)``
compiles examples taken from strings in M, and those examples should use M's
choices, not necessarily the doctest module's choices. In the 2.1 release,
this isn't possible, and no scheme has yet been suggested for working around
this. NOTE: PEP 264 later addressed this in a flexible way, by adding
optional arguments to ``compile()``.
In any case, a future_statement appearing "near the top" (see Syntax
above) of text compiled dynamically by an exec, execfile() or compile()
applies to the code block generated, but has no further effect on the
module that executes such an exec, execfile() or compile(). This
can't be used to affect eval() or input(), however, because they only
allow expression input, and a future_statement is not an expression.
In any case, a future_statement appearing "near the top" (see Syntax above) of
text compiled dynamically by an ``exec``, ``execfile()`` or ``compile()``
applies to the code block generated, but has no further effect on the module
that executes such an ``exec``, ``execfile()`` or ``compile()``. This can't
be used to affect ``eval()`` or ``input()``, however, because they only allow
expression input, and a future_statement is not an expression.
Resolved Problem: Native Interactive Shells
============================================
There are two ways to get an interactive shell:
There are two ways to get an interactive shell:
1. By invoking Python from a command line without a script argument.
1. By invoking Python from a command line without a script argument.
2. By invoking Python from a command line with the -i switch and with a
2. By invoking Python from a command line with the ``-i`` switch and with a
script argument.
An interactive shell can be seen as an extreme case of runtime
compilation (see above): in effect, each statement typed at an
interactive shell prompt runs a new instance of exec, compile() or
execfile(). A future_statement typed at an interactive shell applies to
the rest of the shell session's life, as if the future_statement had
appeared at the top of a module.
An interactive shell can be seen as an extreme case of runtime compilation
(see above): in effect, each statement typed at an interactive shell prompt
runs a new instance of ``exec``, ``compile()`` or ``execfile()``. A
future_statement typed at an interactive shell applies to the rest of the
shell session's life, as if the future_statement had appeared at the top of a
module.
Resolved Problem: Simulated Interactive Shells
===============================================
Interactive shells "built by hand" (by tools such as IDLE and the Emacs
Python-mode) should behave like native interactive shells (see above).
However, the machinery used internally by native interactive shells has
not been exposed, and there isn't a clear way for tools building their
own interactive shells to achieve the desired behavior.
Interactive shells "built by hand" (by tools such as IDLE and the Emacs
Python-mode) should behave like native interactive shells (see above).
However, the machinery used internally by native interactive shells has not
been exposed, and there isn't a clear way for tools building their own
interactive shells to achieve the desired behavior.
NOTE: PEP 264 later addressed this, by adding intelligence to the
standard codeop.py. Simulated shells that don't use the standard
library shell helpers can get a similar effect by exploiting the
new optional arguments to compile() added by PEP 264.
NOTE: PEP 264 later addressed this, by adding intelligence to the standard
``codeop.py``. Simulated shells that don't use the standard library shell
helpers can get a similar effect by exploiting the new optional arguments to
``compile()`` added by PEP 264.
Questions and Answers
=====================
Q: What about a "from __past__" version, to get back *old* behavior?
What about a "from __past__" version, to get back *old* behavior?
-----------------------------------------------------------------
A: Outside the scope of this PEP. Seems unlikely to the author,
though. Write a PEP if you want to pursue it.
Outside the scope of this PEP. Seems unlikely to the author, though. Write a
PEP if you want to pursue it.
Q: What about incompatibilities due to changes in the Python virtual
machine?
What about incompatibilities due to changes in the Python virtual machine?
--------------------------------------------------------------------------
A: Outside the scope of this PEP, although PEP 5 [1] suggests a grace
period there too, and the future_statement may also have a role to
play there.
Outside the scope of this PEP, although PEP 5 [1]_ suggests a grace period
there too, and the future_statement may also have a role to play there.
Q: What about incompatibilities due to changes in Python's C API?
What about incompatibilities due to changes in Python's C API?
--------------------------------------------------------------
A: Outside the scope of this PEP.
Outside the scope of this PEP.
Q: I want to wrap future_statements in try/except blocks, so I can
use different code depending on which version of Python I'm running.
Why can't I?
I want to wrap future_statements in try/except blocks, so I can use different code depending on which version of Python I'm running. Why can't I?
-------------------------------------------------------------------------------------------------------------------------------------------------
A: Sorry! try/except is a runtime feature; future_statements are
primarily compile-time gimmicks, and your try/except happens long
after the compiler is done. That is, by the time you do
try/except, the semantics in effect for the module are already a
done deal. Since the try/except wouldn't accomplish what it
*looks* like it should accomplish, it's simply not allowed. We
also want to keep these special statements very easy to find and to
recognize.
Sorry! ``try/except`` is a runtime feature; future_statements are primarily
compile-time gimmicks, and your ``try/except`` happens long after the compiler
is done. That is, by the time you do ``try/except``, the semantics in effect
for the module are already a done deal. Since the ``try/except`` wouldn't
accomplish what it *looks* like it should accomplish, it's simply not allowed.
We also want to keep these special statements very easy to find and to
recognize.
Note that you *can* import __future__ directly, and use the
information in it, along with sys.version_info, to figure out where
the release you're running under stands in relation to a given
feature's status.
Note that you *can* import ``__future__`` directly, and use the information in
it, along with ``sys.version_info``, to figure out where the release you're
running under stands in relation to a given feature's status.
Q: Going back to the nested_scopes example, what if release 2.2 comes
along and I still haven't changed my code? How can I keep the 2.1
behavior then?
Going back to the nested_scopes example, what if release 2.2 comes along and I still haven't changed my code? How can I keep the 2.1 behavior then?
----------------------------------------------------------------------------------------------------------------------------------------------------
A: By continuing to use 2.1, and not moving to 2.2 until you do change
your code. The purpose of future_statement is to make life easier
for people who keep current with the latest release in a timely
fashion. We don't hate you if you don't, but your problems are
much harder to solve, and somebody with those problems will need to
write a PEP addressing them. future_statement is aimed at a
different audience.
By continuing to use 2.1, and not moving to 2.2 until you do change your
code. The purpose of future_statement is to make life easier for people who
keep current with the latest release in a timely fashion. We don't hate you
if you don't, but your problems are much harder to solve, and somebody with
those problems will need to write a PEP addressing them. future_statement is
aimed at a different audience.
Q: Overloading "import" sucks. Why not introduce a new statement for
this?
Overloading ``import`` sucks. Why not introduce a new statement for this?
--------------------------------------------------------------------------
A: Like maybe "lambda lambda nested_scopes"? That is, unless we
introduce a new keyword, we can't introduce an entirely new
statement. But if we introduce a new keyword, that in itself
would break old code. That would be too ironic to bear. Yes,
overloading "import" does suck, but not as energetically as the
alternatives -- as is, future_statements are 100% backward
compatible.
Like maybe ``lambda lambda nested_scopes``? That is, unless we introduce a
new keyword, we can't introduce an entirely new statement. But if we
introduce a new keyword, that in itself would break old code. That would be
too ironic to bear. Yes, overloading ``import`` does suck, but not as
energetically as the alternatives -- as is, future_statements are 100%
backward compatible.
Copyright
=========
This document has been placed in the public domain.
This document has been placed in the public domain.
References and Footnotes
========================
[1] PEP 5, Guidelines for Language Evolution, Prescod
.. [1] PEP 5, Guidelines for Language Evolution, Prescod
http://www.python.org/dev/peps/pep-0005/
[2] PEP 227, Statically Nested Scopes, Hylton
.. [2] PEP 227, Statically Nested Scopes, Hylton
http://www.python.org/dev/peps/pep-0227/
[3] PEP 230, Warning Framework, Van Rossum
.. [3] PEP 230, Warning Framework, Van Rossum
http://www.python.org/dev/peps/pep-0230/
[4] Note that this is "may" and not "will": better safe than sorry. Of
course spurious warnings won't be generated when avoidable with
reasonable cost.
.. [4] Note that this is *may* and not *will*: better safe than sorry. Of course
spurious warnings won't be generated when avoidable with reasonable cost.
[5] This ensures that a future_statement run under a release prior to
the first one in which a given feature is known (but >= 2.1) will
raise a compile-time error rather than silently do a wrong thing.
If transported to a release prior to 2.1, a runtime error will be
raised because of the failure to import __future__ (no such module
existed in the standard distribution before the 2.1 release, and
the double underscores make it a reserved name).
.. [5] This ensures that a future_statement run under a release prior to the
first one in which a given feature is known (but >= 2.1) will raise a
compile-time error rather than silently do a wrong thing. If transported
to a release prior to 2.1, a runtime error will be raised because of the
failure to import ``__future__`` (no such module existed in the standard
distribution before the 2.1 release, and the double underscores make it a
reserved name).
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:

202
pep-0247.txt
View File

@ -5,104 +5,100 @@ Last-Modified: $Date$
Author: A.M. Kuchling <amk@amk.ca>
Status: Final
Type: Informational
Content-Type: text/x-rst
Created: 23-Mar-2001
Post-History: 20-Sep-2001
Abstract
There are several different modules available that implement
cryptographic hashing algorithms such as MD5 or SHA. This
document specifies a standard API for such algorithms, to make it
easier to switch between different implementations.
Abstract
========
There are several different modules available that implement cryptographic
hashing algorithms such as MD5 or SHA. This document specifies a standard API
for such algorithms, to make it easier to switch between different
implementations.
Specification
=============
All hashing modules should present the same interface. Additional
methods or variables can be added, but those described in this
document should always be present.
All hashing modules should present the same interface. Additional methods or
variables can be added, but those described in this document should always be
present.
Hash function modules define one function:
Hash function modules define one function:
new([string]) (unkeyed hashes)
new([key] , [string]) (keyed hashes)
| ``new([string]) (unkeyed hashes)``
| ``new([key] , [string]) (keyed hashes)``
Create a new hashing object and return it. The first form is
for hashes that are unkeyed, such as MD5 or SHA. For keyed
hashes such as HMAC, 'key' is a required parameter containing
a string giving the key to use. In both cases, the optional
'string' parameter, if supplied, will be immediately hashed
into the object's starting state, as if obj.update(string) was
called.
Create a new hashing object and return it. The first form is for hashes
that are unkeyed, such as MD5 or SHA. For keyed hashes such as HMAC, *key*
is a required parameter containing a string giving the key to use. In both
cases, the optional *string* parameter, if supplied, will be immediately
hashed into the object's starting state, as if ``obj.update(string)``
was called.
After creating a hashing object, arbitrary strings can be fed
into the object using its update() method, and the hash value
can be obtained at any time by calling the object's digest()
method.
After creating a hashing object, arbitrary strings can be fed into the
object using its ``update()`` method, and the hash value can be obtained at
any time by calling the object's ``digest()`` method.
Arbitrary additional keyword arguments can be added to this
function, but if they're not supplied, sensible default values
should be used. For example, 'rounds' and 'digest_size'
keywords could be added for a hash function which supports a
variable number of rounds and several different output sizes,
and they should default to values believed to be secure.
Arbitrary additional keyword arguments can be added to this function, but if
they're not supplied, sensible default values should be used. For example,
``rounds`` and ``digest_size`` keywords could be added for a hash function
which supports a variable number of rounds and several different output
sizes, and they should default to values believed to be secure.
Hash function modules define one variable:
Hash function modules define one variable:
digest_size
| ``digest_size``
An integer value; the size of the digest produced by the
hashing objects created by this module, measured in bytes.
You could also obtain this value by creating a sample object
and accessing its 'digest_size' attribute, but it can be
convenient to have this value available from the module.
Hashes with a variable output size will set this variable to
None.
An integer value; the size of the digest produced by the hashing objects
created by this module, measured in bytes. You could also obtain this value
by creating a sample object and accessing its ``digest_size`` attribute, but
it can be convenient to have this value available from the module. Hashes
with a variable output size will set this variable to ``None``.
Hashing objects require a single attribute:
Hashing objects require a single attribute:
digest_size
| ``digest_size``
This attribute is identical to the module-level digest_size
variable, measuring the size of the digest produced by the
hashing object, measured in bytes. If the hash has a variable
output size, this output size must be chosen when the hashing
object is created, and this attribute must contain the
selected size. Therefore, None is *not* a legal value for this
This attribute is identical to the module-level ``digest_size`` variable,
measuring the size of the digest produced by the hashing object, measured in
bytes. If the hash has a variable output size, this output size must be
chosen when the hashing object is created, and this attribute must contain
the selected size. Therefore, ``None`` is *not* a legal value for this
attribute.
Hashing objects require the following methods:
Hashing objects require the following methods:
copy()
| ``copy()``
Return a separate copy of this hashing object. An update to
this copy won't affect the original object.
Return a separate copy of this hashing object. An update to this copy won't
affect the original object.
digest()
| ``digest()``
Return the hash value of this hashing object as a string
containing 8-bit data. The object is not altered in any way
by this function; you can continue updating the object after
calling this function.
Return the hash value of this hashing object as a string containing 8-bit
data. The object is not altered in any way by this function; you can
continue updating the object after calling this function.
hexdigest()
| ``hexdigest()``
Return the hash value of this hashing object as a string
containing hexadecimal digits. Lowercase letters should be used
for the digits 'a' through 'f'. Like the .digest() method, this
method mustn't alter the object.
Return the hash value of this hashing object as a string containing
hexadecimal digits. Lowercase letters should be used for the digits ``a``
through ``f``. Like the ``.digest()`` method, this method mustn't alter the
object.
update(string)
| ``update(string)``
Hash 'string' into the current state of the hashing object.
update() can be called any number of times during a hashing
object's lifetime.
Hash *string* into the current state of the hashing object. ``update()`` can
be called any number of times during a hashing object's lifetime.
Hashing modules can define additional module-level functions or
object methods and still be compliant with this specification.
Hashing modules can define additional module-level functions or object methods
and still be compliant with this specification.
Here's an example, using a module named 'MD5':
Here's an example, using a module named ``MD5``::
>>> from Crypto.Hash import MD5
>>> m = MD5.new()
@ -118,56 +114,60 @@ Specification
Rationale
=========
The digest size is measured in bytes, not bits, even though hash
algorithm sizes are usually quoted in bits; MD5 is a 128-bit
algorithm and not a 16-byte one, for example. This is because, in
the sample code I looked at, the length in bytes is often needed
(to seek ahead or behind in a file; to compute the length of an
output string) while the length in bits is rarely used.
Therefore, the burden will fall on the few people actually needing
the size in bits, who will have to multiply digest_size by 8.
The digest size is measured in bytes, not bits, even though hash algorithm
sizes are usually quoted in bits; MD5 is a 128-bit algorithm and not a 16-byte
one, for example. This is because, in the sample code I looked at, the length
in bytes is often needed (to seek ahead or behind in a file; to compute the
length of an output string) while the length in bits is rarely used. Therefore,
the burden will fall on the few people actually needing the size in bits, who
will have to multiply ``digest_size`` by 8.
It's been suggested that the update() method would be better named
append(). However, that method is really causing the current
state of the hashing object to be updated, and update() is already
used by the md5 and sha modules included with Python, so it seems
simplest to leave the name update() alone.
It's been suggested that the ``update()`` method would be better named
``append()``. However, that method is really causing the current state of the
hashing object to be updated, and ``update()`` is already used by the md5 and
sha modules included with Python, so it seems simplest to leave the name
``update()`` alone.
The order of the constructor's arguments for keyed hashes was a
sticky issue. It wasn't clear whether the key should come first
or second. It's a required parameter, and the usual convention is
to place required parameters first, but that also means that the
'string' parameter moves from the first position to the second.
It would be possible to get confused and pass a single argument to
a keyed hash, thinking that you're passing an initial string to an
unkeyed hash, but it doesn't seem worth making the interface
for keyed hashes more obscure to avoid this potential error.
The order of the constructor's arguments for keyed hashes was a sticky issue.
It wasn't clear whether the *key* should come first or second. It's a required
parameter, and the usual convention is to place required parameters first, but
that also means that the *string* parameter moves from the first position to
the second. It would be possible to get confused and pass a single argument to
a keyed hash, thinking that you're passing an initial string to an unkeyed
hash, but it doesn't seem worth making the interface for keyed hashes more
obscure to avoid this potential error.
Changes
=======
2001-09-17: Renamed clear() to reset(); added digest_size attribute
to objects; added .hexdigest() method.
2001-09-20: Removed reset() method completely.
2001-09-28: Set digest_size to None for variable-size hashes.
2001-09-17: Renamed ``clear()`` to ``reset()``; added ``digest_size`` attribute
to objects; added ``.hexdigest()`` method.
2001-09-20: Removed ``reset()`` method completely.
2001-09-28: Set ``digest_size`` to ``None`` for variable-size hashes.
Acknowledgements
================
Thanks to Aahz, Andrew Archibald, Rich Salz, Itamar
Shtull-Trauring, and the readers of the python-crypto list for
their comments on this PEP.
Thanks to Aahz, Andrew Archibald, Rich Salz, Itamar Shtull-Trauring, and the
readers of the python-crypto list for their comments on this PEP.
Copyright
=========
This document has been placed in the public domain.
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:

629
pep-0342.txt
View File

@ -5,161 +5,162 @@ Last-Modified: $Date$
Author: Guido van Rossum, Phillip J. Eby
Status: Final
Type: Standards Track
Content-Type: text/plain
Content-Type: text/x-rst
Created: 10-May-2005
Python-Version: 2.5
Post-History:
Introduction
============
This PEP proposes some enhancements to the API and syntax of
generators, to make them usable as simple coroutines. It is
basically a combination of ideas from these two PEPs, which
may be considered redundant if this PEP is accepted:
This PEP proposes some enhancements to the API and syntax of generators, to
make them usable as simple coroutines. It is basically a combination of ideas
from these two PEPs, which may be considered redundant if this PEP is
accepted:
- PEP 288, Generators Attributes and Exceptions. The current PEP
covers its second half, generator exceptions (in fact the
throw() method name was taken from PEP 288). PEP 342 replaces
generator attributes, however, with a concept from an earlier
revision of PEP 288, the "yield expression".
- PEP 288, Generators Attributes and Exceptions. The current PEP covers its
second half, generator exceptions (in fact the ``throw()`` method name was
taken from PEP 288). PEP 342 replaces generator attributes, however, with a
concept from an earlier revision of PEP 288, the *yield expression*.
- PEP 325, Resource-Release Support for Generators. PEP 342 ties up a few
loose ends in the PEP 325 spec, to make it suitable for actual
implementation.
- PEP 325, Resource-Release Support for Generators. PEP 342
ties up a few loose ends in the PEP 325 spec, to make it suitable
for actual implementation.
Motivation
==========
Coroutines are a natural way of expressing many algorithms, such as
simulations, games, asynchronous I/O, and other forms of event-
driven programming or co-operative multitasking. Python's generator
functions are almost coroutines -- but not quite -- in that they
allow pausing execution to produce a value, but do not provide for
values or exceptions to be passed in when execution resumes. They
also do not allow execution to be paused within the "try" portion of
try/finally blocks, and therefore make it difficult for an aborted
coroutine to clean up after itself.
Coroutines are a natural way of expressing many algorithms, such as
simulations, games, asynchronous I/O, and other forms of event-driven
programming or co-operative multitasking. Python's generator functions are
almost coroutines -- but not quite -- in that they allow pausing execution to
produce a value, but do not provide for values or exceptions to be passed in
when execution resumes. They also do not allow execution to be paused within
the ``try`` portion of ``try/finally`` blocks, and therefore make it difficult
for an aborted coroutine to clean up after itself.
Also, generators cannot yield control while other functions are
executing, unless those functions are themselves expressed as
generators, and the outer generator is written to yield in response
to values yielded by the inner generator. This complicates the
implementation of even relatively simple use cases like asynchronous
communications, because calling any functions either requires the
generator to "block" (i.e. be unable to yield control), or else a
lot of boilerplate looping code must be added around every needed
function call.
Also, generators cannot yield control while other functions are executing,
unless those functions are themselves expressed as generators, and the outer
generator is written to yield in response to values yielded by the inner
generator. This complicates the implementation of even relatively simple use
cases like asynchronous communications, because calling any functions either
requires the generator to *block* (i.e. be unable to yield control), or else a
lot of boilerplate looping code must be added around every needed function
call.
However, if it were possible to pass values or exceptions *into* a
generator at the point where it was suspended, a simple co-routine
scheduler or "trampoline function" would let coroutines "call" each
other without blocking -- a tremendous boon for asynchronous
applications. Such applications could then write co-routines to
do non-blocking socket I/O by yielding control to an I/O scheduler
until data has been sent or becomes available. Meanwhile, code that
performs the I/O would simply do something like this:
However, if it were possible to pass values or exceptions *into* a generator at
the point where it was suspended, a simple co-routine scheduler or *trampoline
function* would let coroutines *call* each other without blocking -- a
tremendous boon for asynchronous applications. Such applications could then
write co-routines to do non-blocking socket I/O by yielding control to an I/O
scheduler until data has been sent or becomes available. Meanwhile, code that
performs the I/O would simply do something like this::
data = (yield nonblocking_read(my_socket, nbytes))
in order to pause execution until the nonblocking_read() coroutine
produced a value.
in order to pause execution until the ``nonblocking_read()`` coroutine produced
a value.
In other words, with a few relatively minor enhancements to the language and to
the implementation of the generator-iterator type, Python will be able to
support performing asynchronous operations without needing to write the entire
application as a series of callbacks, and without requiring the use of
resource-intensive threads for programs that need hundreds or even thousands of
co-operatively multitasking pseudothreads. Thus, these enhancements will give
standard Python many of the benefits of the Stackless Python fork, without
requiring any significant modification to the CPython core or its APIs. In
addition, these enhancements should be readily implementable by any Python
implementation (such as Jython) that already supports generators.
In other words, with a few relatively minor enhancements to the
language and to the implementation of the generator-iterator type,
Python will be able to support performing asynchronous operations
without needing to write the entire application as a series of
callbacks, and without requiring the use of resource-intensive threads
for programs that need hundreds or even thousands of co-operatively
multitasking pseudothreads. Thus, these enhancements will give
standard Python many of the benefits of the Stackless Python fork,
without requiring any significant modification to the CPython core
or its APIs. In addition, these enhancements should be readily
implementable by any Python implementation (such as Jython) that
already supports generators.
Specification Summary
=====================
By adding a few simple methods to the generator-iterator type, and
with two minor syntax adjustments, Python developers will be able
to use generator functions to implement co-routines and other forms
of co-operative multitasking. These methods and adjustments are:
By adding a few simple methods to the generator-iterator type, and with two
minor syntax adjustments, Python developers will be able to use generator
functions to implement co-routines and other forms of co-operative
multitasking. These methods and adjustments are:
1. Redefine "yield" to be an expression, rather than a statement.
The current yield statement would become a yield expression
whose value is thrown away. A yield expression's value is
None whenever the generator is resumed by a normal next() call.
1. Redefine ``yield`` to be an expression, rather than a statement. The current
yield statement would become a yield expression whose value is thrown away.
A yield expression's value is ``None`` whenever the generator is resumed by
a normal ``next()`` call.
2. Add a new send() method for generator-iterators, which resumes
the generator and "sends" a value that becomes the result of the
current yield-expression. The send() method returns the next
value yielded by the generator, or raises StopIteration if the
generator exits without yielding another value.
2. Add a new ``send()`` method for generator-iterators, which resumes the
generator and *sends* a value that becomes the result of the current
yield-expression. The ``send()`` method returns the next value yielded by
the generator, or raises ``StopIteration`` if the generator exits without
yielding another value.
3. Add a new throw() method for generator-iterators, which raises
an exception at the point where the generator was paused, and
which returns the next value yielded by the generator, raising
StopIteration if the generator exits without yielding another
value. (If the generator does not catch the passed-in exception,
or raises a different exception, then that exception propagates
to the caller.)
3. Add a new ``throw()`` method for generator-iterators, which raises an
exception at the point where the generator was paused, and which returns the
next value yielded by the generator, raising ``StopIteration`` if the
generator exits without yielding another value. (If the generator does not
catch the passed-in exception, or raises a different exception, then that
exception propagates to the caller.)
4. Add a close() method for generator-iterators, which raises
GeneratorExit at the point where the generator was paused. If
the generator then raises StopIteration (by exiting normally, or
due to already being closed) or GeneratorExit (by not catching
the exception), close() returns to its caller. If the generator
yields a value, a RuntimeError is raised. If the generator
raises any other exception, it is propagated to the caller.
close() does nothing if the generator has already exited due to
an exception or normal exit.
4. Add a ``close()`` method for generator-iterators, which raises
``GeneratorExit`` at the point where the generator was paused. If the
generator then raises ``StopIteration`` (by exiting normally, or due to
already being closed) or ``GeneratorExit`` (by not catching the exception),
``close()`` returns to its caller. If the generator yields a value, a
``RuntimeError`` is raised. If the generator raises any other exception, it
is propagated to the caller. ``close()`` does nothing if the generator has
already exited due to an exception or normal exit.
5. Add support to ensure that close() is called when a generator
iterator is garbage-collected.
5. Add support to ensure that ``close()`` is called when a generator iterator
is garbage-collected.
6. Allow "yield" to be used in try/finally blocks, since garbage
collection or an explicit close() call would now allow the
finally clause to execute.
6. Allow ``yield`` to be used in ``try/finally`` blocks, since garbage
collection or an explicit ``close()`` call would now allow the ``finally``
clause to execute.
A prototype patch implementing all of these changes against the
current Python CVS HEAD is available as SourceForge patch #1223381
(http://python.org/sf/1223381).
A prototype patch implementing all of these changes against the current Python
CVS HEAD is available as SourceForge patch #1223381
(http://python.org/sf/1223381).
Specification: Sending Values into Generators
=============================================
New generator method: send(value)
New generator method: ``send(value)``
-------------------------------------
A new method for generator-iterators is proposed, called send(). It
takes exactly one argument, which is the value that should be "sent
in" to the generator. Calling send(None) is exactly equivalent to
calling a generator's next() method. Calling send() with any other
value is the same, except that the value produced by the generator's
current yield expression will be different.
A new method for generator-iterators is proposed, called ``send()``. It
takes exactly one argument, which is the value that should be *sent in* to
the generator. Calling ``send(None)`` is exactly equivalent to calling a
generator's ``next()`` method. Calling ``send()`` with any other value is
the same, except that the value produced by the generator's current
yield expression will be different.
Because generator-iterators begin execution at the top of the
generator's function body, there is no yield expression to receive
a value when the generator has just been created. Therefore,
calling send() with a non-None argument is prohibited when the
generator iterator has just started, and a TypeError is raised if
this occurs (presumably due to a logic error of some kind). Thus,
before you can communicate with a coroutine you must first call
next() or send(None) to advance its execution to the first yield
expression.
Because generator-iterators begin execution at the top of the generator's
function body, there is no yield expression to receive a value when the
generator has just been created. Therefore, calling ``send()`` with a
non-``None`` argument is prohibited when the generator iterator has just
started, and a ``TypeError`` is raised if this occurs (presumably due to a
logic error of some kind). Thus, before you can communicate with a
coroutine you must first call ``next()`` or ``send(None)`` to advance its
execution to the first yield expression.
As with the next() method, the send() method returns the next value
yielded by the generator-iterator, or raises StopIteration if the
generator exits normally, or has already exited. If the generator
raises an uncaught exception, it is propagated to send()'s caller.
As with the ``next()`` method, the ``send()`` method returns the next value
yielded by the generator-iterator, or raises ``StopIteration`` if the
generator exits normally, or has already exited. If the generator raises an
uncaught exception, it is propagated to ``send()``'s caller.
New syntax: Yield Expressions
New syntax: Yield Expressions
-----------------------------
The yield-statement will be allowed to be used on the right-hand
side of an assignment; in that case it is referred to as
yield-expression. The value of this yield-expression is None
unless send() was called with a non-None argument; see below.
The yield-statement will be allowed to be used on the right-hand side of an
assignment; in that case it is referred to as yield-expression. The value
of this yield-expression is ``None`` unless ``send()`` was called with a
non-``None`` argument; see below.
A yield-expression must always be parenthesized except when it
occurs at the top-level expression on the right-hand side of an
assignment. So
A yield-expression must always be parenthesized except when it occurs at the
top-level expression on the right-hand side of an assignment. So
::
x = yield 42
x = yield
@ -168,107 +169,108 @@ Specification: Sending Values into Generators
foo(yield 42)
foo(yield)
are all legal, but
are all legal, but
::
x = 12 + yield 42
x = 12 + yield
foo(yield 42, 12)
foo(yield, 12)
are all illegal. (Some of the edge cases are motivated by the
current legality of "yield 12, 42".)
are all illegal. (Some of the edge cases are motivated by the current
legality of ``yield 12, 42``.)
Note that a yield-statement or yield-expression without an
expression is now legal. This makes sense: when the information
flow in the next() call is reversed, it should be possible to
yield without passing an explicit value ("yield" is of course
equivalent to "yield None").
Note that a yield-statement or yield-expression without an expression is now
legal. This makes sense: when the information flow in the ``next()`` call
is reversed, it should be possible to yield without passing an explicit
value (``yield`` is of course equivalent to ``yield None``).
When send(value) is called, the yield-expression that it resumes
will return the passed-in value. When next() is called, the resumed
yield-expression will return None. If the yield-expression is a
yield-statement, this returned value is ignored, similar to ignoring
the value returned by a function call used as a statement.
When ``send(value)`` is called, the yield-expression that it resumes will
return the passed-in value. When ``next()`` is called, the resumed
yield-expression will return ``None``. If the yield-expression is a
yield-statement, this returned value is ignored, similar to ignoring the
value returned by a function call used as a statement.
In effect, a yield-expression is like an inverted function call; the
argument to yield is in fact returned (yielded) from the currently
executing function, and the "return value" of yield is the argument
passed in via send().
In effect, a yield-expression is like an inverted function call; the
argument to yield is in fact returned (yielded) from the currently executing
function, and the *return value* of yield is the argument passed in via
``send()``.
Note: the syntactic extensions to yield make its use very similar to that in
Ruby. This is intentional. Do note that in Python the block passes a value
to the generator using ``send(EXPR)`` rather than ``return EXPR``, and the
underlying mechanism whereby control is passed between the generator and the
block is completely different. Blocks in Python are not compiled into
thunks; rather, ``yield`` suspends execution of the generator's frame. Some
edge cases work differently; in Python, you cannot save the block for later
use, and you cannot test whether there is a block or not. (XXX - this stuff
about blocks seems out of place now, perhaps Guido can edit to clarify.)
Note: the syntactic extensions to yield make its use very similar
to that in Ruby. This is intentional. Do note that in Python the
block passes a value to the generator using "send(EXPR)" rather
than "return EXPR", and the underlying mechanism whereby control
is passed between the generator and the block is completely
different. Blocks in Python are not compiled into thunks; rather,
yield suspends execution of the generator's frame. Some edge
cases work differently; in Python, you cannot save the block for
later use, and you cannot test whether there is a block or not.
(XXX - this stuff about blocks seems out of place now, perhaps
Guido can edit to clarify.)
Specification: Exceptions and Cleanup
=====================================
Let a generator object be the iterator produced by calling a
generator function. Below, 'g' always refers to a generator
object.
Let a generator object be the iterator produced by calling a generator
function. Below, *g* always refers to a generator object.
New syntax: yield allowed inside try-finally
New syntax: ``yield`` allowed inside ``try-finally``
----------------------------------------------------
The syntax for generator functions is extended to allow a
yield-statement inside a try-finally statement.
The syntax for generator functions is extended to allow a yield-statement
inside a ``try-finally`` statement.
New generator method: throw(type, value=None, traceback=None)
New generator method: ``throw(type, value=None, traceback=None)``
-----------------------------------------------------------------
g.throw(type, value, traceback) causes the specified exception to
be thrown at the point where the generator g is currently
suspended (i.e. at a yield-statement, or at the start of its
function body if next() has not been called yet). If the
generator catches the exception and yields another value, that is
the return value of g.throw(). If it doesn't catch the exception,
the throw() appears to raise the same exception passed it (it
"falls through"). If the generator raises another exception (this
includes the StopIteration produced when it returns) that
exception is raised by the throw() call. In summary, throw()
behaves like next() or send(), except it raises an exception at the
suspension point. If the generator is already in the closed
state, throw() just raises the exception it was passed without
executing any of the generator's code.
``g.throw(type, value, traceback)`` causes the specified exception to be
thrown at the point where the generator *g* is currently suspended (i.e. at
a yield-statement, or at the start of its function body if ``next()`` has
not been called yet). If the generator catches the exception and yields
another value, that is the return value of ``g.throw()``. If it doesn't
catch the exception, the ``throw()`` appears to raise the same exception
passed it (it *falls through*). If the generator raises another exception
(this includes the ``StopIteration`` produced when it returns) that
exception is raised by the ``throw()`` call. In summary, ``throw()``
behaves like ``next()`` or ``send()``, except it raises an exception at the
suspension point. If the generator is already in the closed state,
``throw()`` just raises the exception it was passed without executing any of
the generator's code.
The effect of raising the exception is exactly as if the
statement:
The effect of raising the exception is exactly as if the statement::
raise type, value, traceback
was executed at the suspension point. The type argument must
not be None, and the type and value must be compatible. If the
value is not an instance of the type, a new exception instance
is created using the value, following the same rules that the raise
statement uses to create an exception instance. The traceback, if
supplied, must be a valid Python traceback object, or a TypeError
occurs.
was executed at the suspension point. The type argument must not be
``None``, and the type and value must be compatible. If the value is not an
instance of the type, a new exception instance is created using the value,
following the same rules that the ``raise`` statement uses to create an
exception instance. The traceback, if supplied, must be a valid Python
traceback object, or a ``TypeError`` occurs.
Note: The name of the throw() method was selected for several
reasons. Raise is a keyword and so cannot be used as a method
name. Unlike raise (which immediately raises an exception from the
current execution point), throw() first resumes the generator, and
only then raises the exception. The word throw is suggestive of
putting the exception in another location, and is already associated
with exceptions in other languages.
Note: The name of the ``throw()`` method was selected for several reasons.
``Raise`` is a keyword and so cannot be used as a method name. Unlike
``raise`` (which immediately raises an exception from the current execution
point), ``throw()`` first resumes the generator, and only then raises the
exception. The word *throw* is suggestive of putting the exception in
another location, and is already associated with exceptions in other
languages.
Alternative method names were considered: resolve(), signal(),
genraise(), raiseinto(), and flush(). None of these seem to fit
as well as throw().
Alternative method names were considered: ``resolve()``, ``signal()``,
``genraise()``, ``raiseinto()``, and ``flush()``. None of these seem to fit
as well as ``throw()``.
New standard exception: GeneratorExit
New standard exception: ``GeneratorExit``
-----------------------------------------
A new standard exception is defined, GeneratorExit, inheriting
from Exception. A generator should handle this by re-raising it
(or just not catching it) or by raising StopIteration.
A new standard exception is defined, ``GeneratorExit``, inheriting from
``Exception``. A generator should handle this by re-raising it (or just not
catching it) or by raising ``StopIteration``.
New generator method: close()
New generator method: ``close()``
---------------------------------
g.close() is defined by the following pseudo-code:
``g.close()`` is defined by the following pseudo-code::
def close(self):
try:
@ -279,101 +281,101 @@ Specification: Exceptions and Cleanup
raise RuntimeError("generator ignored GeneratorExit")
# Other exceptions are not caught
New generator method: __del__()
New generator method: __del__()
-------------------------------
g.__del__() is a wrapper for g.close(). This will be called when
the generator object is garbage-collected (in CPython, this is
when its reference count goes to zero). If close() raises an
exception, a traceback for the exception is printed to sys.stderr
and further ignored; it is not propagated back to the place that
triggered the garbage collection. This is consistent with the
handling of exceptions in __del__() methods on class instances.
``g.__del__()`` is a wrapper for ``g.close()``. This will be called when
the generator object is garbage-collected (in CPython, this is when its
reference count goes to zero). If ``close()`` raises an exception, a
traceback for the exception is printed to ``sys.stderr`` and further
ignored; it is not propagated back to the place that triggered the garbage
collection. This is consistent with the handling of exceptions in
``__del__()`` methods on class instances.
If the generator object participates in a cycle, g.__del__() may
not be called. This is the behavior of CPython's current garbage
collector. The reason for the restriction is that the GC code
needs to "break" a cycle at an arbitrary point in order to collect
it, and from then on no Python code should be allowed to see the
objects that formed the cycle, as they may be in an invalid state.
Objects "hanging off" a cycle are not subject to this restriction.
If the generator object participates in a cycle, ``g.__del__()`` may not be
called. This is the behavior of CPython's current garbage collector. The
reason for the restriction is that the GC code needs to *break* a cycle at
an arbitrary point in order to collect it, and from then on no Python code
should be allowed to see the objects that formed the cycle, as they may be
in an invalid state. Objects *hanging off* a cycle are not subject to this
restriction.
Note that it is unlikely to see a generator object participate in
a cycle in practice. However, storing a generator object in a
global variable creates a cycle via the generator frame's
f_globals pointer. Another way to create a cycle would be to
store a reference to the generator object in a data structure that
is passed to the generator as an argument (e.g., if an object has
a method that's a generator, and keeps a reference to a running
iterator created by that method). Neither of these cases
are very likely given the typical patterns of generator use.
Note that it is unlikely to see a generator object participate in a cycle in
practice. However, storing a generator object in a global variable creates
a cycle via the generator frame's ``f_globals`` pointer. Another way to
create a cycle would be to store a reference to the generator object in a
data structure that is passed to the generator as an argument (e.g., if an
object has a method that's a generator, and keeps a reference to a running
iterator created by that method). Neither of these cases are very likely
given the typical patterns of generator use.
Also, in the CPython implementation of this PEP, the frame object used by
the generator should be released whenever its execution is terminated due to
an error or normal exit. This will ensure that generators that cannot be
resumed do not remain part of an uncollectable reference cycle. This allows
other code to potentially use ``close()`` in a ``try/finally`` or ``with``
block (per PEP 343) to ensure that a given generator is properly finalized.
Also, in the CPython implementation of this PEP, the frame object
used by the generator should be released whenever its execution is
terminated due to an error or normal exit. This will ensure that
generators that cannot be resumed do not remain part of an
uncollectable reference cycle. This allows other code to
potentially use close() in a try/finally or "with" block (per PEP
343) to ensure that a given generator is properly finalized.
Optional Extensions
===================
The Extended 'continue' Statement
The Extended ``continue`` Statement
-----------------------------------
An earlier draft of this PEP proposed a new ``continue EXPR`` syntax for use
in for-loops (carried over from PEP 340), that would pass the value of
*EXPR* into the iterator being looped over. This feature has been withdrawn
for the time being, because the scope of this PEP has been narrowed to focus
only on passing values into generator-iterators, and not other kinds of
iterators. It was also felt by some on the Python-Dev list that adding new
syntax for this particular feature would be premature at best.
An earlier draft of this PEP proposed a new "continue EXPR"
syntax for use in for-loops (carried over from PEP 340), that
would pass the value of EXPR into the iterator being looped over.
This feature has been withdrawn for the time being, because the
scope of this PEP has been narrowed to focus only on passing values
into generator-iterators, and not other kinds of iterators. It
was also felt by some on the Python-Dev list that adding new syntax
for this particular feature would be premature at best.
Open Issues
===========
Discussion on python-dev has revealed some open issues. I list
them here, with my preferred resolution and its motivation. The
PEP as currently written reflects this preferred resolution.
Discussion on python-dev has revealed some open issues. I list them here, with
my preferred resolution and its motivation. The PEP as currently written
reflects this preferred resolution.
1. What exception should be raised by close() when the generator
yields another value as a response to the GeneratorExit
exception?
1. What exception should be raised by ``close()`` when the generator yields
another value as a response to the ``GeneratorExit`` exception?
I originally chose TypeError because it represents gross
misbehavior of the generator function, which should be fixed by
changing the code. But the with_template decorator class in
PEP 343 uses RuntimeError for similar offenses. Arguably they
should all use the same exception. I'd rather not introduce a
new exception class just for this purpose, since it's not an
exception that I want people to catch: I want it to turn into a
traceback which is seen by the programmer who then fixes the
code. So now I believe they should both raise RuntimeError.
There are some precedents for that: it's raised by the core
Python code in situations where endless recursion is detected,
and for uninitialized objects (and for a variety of
I originally chose ``TypeError`` because it represents gross misbehavior of
the generator function, which should be fixed by changing the code. But the
``with_template`` decorator class in PEP 343 uses ``RuntimeError`` for
similar offenses. Arguably they should all use the same exception. I'd
rather not introduce a new exception class just for this purpose, since it's
not an exception that I want people to catch: I want it to turn into a
traceback which is seen by the programmer who then fixes the code. So now I
believe they should both raise ``RuntimeError``. There are some precedents
for that: it's raised by the core Python code in situations where endless
recursion is detected, and for uninitialized objects (and for a variety of
miscellaneous conditions).
2. Oren Tirosh has proposed renaming the send() method to feed(),
for compatibility with the "consumer interface" (see
2. Oren Tirosh has proposed renaming the ``send()`` method to ``feed()``, for
compatibility with the *consumer interface* (see
http://effbot.org/zone/consumer.htm for the specification.)
However, looking more closely at the consumer interface, it seems
that the desired semantics for feed() are different than for
send(), because send() can't be meaningfully called on a just-
started generator. Also, the consumer interface as currently
defined doesn't include handling for StopIteration.
However, looking more closely at the consumer interface, it seems that the
desired semantics for ``feed()`` are different than for ``send()``, because
``send()`` can't be meaningfully called on a just-started generator. Also,
the consumer interface as currently defined doesn't include handling for
``StopIteration``.
Therefore, it seems like it would probably be more useful to create a simple
decorator that wraps a generator function to make it conform to the consumer
interface. For example, it could *warm up* the generator with an initial
``next()`` call, trap StopIteration, and perhaps even provide ``reset()`` by
re-invoking the generator function.
Therefore, it seems like it would probably be more useful to
create a simple decorator that wraps a generator function to make
it conform to the consumer interface. For example, it could
"warm up" the generator with an initial next() call, trap
StopIteration, and perhaps even provide reset() by re-invoking
the generator function.
Examples
========
1. A simple "consumer" decorator that makes a generator function
automatically advance to its first yield point when initially
called:
1. A simple *consumer* decorator that makes a generator function automatically
advance to its first yield point when initially called::
def consumer(func):
def wrapper(*args,**kw):
@ -385,12 +387,11 @@ Examples
wrapper.__doc__ = func.__doc__
return wrapper
2. An example of using the "consumer" decorator to create a
"reverse generator" that receives images and creates thumbnail
pages, sending them on to another consumer. Functions like
this can be chained together to form efficient processing
pipelines of "consumers" that each can have complex internal
state:
2. An example of using the *consumer* decorator to create a *reverse generator*
that receives images and creates thumbnail pages, sending them on to another
consumer. Functions like this can be chained together to form efficient
processing pipelines of *consumers* that each can have complex internal
state::
@consumer
def thumbnail_pager(pagesize, thumbsize, destination):
@ -403,8 +404,7 @@ Examples
for column in xrange(columns):
thumb = create_thumbnail((yield), thumbsize)
page.write(
thumb, col*thumbsize.x, row*thumbsize.y
)
thumb, col*thumbsize.x, row*thumbsize.y )
pending = True
except GeneratorExit:
# close() was called, so flush any pending output
@ -420,7 +420,7 @@ Examples
destination.send(page)
@consumer
def jpeg_writer(dirname):
def jpeg_writer(dirname)::
fileno = 1
while True:
filename = os.path.join(dirname,"page%04d.jpg" % fileno)
@ -441,22 +441,21 @@ Examples
pipeline.close()
3. A simple co-routine scheduler or "trampoline" that lets
coroutines "call" other coroutines by yielding the coroutine
they wish to invoke. Any non-generator value yielded by
a coroutine is returned to the coroutine that "called" the
one yielding the value. Similarly, if a coroutine raises an
exception, the exception is propagated to its "caller". In
effect, this example emulates simple tasklets as are used
in Stackless Python, as long as you use a yield expression to
invoke routines that would otherwise "block". This is only
a very simple example, and far more sophisticated schedulers
are possible. (For example, the existing GTasklet framework
for Python (http://www.gnome.org/~gjc/gtasklet/gtasklets.html)
and the peak.events framework (http://peak.telecommunity.com/)
already implement similar scheduling capabilities, but must
currently use awkward workarounds for the inability to pass
values or exceptions into generators.)
3. A simple co-routine scheduler or *trampoline* that lets coroutines *call*
other coroutines by yielding the coroutine they wish to invoke. Any
non-generator value yielded by a coroutine is returned to the coroutine that
*called* the one yielding the value. Similarly, if a coroutine raises an
exception, the exception is propagated to its *caller*. In effect, this
example emulates simple tasklets as are used in Stackless Python, as long as
you use a yield expression to invoke routines that would otherwise *block*.
This is only a very simple example, and far more sophisticated schedulers
are possible. (For example, the existing GTasklet framework for Python
(http://www.gnome.org/~gjc/gtasklet/gtasklets.html) and the peak.events
framework (http://peak.telecommunity.com/) already implement similar
scheduling capabilities, but must currently use awkward workarounds for the
inability to pass values or exceptions into generators.)
::
import collections
@ -476,7 +475,7 @@ Examples
result = None
self.running = True
try:
while self.running and self.queue:
while self.running and self.queue::
func = self.queue.popleft()
result = func()
return result
@ -521,10 +520,10 @@ Examples
self.queue.append(resume)
4. A simple "echo" server, and code to run it using a trampoline
(presumes the existence of "nonblocking_read",
"nonblocking_write", and other I/O coroutines, that e.g. raise
ConnectionLost if the connection is closed):
4. A simple *echo* server, and code to run it using a trampoline (presumes the
existence of ``nonblocking_read``, ``nonblocking_write``, and other I/O
coroutines, that e.g. raise ``ConnectionLost`` if the connection is
closed)::
# coroutine function that echos data back on a connected
# socket
@ -569,27 +568,41 @@ Examples
Reference Implementation
========================
A prototype patch implementing all of the features described in this
PEP is available as SourceForge patch #1223381
(http://python.org/sf/1223381).
A prototype patch implementing all of the features described in this PEP is
available as SourceForge patch #1223381 (http://python.org/sf/1223381).
This patch was committed to CVS 01-02 August 2005.
This patch was committed to CVS 01-02 August 2005.
Acknowledgements
================
Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first formally
proposed the ideas of communicating values or exceptions into generators, and
the ability to *close* generators. Timothy Delaney suggested the title of this
PEP, and Steven Bethard helped edit a previous version. See also the
Acknowledgements section of PEP 340.
Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first
formally proposed the ideas of communicating values or exceptions
into generators, and the ability to "close" generators. Timothy
Delaney suggested the title of this PEP, and Steven Bethard helped
edit a previous version. See also the Acknowledgements section
of PEP 340.
References
==========
TBD.
TBD.
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
This document has been placed in the public domain.

539
pep-0344.txt
View File

@ -5,144 +5,146 @@ Last-Modified: $Date$
Author: Ka-Ping Yee
Status: Superseded
Type: Standards Track
Content-Type: text/plain
Content-Type: text/x-rst
Created: 12-May-2005
Python-Version: 2.5
Post-History:
Numbering Note
==============
This PEP has been renumbered to PEP 3134. The text below is the
last version submitted under the old number.
This PEP has been renumbered to PEP 3134. The text below is the last version
submitted under the old number.
Abstract
========
This PEP proposes three standard attributes on exception instances:
the '__context__' attribute for implicitly chained exceptions, the
'__cause__' attribute for explicitly chained exceptions, and the
'__traceback__' attribute for the traceback. A new "raise ... from"
statement sets the '__cause__' attribute.
This PEP proposes three standard attributes on exception instances: the
``__context__`` attribute for implicitly chained exceptions, the
``__cause__`` attribute for explicitly chained exceptions, and the
``__traceback__`` attribute for the traceback. A new ``raise ... from``
statement sets the ``__cause__`` attribute.
Motivation
==========
During the handling of one exception (exception A), it is possible
that another exception (exception B) may occur. In today's Python
(version 2.4), if this happens, exception B is propagated outward
and exception A is lost. In order to debug the problem, it is
useful to know about both exceptions. The '__context__' attribute
retains this information automatically.
During the handling of one exception (exception A), it is possible that another
exception (exception B) may occur. In today's Python (version 2.4), if this
happens, exception B is propagated outward and exception A is lost. In order
to debug the problem, it is useful to know about both exceptions. The
``__context__`` attribute retains this information automatically.
Sometimes it can be useful for an exception handler to intentionally
re-raise an exception, either to provide extra information or to
translate an exception to another type. The '__cause__' attribute
provides an explicit way to record the direct cause of an exception.
Sometimes it can be useful for an exception handler to intentionally re-raise
an exception, either to provide extra information or to translate an exception
to another type. The ``__cause__`` attribute provides an explicit way to
record the direct cause of an exception.
In today's Python implementation, exceptions are composed of three
parts: the type, the value, and the traceback. The 'sys' module,
exposes the current exception in three parallel variables, exc_type,
exc_value, and exc_traceback, the sys.exc_info() function returns a
tuple of these three parts, and the 'raise' statement has a
three-argument form accepting these three parts. Manipulating
exceptions often requires passing these three things in parallel,
which can be tedious and error-prone. Additionally, the 'except'
statement can only provide access to the value, not the traceback.
Adding the '__traceback__' attribute to exception values makes all
the exception information accessible from a single place.
In today's Python implementation, exceptions are composed of three parts: the
type, the value, and the traceback. The ``sys`` module, exposes the current
exception in three parallel variables, ``exc_type``, ``exc_value``, and
``exc_traceback``, the ``sys.exc_info()`` function returns a tuple of these
three parts, and the ``raise`` statement has a three-argument form accepting
these three parts. Manipulating exceptions often requires passing these three
things in parallel, which can be tedious and error-prone. Additionally, the
``except`` statement can only provide access to the value, not the traceback.
Adding the ``__traceback__`` attribute to exception values makes all the
exception information accessible from a single place.
History
=======
Raymond Hettinger [1] raised the issue of masked exceptions on
Python-Dev in January 2003 and proposed a PyErr_FormatAppend()
function that C modules could use to augment the currently active
exception with more information. Brett Cannon [2] brought up
chained exceptions again in June 2003, prompting a long discussion.
Raymond Hettinger [1]_ raised the issue of masked exceptions on Python-Dev in
January 2003 and proposed a ``PyErr_FormatAppend()`` function that C modules
could use to augment the currently active exception with more information.
Brett Cannon [2]_ brought up chained exceptions again in June 2003, prompting
a long discussion.
Greg Ewing [3] identified the case of an exception occurring in a
'finally' block during unwinding triggered by an original exception,
as distinct from the case of an exception occurring in an 'except'
block that is handling the original exception.
Greg Ewing [3]_ identified the case of an exception occurring in a ``finally``
block during unwinding triggered by an original exception, as distinct from
the case of an exception occurring in an ``except`` block that is handling the
original exception.
Greg Ewing [4] and Guido van Rossum [5], and probably others, have
previously mentioned adding a traceback attribute to Exception
instances. This is noted in PEP 3000.
Greg Ewing [4]_ and Guido van Rossum [5]_, and probably others, have
previously mentioned adding a traceback attribute to ``Exception`` instances.
This is noted in PEP 3000.
This PEP was motivated by yet another recent Python-Dev reposting
of the same ideas [6] [7].
This PEP was motivated by yet another recent Python-Dev reposting of the same
ideas [6]_ [7]_.
Rationale
=========
The Python-Dev discussions revealed interest in exception chaining
for two quite different purposes. To handle the unexpected raising
of a secondary exception, the exception must be retained implicitly.
To support intentional translation of an exception, there must be a
way to chain exceptions explicitly. This PEP addresses both.
The Python-Dev discussions revealed interest in exception chaining for two
quite different purposes. To handle the unexpected raising of a secondary
exception, the exception must be retained implicitly. To support intentional
translation of an exception, there must be a way to chain exceptions
explicitly. This PEP addresses both.
Several attribute names for chained exceptions have been suggested
on Python-Dev [2], including 'cause', 'antecedent', 'reason',
'original', 'chain', 'chainedexc', 'exc_chain', 'excprev',
'previous', and 'precursor'. For an explicitly chained exception,
this PEP suggests '__cause__' because of its specific meaning. For
an implicitly chained exception, this PEP proposes the name
'__context__' because the intended meaning is more specific than
temporal precedence but less specific than causation: an exception
occurs in the context of handling another exception.
Several attribute names for chained exceptions have been suggested on Python-
Dev [2]_, including ``cause``, ``antecedent``, ``reason``, ``original``,
``chain``, ``chainedexc``, ``xc_chain``, ``excprev``, ``previous`` and
``precursor``. For an explicitly chained exception, this PEP suggests
``__cause__`` because of its specific meaning. For an implicitly chained
exception, this PEP proposes the name ``__context__`` because the intended
meaning is more specific than temporal precedence but less specific than
causation: an exception occurs in the context of handling another exception.
This PEP suggests names with leading and trailing double-underscores
for these three attributes because they are set by the Python VM.
Only in very special cases should they be set by normal assignment.
This PEP suggests names with leading and trailing double-underscores for these
three attributes because they are set by the Python VM. Only in very special
cases should they be set by normal assignment.
This PEP handles exceptions that occur during 'except' blocks and
'finally' blocks in the same way. Reading the traceback makes it
clear where the exceptions occurred, so additional mechanisms for
distinguishing the two cases would only add unnecessary complexity.
This PEP handles exceptions that occur during ``except`` blocks and
``finally`` blocks in the same way. Reading the traceback makes it clear
where the exceptions occurred, so additional mechanisms for distinguishing
the two cases would only add unnecessary complexity.
This PEP proposes that the outermost exception object (the one
exposed for matching by 'except' clauses) be the most recently
raised exception for compatibility with current behaviour.
This PEP proposes that the outermost exception object (the one exposed for
matching by ``except`` clauses) be the most recently raised exception for
compatibility with current behaviour.
This PEP proposes that tracebacks display the outermost exception
last, because this would be consistent with the chronological order
of tracebacks (from oldest to most recent frame) and because the
actual thrown exception is easier to find on the last line.
This PEP proposes that tracebacks display the outermost exception last,
because this would be consistent with the chronological order of tracebacks
(from oldest to most recent frame) and because the actual thrown exception is
easier to find on the last line.
To keep things simpler, the C API calls for setting an exception
will not automatically set the exception's '__context__'. Guido
van Rossum has expressed concerns with making such changes [8].
To keep things simpler, the C API calls for setting an exception will not
automatically set the exception's ``__context__``. Guido van Rossum has
expressed concerns with making such changes [8]_.
As for other languages, Java and Ruby both discard the original
exception when another exception occurs in a 'catch'/'rescue' or
'finally'/'ensure' clause. Perl 5 lacks built-in structured
exception handling. For Perl 6, RFC number 88 [9] proposes an exception
mechanism that implicitly retains chained exceptions in an array
named @@. In that RFC, the most recently raised exception is
exposed for matching, as in this PEP; also, arbitrary expressions
(possibly involving @@) can be evaluated for exception matching.
As for other languages, Java and Ruby both discard the original exception when
another exception occurs in a ``catch/rescue`` or ``finally/ensure`` clause.
Perl 5 lacks built-in structured exception handling. For Perl 6, RFC number
88 [9]_ proposes an exception mechanism that implicitly retains chained
exceptions in an array named ``@@``. In that RFC, the most recently raised
exception is exposed for matching, as in this PEP; also, arbitrary expressions
(possibly involving ``@@``) can be evaluated for exception matching.
Exceptions in C# contain a read-only 'InnerException' property that
may point to another exception. Its documentation [10] says that
"When an exception X is thrown as a direct result of a previous
exception Y, the InnerException property of X should contain a
reference to Y." This property is not set by the VM automatically;
rather, all exception constructors take an optional 'innerException'
argument to set it explicitly. The '__cause__' attribute fulfills
the same purpose as InnerException, but this PEP proposes a new form
of 'raise' rather than extending the constructors of all exceptions.
C# also provides a GetBaseException method that jumps directly to
the end of the InnerException chain; this PEP proposes no analog.
Exceptions in C# contain a read-only ``InnerException`` property that may
point to another exception. Its documentation [10]_ says that "When an
exception X is thrown as a direct result of a previous exception Y, the
``InnerException`` property of X should contain a reference to Y." This
property is not set by the VM automatically; rather, all exception
constructors take an optional ``innerException`` argument to set it
explicitly. The ``__cause__`` attribute fulfills the same purpose as
``InnerException``, but this PEP proposes a new form of ``raise`` rather than
extending the constructors of all exceptions. C# also provides a
``GetBaseException`` method that jumps directly to the end of the
``InnerException`` chain; this PEP proposes no analog.
The reason all three of these attributes are presented together in
one proposal is that the '__traceback__' attribute provides
convenient access to the traceback on chained exceptions.
The reason all three of these attributes are presented together in one proposal
is that the ``__traceback__`` attribute provides convenient access to the
traceback on chained exceptions.
Implicit Exception Chaining
===========================
Here is an example to illustrate the '__context__' attribute.
Here is an example to illustrate the ``__context__`` attribute::
def compute(a, b):
try:
@ -155,18 +157,18 @@ Implicit Exception Chaining
print >>file, exc
file.close()
Calling compute(0, 0) causes a ZeroDivisionError. The compute()
function catches this exception and calls log(exc), but the log()
function also raises an exception when it tries to write to a
file that wasn't opened for writing.
Calling ``compute(0, 0)`` causes a ``ZeroDivisionError``. The ``compute()``
function catches this exception and calls ``log(exc)``, but the ``log()``
function also raises an exception when it tries to write to a file that wasn't
opened for writing.
In today's Python, the caller of compute() gets thrown an IOError.
The ZeroDivisionError is lost. With the proposed change, the
instance of IOError has an additional '__context__' attribute that
retains the ZeroDivisionError.
In today's Python, the caller of ``compute()`` gets thrown an ``IOError``. The
``ZeroDivisionError`` is lost. With the proposed change, the instance of
``IOError`` has an additional ``__context__`` attribute that retains the
``ZeroDivisionError``.
The following more elaborate example demonstrates the handling of a
mixture of 'finally' and 'except' clauses:
The following more elaborate example demonstrates the handling of a mixture of
``finally`` and ``except`` clauses::
def main(filename):
file = open(filename) # oops, forgot the 'w'
@ -190,47 +192,48 @@ Implicit Exception Chaining
def display(exc):
print ex # oops, misspelled 'exc'
Calling main() with the name of an existing file will trigger four
exceptions. The ultimate result will be an AttributeError due to
the misspelling of 'clos', whose __context__ points to a NameError
due to the misspelling of 'ex', whose __context__ points to an
IOError due to the file being read-only, whose __context__ points to
a ZeroDivisionError, whose __context__ attribute is None.
Calling ``main()`` with the name of an existing file will trigger four
exceptions. The ultimate result will be an ``AttributeError`` due to the
misspelling of ``clos``, whose ``__context__`` points to a ``NameError`` due
to the misspelling of ``ex``, whose ``__context__`` points to an ``IOError``
due to the file being read-only, whose ``__context__`` points to a
``ZeroDivisionError``, whose ``__context__`` attribute is ``None``.
The proposed semantics are as follows:
The proposed semantics are as follows:
1. Each thread has an exception context initially set to None.
1. Each thread has an exception context initially set to ``None``.
2. Whenever an exception is raised, if the exception instance does
not already have a '__context__' attribute, the interpreter sets
it equal to the thread's exception context.
2. Whenever an exception is raised, if the exception instance does not
already have a ``__context__`` attribute, the interpreter sets it equal to
the thread's exception context.
3. Immediately after an exception is raised, the thread's exception
context is set to the exception.
3. Immediately after an exception is raised, the thread's exception context is
set to the exception.
4. Whenever the interpreter exits an 'except' block by reaching the
end or executing a 'return', 'yield', 'continue', or 'break'
statement, the thread's exception context is set to None.
4. Whenever the interpreter exits an ``except`` block by reaching the end or
executing a ``return``, ``yield``, ``continue``, or ``break`` statement,
the thread's exception context is set to ``None``.
Explicit Exception Chaining
===========================
The '__cause__' attribute on exception objects is always initialized
to None. It is set by a new form of the 'raise' statement:
The ``__cause__`` attribute on exception objects is always initialized to
``None``. It is set by a new form of the ``raise`` statement::
raise EXCEPTION from CAUSE
which is equivalent to:
which is equivalent to::
exc = EXCEPTION
exc.__cause__ = CAUSE
raise exc
In the following example, a database provides implementations for a
few different kinds of storage, with file storage as one kind. The
database designer wants errors to propagate as DatabaseError objects
so that the client doesn't have to be aware of the storage-specific
details, but doesn't want to lose the underlying error information.
In the following example, a database provides implementations for a few
different kinds of storage, with file storage as one kind. The database
designer wants errors to propagate as ``DatabaseError`` objects so that the
client doesn't have to be aware of the storage-specific details, but doesn't
want to lose the underlying error information::
class DatabaseError(StandardError):
pass
@ -242,14 +245,15 @@ Explicit Exception Chaining
except IOError, exc:
raise DatabaseError('failed to open') from exc
If the call to open() raises an exception, the problem will be
reported as a DatabaseError, with a __cause__ attribute that reveals
the IOError as the original cause.
If the call to ``open()`` raises an exception, the problem will be reported as
a ``DatabaseError``, with a ``__cause__`` attribute that reveals the
``IOError`` as the original cause.
Traceback Attribute
===================
The following example illustrates the '__traceback__' attribute.
The following example illustrates the ``__traceback__`` attribute::
def do_logged(file, work):
try:
@ -268,38 +272,40 @@ Traceback Attribute
file.write(... type ... message ... lines ...)
...
In today's Python, the do_logged() function would have to extract
the traceback from sys.exc_traceback or sys.exc_info()[2] and pass
both the value and the traceback to write_exception(). With the
proposed change, write_exception() simply gets one argument and
obtains the exception using the '__traceback__' attribute.
In today's Python, the ``do_logged()`` function would have to extract the
traceback from ``sys.exc_traceback`` or ``sys.exc_info()`` [2]_ and pass both
the value and the traceback to ``write_exception()``. With the proposed
change, ``write_exception()`` simply gets one argument and obtains the
exception using the ``__traceback__`` attribute.
The proposed semantics are as follows:
The proposed semantics are as follows:
1. Whenever an exception is caught, if the exception instance does
not already have a '__traceback__' attribute, the interpreter
sets it to the newly caught traceback.
1. Whenever an exception is caught, if the exception instance does not already
have a ``__traceback__`` attribute, the interpreter sets it to the newly
caught traceback.
Enhanced Reporting
==================
The default exception handler will be modified to report chained
exceptions. The chain of exceptions is traversed by following the
'__cause__' and '__context__' attributes, with '__cause__' taking
priority. In keeping with the chronological order of tracebacks,
the most recently raised exception is displayed last; that is, the
display begins with the description of the innermost exception and
backs up the chain to the outermost exception. The tracebacks are
formatted as usual, with one of the lines:
The default exception handler will be modified to report chained exceptions.
The chain of exceptions is traversed by following the ``__cause__`` and
``__context__`` attributes, with ``__cause__`` taking priority. In keeping
with the chronological order of tracebacks, the most recently raised exception
is displayed last; that is, the display begins with the description of the
innermost exception and backs up the chain to the outermost exception. The
tracebacks are formatted as usual, with one of the lines::
The above exception was the direct cause of the following exception:
or
or
::
During handling of the above exception, another exception occurred:
between tracebacks, depending whether they are linked by __cause__
or __context__ respectively. Here is a sketch of the procedure:
between tracebacks, depending whether they are linked by ``__cause__`` or
``__context__`` respectively. Here is a sketch of the procedure::
def print_chain(exc):
if exc.__cause__:
@ -310,64 +316,68 @@ Enhanced Reporting
print '\nDuring handling of the above exception, ...'
print_exc(exc)
In the 'traceback' module, the format_exception, print_exception,
print_exc, and print_last functions will be updated to accept an
optional 'chain' argument, True by default. When this argument is
True, these functions will format or display the entire chain of
exceptions as just described. When it is False, these functions
will format or display only the outermost exception.
In the ``traceback`` module, the ``format_exception``, ``print_exception``,
``print_exc``, and ``print_last functions`` will be updated to accept an
optional ``chain`` argument, ``True`` by default. When this argument is
``True``, these functions will format or display the entire chain of
exceptions as just described. When it is ``False``, these functions will
format or display only the outermost exception.
The 'cgitb' module should also be updated to display the entire
chain of exceptions.
The ``cgitb`` module should also be updated to display the entire chain of
exceptions.
C API
=====
The PyErr_Set* calls for setting exceptions will not set the
'__context__' attribute on exceptions. PyErr_NormalizeException
will always set the 'traceback' attribute to its 'tb' argument and
the '__context__' and '__cause__' attributes to None.
The ``PyErr_Set*`` calls for setting exceptions will not set the
``__context__`` attribute on exceptions. ``PyErr_NormalizeException`` will
always set the ``traceback`` attribute to its ``tb`` argument and the
``__context__`` and ``__cause__`` attributes to ``None``.
A new API function, PyErr_SetContext(context), will help C
programmers provide chained exception information. This function
will first normalize the current exception so it is an instance,
then set its '__context__' attribute. A similar API function,
PyErr_SetCause(cause), will set the '__cause__' attribute.
A new API function, ``PyErr_SetContext(context)``, will help C programmers
provide chained exception information. This function will first normalize the
current exception so it is an instance, then set its ``__context__``
attribute. A similar API function, ``PyErr_SetCause(cause)``, will set the
``__cause__`` attribute.
Compatibility
=============
Chained exceptions expose the type of the most recent exception, so
they will still match the same 'except' clauses as they do now.
Chained exceptions expose the type of the most recent exception, so they will
still match the same ``except`` clauses as they do now.
The proposed changes should not break any code unless it sets or
uses attributes named '__context__', '__cause__', or '__traceback__'
on exception instances. As of 2005-05-12, the Python standard
library contains no mention of such attributes.
The proposed changes should not break any code unless it sets or uses
attributes named ``__context__``, ``__cause__``, or ``__traceback__`` on
exception instances. As of 2005-05-12, the Python standard library contains
no mention of such attributes.
Open Issue: Extra Information
==============================
Walter Dörwald [11] expressed a desire to attach extra information
to an exception during its upward propagation without changing its
type. This could be a useful feature, but it is not addressed by
this PEP. It could conceivably be addressed by a separate PEP
establishing conventions for other informational attributes on
exceptions.
Walter Dörwald [11]_ expressed a desire to attach extra information to an
exception during its upward propagation without changing its type. This could
be a useful feature, but it is not addressed by this PEP. It could
conceivably be addressed by a separate PEP establishing conventions for other
informational attributes on exceptions.
Open Issue: Suppressing Context
================================
As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.
As written, this PEP makes it impossible to suppress ``__context__``, since
setting ``exc.__context__`` to ``None`` in an ``except`` or ``finally`` clause
will only result in it being set again when ``exc`` is raised.
Open Issue: Limiting Exception Types
=====================================
To improve encapsulation, library implementors may want to wrap all
implementation-level exceptions with an application-level exception.
One could try to wrap exceptions by writing this:
To improve encapsulation, library implementors may want to wrap all
implementation-level exceptions with an application-level exception. One could
try to wrap exceptions by writing this::
try:
... implementation may raise an exception ...
@ -375,16 +385,18 @@ Open Issue: Limiting Exception Types
import sys
raise ApplicationError from sys.exc_value
or this:
or this
::
try:
... implementation may raise an exception ...
except Exception, exc:
raise ApplicationError from exc
but both are somewhat flawed. It would be nice to be able to name
the current exception in a catch-all 'except' clause, but that isn't
addressed here. Such a feature would allow something like this:
but both are somewhat flawed. It would be nice to be able to name the current
exception in a catch-all ``except`` clause, but that isn't addressed here.
Such a feature would allow something like this::
try:
... implementation may raise an exception ...
@ -393,11 +405,12 @@ Open Issue: Limiting Exception Types
Open Issue: yield
==================
The exception context is lost when a 'yield' statement is executed;
resuming the frame after the 'yield' does not restore the context.
Addressing this problem is out of the scope of this PEP; it is not a
new problem, as demonstrated by the following example:
The exception context is lost when a ``yield`` statement is executed; resuming
the frame after the ``yield`` does not restore the context. Addressing this
problem is out of the scope of this PEP; it is not a new problem, as
demonstrated by the following example::
>>> def gen():
... try:
@ -415,131 +428,137 @@ Open Issue: yield
Open Issue: Garbage Collection
===============================
The strongest objection to this proposal has been that it creates
cycles between exceptions and stack frames [12]. Collection of
cyclic garbage (and therefore resource release) can be greatly
delayed.
The strongest objection to this proposal has been that it creates cycles
between exceptions and stack frames [12]_. Collection of cyclic garbage (and
therefore resource release) can be greatly delayed::
>>> try:
>>> 1/0
>>> except Exception, err:
>>> pass
will introduce a cycle from err -> traceback -> stack frame -> err,
keeping all locals in the same scope alive until the next GC happens.
will introduce a cycle from err -> traceback -> stack frame -> err, keeping
all locals in the same scope alive until the next GC happens.
Today, these locals would go out of scope. There is lots of code
which assumes that "local" resources -- particularly open files -- will
be closed quickly. If closure has to wait for the next GC, a program
(which runs fine today) may run out of file handles.
Today, these locals would go out of scope. There is lots of code which
assumes that "local" resources -- particularly open files -- will be closed
quickly. If closure has to wait for the next GC, a program (which runs fine
today) may run out of file handles.
Making the __traceback__ attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving
the Exception for later (as unittest does) more awkward, and it would
not allow as much cleanup of the sys module.
Making the ``__traceback__`` attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving the
``Exception`` for later (as ``unittest`` does) more awkward, and it would not
allow as much cleanup of the ``sys`` module.
A possible alternate solution, suggested by Adam Olsen, would be to
instead turn the reference from the stack frame to the 'err' variable
into a weak reference when the variable goes out of scope [13].
A possible alternate solution, suggested by Adam Olsen, would be to instead
turn the reference from the stack frame to the ``err`` variable into a weak
reference when the variable goes out of scope [13]_.
Possible Future Compatible Changes
==================================
These changes are consistent with the appearance of exceptions as
a single object rather than a triple at the interpreter level.
These changes are consistent with the appearance of exceptions as a single
object rather than a triple at the interpreter level.
- If PEP 340 or PEP 343 is accepted, replace the three (type, value,
traceback) arguments to __exit__ with a single exception argument.
- If PEP 340 or PEP 343 is accepted, replace the three (``type``, ``value``,
``traceback``) arguments to ``__exit__`` with a single exception argument.
- Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and
sys.exc_info() in favour of a single member, sys.exception.
- Deprecate ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
``sys.exc_info()`` in favour of a single member, ``sys.exception``.
- Deprecate sys.last_type, sys.last_value, and sys.last_traceback
in favour of a single member, sys.last_exception.
- Deprecate ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``
in favour of a single member, ``sys.last_exception``.
- Deprecate the three-argument form of the 'raise' statement in
favour of the one-argument form.
- Deprecate the three-argument form of the ``raise`` statement in favour of
the one-argument form.
- Upgrade cgitb.html() to accept a single value as its first
argument as an alternative to a (type, value, traceback) tuple.
- Upgrade ``cgitb.html()`` to accept a single value as its first argument as
an alternative to a ``(type, value, traceback)`` tuple.
Possible Future Incompatible Changes
====================================
These changes might be worth considering for Python 3000.
These changes might be worth considering for Python 3000.
- Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and
sys.exc_info().
- Remove ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
``sys.exc_info()``.
- Remove sys.last_type, sys.last_value, and sys.last_traceback.
- Remove ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``.
- Replace the three-argument sys.excepthook with a one-argument
API, and changing the 'cgitb' module to match.
- Replace the three-argument ``sys.excepthook`` with a one-argument API, and
changing the ``cgitb`` module to match.
- Remove the three-argument form of the 'raise' statement.
- Remove the three-argument form of the ``raise`` statement.
- Upgrade traceback.print_exception to accept an 'exception'
argument instead of the type, value, and traceback arguments.
- Upgrade ``traceback.print_exception`` to accept an ``exception`` argument
instead of the ``type``, ``value``, and ``traceback`` arguments.
Acknowledgements
================
Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip
J. Eby, Raymond Hettinger, Walter Dörwald, and others.
Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby,
Raymond Hettinger, Walter Dörwald, and others.
References
==========
[1] Raymond Hettinger, "Idea for avoiding exception masking"
.. [1] Raymond Hettinger, "Idea for avoiding exception masking"
http://mail.python.org/pipermail/python-dev/2003-January/032492.html
[2] Brett Cannon explains chained exceptions
.. [2] Brett Cannon explains chained exceptions
http://mail.python.org/pipermail/python-dev/2003-June/036063.html
[3] Greg Ewing points out masking caused by exceptions during finally
.. [3] Greg Ewing points out masking caused by exceptions during finally
http://mail.python.org/pipermail/python-dev/2003-June/036290.html
[4] Greg Ewing suggests storing the traceback in the exception object
.. [4] Greg Ewing suggests storing the traceback in the exception object
http://mail.python.org/pipermail/python-dev/2003-June/036092.html
[5] Guido van Rossum mentions exceptions having a traceback attribute
.. [5] Guido van Rossum mentions exceptions having a traceback attribute
http://mail.python.org/pipermail/python-dev/2005-April/053060.html
[6] Ka-Ping Yee, "Tidier Exceptions"
.. [6] Ka-Ping Yee, "Tidier Exceptions"
http://mail.python.org/pipermail/python-dev/2005-May/053671.html
[7] Ka-Ping Yee, "Chained Exceptions"
.. [7] Ka-Ping Yee, "Chained Exceptions"
http://mail.python.org/pipermail/python-dev/2005-May/053672.html
[8] Guido van Rossum discusses automatic chaining in PyErr_Set*
.. [8] Guido van Rossum discusses automatic chaining in ``PyErr_Set*``
http://mail.python.org/pipermail/python-dev/2003-June/036180.html
[9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
.. [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
http://dev.perl.org/perl6/rfc/88.html
[10] MSDN .NET Framework Library, "Exception.InnerException Property"
.. [10] MSDN .NET Framework Library, "Exception.InnerException Property"
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp
[11] Walter Dörwald suggests wrapping exceptions to add details
.. [11] Walter Dörwald suggests wrapping exceptions to add details
http://mail.python.org/pipermail/python-dev/2003-June/036148.html
[12] Guido van Rossum restates the objection to cyclic trash
.. [12] Guido van Rossum restates the objection to cyclic trash
http://mail.python.org/pipermail/python-3000/2007-January/005322.html
[13] Adam Olsen suggests using a weakref from stack frame to exception
.. [13] Adam Olsen suggests using a weakref from stack frame to exception
http://mail.python.org/pipermail/python-3000/2007-January/005363.html
Copyright
=========
This document has been placed in the public domain.
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:

550
pep-3134.txt
View File

@ -5,144 +5,145 @@ Last-Modified: $Date$
Author: Ka-Ping Yee
Status: Final
Type: Standards Track
Content-Type: text/plain
Content-Type: text/x-rst
Created: 12-May-2005
Python-Version: 3.0
Post-History:
Numbering Note
==============
This PEP started its life as PEP 344. Since it is now targeted
for Python 3000, it has been moved into the 3xxx space.
This PEP started its life as PEP 344. Since it is now targeted for Python
3000, it has been moved into the 3xxx space.
Abstract
========
This PEP proposes three standard attributes on exception instances:
the '__context__' attribute for implicitly chained exceptions, the
'__cause__' attribute for explicitly chained exceptions, and the
'__traceback__' attribute for the traceback. A new "raise ... from"
statement sets the '__cause__' attribute.
This PEP proposes three standard attributes on exception instances: the
``__context__`` attribute for implicitly chained exceptions, the ``__cause__``
attribute for explicitly chained exceptions, and the ``__traceback__``
attribute for the traceback. A new ``raise ... from`` statement sets the
``__cause__`` attribute.
Motivation
==========
During the handling of one exception (exception A), it is possible
that another exception (exception B) may occur. In today's Python
(version 2.4), if this happens, exception B is propagated outward
and exception A is lost. In order to debug the problem, it is
useful to know about both exceptions. The '__context__' attribute
retains this information automatically.
During the handling of one exception (exception A), it is possible that another
exception (exception B) may occur. In today's Python (version 2.4), if this
happens, exception B is propagated outward and exception A is lost. In order
to debug the problem, it is useful to know about both exceptions. The
``__context__`` attribute retains this information automatically.
Sometimes it can be useful for an exception handler to intentionally
re-raise an exception, either to provide extra information or to
translate an exception to another type. The '__cause__' attribute
provides an explicit way to record the direct cause of an exception.
Sometimes it can be useful for an exception handler to intentionally re-raise
an exception, either to provide extra information or to translate an exception
to another type. The ``__cause__`` attribute provides an explicit way to
record the direct cause of an exception.
In today's Python implementation, exceptions are composed of three
parts: the type, the value, and the traceback. The 'sys' module,
exposes the current exception in three parallel variables, exc_type,
exc_value, and exc_traceback, the sys.exc_info() function returns a
tuple of these three parts, and the 'raise' statement has a
three-argument form accepting these three parts. Manipulating
exceptions often requires passing these three things in parallel,
which can be tedious and error-prone. Additionally, the 'except'
statement can only provide access to the value, not the traceback.
Adding the '__traceback__' attribute to exception values makes all
the exception information accessible from a single place.
In today's Python implementation, exceptions are composed of three parts: the
type, the value, and the traceback. The ``sys`` module, exposes the current
exception in three parallel variables, ``exc_type``, ``exc_value``, and
``exc_traceback``, the ``sys.exc_info()`` function returns a tuple of these
three parts, and the ``raise`` statement has a three-argument form accepting
these three parts. Manipulating exceptions often requires passing these three
things in parallel, which can be tedious and error-prone. Additionally, the
``except`` statement can only provide access to the value, not the traceback.
Adding the ``__traceback__`` attribute to exception values makes all the
exception information accessible from a single place.
History
=======
Raymond Hettinger [1] raised the issue of masked exceptions on
Python-Dev in January 2003 and proposed a PyErr_FormatAppend()
function that C modules could use to augment the currently active
exception with more information. Brett Cannon [2] brought up
chained exceptions again in June 2003, prompting a long discussion.
Raymond Hettinger [1]_ raised the issue of masked exceptions on Python-Dev in
January 2003 and proposed a ``PyErr_FormatAppend()`` function that C modules
could use to augment the currently active exception with more information.
Brett Cannon [2]_ brought up chained exceptions again in June 2003, prompting
a long discussion.
Greg Ewing [3] identified the case of an exception occurring in a
'finally' block during unwinding triggered by an original exception,
as distinct from the case of an exception occurring in an 'except'
block that is handling the original exception.
Greg Ewing [3]_ identified the case of an exception occurring in a ``finally``
block during unwinding triggered by an original exception, as distinct from
the case of an exception occurring in an ``except`` block that is handling the
original exception.
Greg Ewing [4] and Guido van Rossum [5], and probably others, have
previously mentioned adding a traceback attribute to Exception
instances. This is noted in PEP 3000.
Greg Ewing [4]_ and Guido van Rossum [5]_, and probably others, have
previously mentioned adding a traceback attribute to Exception instances.
This is noted in PEP 3000.
This PEP was motivated by yet another recent Python-Dev reposting
of the same ideas [6] [7].
This PEP was motivated by yet another recent Python-Dev reposting of the same
ideas [6]_ [7]_.
Rationale
=========
The Python-Dev discussions revealed interest in exception chaining
for two quite different purposes. To handle the unexpected raising
of a secondary exception, the exception must be retained implicitly.
To support intentional translation of an exception, there must be a
way to chain exceptions explicitly. This PEP addresses both.
The Python-Dev discussions revealed interest in exception chaining for two
quite different purposes. To handle the unexpected raising of a secondary
exception, the exception must be retained implicitly. To support intentional
translation of an exception, there must be a way to chain exceptions
explicitly. This PEP addresses both.
Several attribute names for chained exceptions have been suggested
on Python-Dev [2], including 'cause', 'antecedent', 'reason',
'original', 'chain', 'chainedexc', 'exc_chain', 'excprev',
'previous', and 'precursor'. For an explicitly chained exception,
this PEP suggests '__cause__' because of its specific meaning. For
an implicitly chained exception, this PEP proposes the name
'__context__' because the intended meaning is more specific than
temporal precedence but less specific than causation: an exception
occurs in the context of handling another exception.
Several attribute names for chained exceptions have been suggested on
Python-Dev [2]_, including ``cause``, ``antecedent``, ``reason``, ``original``,
``chain``, ``chainedexc``, ``exc_chain``, ``excprev``, ``previous``, and
``precursor``. For an explicitly chained exception, this PEP suggests
``__cause__`` because of its specific meaning. For an implicitly chained
exception, this PEP proposes the name ``__context__`` because the intended
meaning is more specific than temporal precedence but less specific than
causation: an exception occurs in the context of handling another exception.
This PEP suggests names with leading and trailing double-underscores
for these three attributes because they are set by the Python VM.
Only in very special cases should they be set by normal assignment.
This PEP suggests names with leading and trailing double-underscores for these
three attributes because they are set by the Python VM. Only in very special
cases should they be set by normal assignment.
This PEP handles exceptions that occur during 'except' blocks and
'finally' blocks in the same way. Reading the traceback makes it
clear where the exceptions occurred, so additional mechanisms for
distinguishing the two cases would only add unnecessary complexity.
This PEP handles exceptions that occur during ``except`` blocks and ``finally``
blocks in the same way. Reading the traceback makes it clear where the
exceptions occurred, so additional mechanisms for distinguishing the two cases
would only add unnecessary complexity.
This PEP proposes that the outermost exception object (the one
exposed for matching by 'except' clauses) be the most recently
raised exception for compatibility with current behaviour.
This PEP proposes that the outermost exception object (the one exposed for
matching by ``except`` clauses) be the most recently raised exception for
compatibility with current behaviour.
This PEP proposes that tracebacks display the outermost exception
last, because this would be consistent with the chronological order
of tracebacks (from oldest to most recent frame) and because the
actual thrown exception is easier to find on the last line.
This PEP proposes that tracebacks display the outermost exception last, because
this would be consistent with the chronological order of tracebacks (from
oldest to most recent frame) and because the actual thrown exception is easier
to find on the last line.
To keep things simpler, the C API calls for setting an exception
will not automatically set the exception's '__context__'. Guido
van Rossum has expressed concerns with making such changes [8].
To keep things simpler, the C API calls for setting an exception will not
automatically set the exception's ``__context__``. Guido van Rossum has
expressed concerns with making such changes [8]_.
As for other languages, Java and Ruby both discard the original
exception when another exception occurs in a 'catch'/'rescue' or
'finally'/'ensure' clause. Perl 5 lacks built-in structured
exception handling. For Perl 6, RFC number 88 [9] proposes an exception
mechanism that implicitly retains chained exceptions in an array
named @@. In that RFC, the most recently raised exception is
exposed for matching, as in this PEP; also, arbitrary expressions
(possibly involving @@) can be evaluated for exception matching.
As for other languages, Java and Ruby both discard the original exception when
another exception occurs in a ``catch``/``rescue`` or ``finally``/``ensure``
clause. Perl 5 lacks built-in structured exception handling. For Perl 6, RFC
number 88 [9]_ proposes an exception mechanism that implicitly retains chained
exceptions in an array named ``@@``. In that RFC, the most recently raised
exception is exposed for matching, as in this PEP; also, arbitrary expressions
(possibly involving ``@@``) can be evaluated for exception matching.
Exceptions in C# contain a read-only 'InnerException' property that
may point to another exception. Its documentation [10] says that
"When an exception X is thrown as a direct result of a previous
exception Y, the InnerException property of X should contain a
reference to Y." This property is not set by the VM automatically;
rather, all exception constructors take an optional 'innerException'
argument to set it explicitly. The '__cause__' attribute fulfills
the same purpose as InnerException, but this PEP proposes a new form
of 'raise' rather than extending the constructors of all exceptions.
C# also provides a GetBaseException method that jumps directly to
the end of the InnerException chain; this PEP proposes no analog.
Exceptions in C# contain a read-only ``InnerException`` property that may point
to another exception. Its documentation [10]_ says that "When an exception X
is thrown as a direct result of a previous exception Y, the ``InnerException``
property of X should contain a reference to Y." This property is not set by
the VM automatically; rather, all exception constructors take an optional
``innerException`` argument to set it explicitly. The ``__cause__`` attribute
fulfills the same purpose as ``InnerException``, but this PEP proposes a new
form of ``raise`` rather than extending the constructors of all exceptions. C#
also provides a ``GetBaseException`` method that jumps directly to the end of
the ``InnerException`` chain; this PEP proposes no analog.
The reason all three of these attributes are presented together in
one proposal is that the '__traceback__' attribute provides
convenient access to the traceback on chained exceptions.
The reason all three of these attributes are presented together in one proposal
is that the ``__traceback__`` attribute provides convenient access to the
traceback on chained exceptions.
Implicit Exception Chaining
===========================
Here is an example to illustrate the '__context__' attribute.
Here is an example to illustrate the ``__context__`` attribute::
def compute(a, b):
try:
@ -155,18 +156,18 @@ Implicit Exception Chaining
print >>file, exc
file.close()
Calling compute(0, 0) causes a ZeroDivisionError. The compute()
function catches this exception and calls log(exc), but the log()
function also raises an exception when it tries to write to a
file that wasn't opened for writing.
Calling ``compute(0, 0)`` causes a ``ZeroDivisionError``. The ``compute()``
function catches this exception and calls ``log(exc)``, but the ``log()``
function also raises an exception when it tries to write to a file that wasn't
opened for writing.
In today's Python, the caller of compute() gets thrown an IOError.
The ZeroDivisionError is lost. With the proposed change, the
instance of IOError has an additional '__context__' attribute that
retains the ZeroDivisionError.
In today's Python, the caller of ``compute()`` gets thrown an ``IOError``. The
``ZeroDivisionError`` is lost. With the proposed change, the instance of
``IOError`` has an additional ``__context__`` attribute that retains the
``ZeroDivisionError``.
The following more elaborate example demonstrates the handling of a
mixture of 'finally' and 'except' clauses:
The following more elaborate example demonstrates the handling of a mixture of
``finally`` and ``except`` clauses::
def main(filename):
file = open(filename) # oops, forgot the 'w'
@ -190,47 +191,50 @@ Implicit Exception Chaining
def display(exc):
print ex # oops, misspelled 'exc'
Calling main() with the name of an existing file will trigger four
exceptions. The ultimate result will be an AttributeError due to
the misspelling of 'clos', whose __context__ points to a NameError
due to the misspelling of 'ex', whose __context__ points to an
IOError due to the file being read-only, whose __context__ points to
a ZeroDivisionError, whose __context__ attribute is None.
Calling ``main()`` with the name of an existing file will trigger four
exceptions. The ultimate result will be an ``AttributeError`` due to the
misspelling of ``clos``, whose ``__context__`` points to a ``NameError`` due
to the misspelling of ``ex``, whose ``__context__`` points to an ``IOError``
due to the file being read-only, whose ``__context__`` points to a
``ZeroDivisionError``, whose ``__context__`` attribute is ``None``.
The proposed semantics are as follows:
The proposed semantics are as follows:
1. Each thread has an exception context initially set to None.
1. Each thread has an exception context initially set to ``None``.
2. Whenever an exception is raised, if the exception instance does
not already have a '__context__' attribute, the interpreter sets
it equal to the thread's exception context.
2. Whenever an exception is raised, if the exception instance does not already
have a ``__context__`` attribute, the interpreter sets it equal to the
thread's exception context.
3. Immediately after an exception is raised, the thread's exception
context is set to the exception.
3. Immediately after an exception is raised, the thread's exception context is
set to the exception.
4. Whenever the interpreter exits an 'except' block by reaching the
end or executing a 'return', 'yield', 'continue', or 'break'
statement, the thread's exception context is set to None.
4. Whenever the interpreter exits an ``except`` block by reaching the end or
executing a ``return``, ``yield``, ``continue``, or ``break`` statement, the
thread's exception context is set to ``None``.
Explicit Exception Chaining
===========================
The '__cause__' attribute on exception objects is always initialized
to None. It is set by a new form of the 'raise' statement:
The ``__cause__`` attribute on exception objects is always initialized to
``None``. It is set by a new form of the ``raise`` statement::
raise EXCEPTION from CAUSE
which is equivalent to:
which is equivalent to::
exc = EXCEPTION
exc.__cause__ = CAUSE
raise exc
In the following example, a database provides implementations for a
few different kinds of storage, with file storage as one kind. The
database designer wants errors to propagate as DatabaseError objects
so that the client doesn't have to be aware of the storage-specific
details, but doesn't want to lose the underlying error information.
In the following example, a database provides implementations for a few
different kinds of storage, with file storage as one kind. The database
designer wants errors to propagate as ``DatabaseError`` objects so that the
client doesn't have to be aware of the storage-specific details, but doesn't
want to lose the underlying error information.
::
class DatabaseError(Exception):
pass
@ -242,14 +246,17 @@ Explicit Exception Chaining
except IOError, exc:
raise DatabaseError('failed to open') from exc
If the call to open() raises an exception, the problem will be
reported as a DatabaseError, with a __cause__ attribute that reveals
the IOError as the original cause.
If the call to ``open()`` raises an exception, the problem will be reported as
a ``DatabaseError``, with a ``__cause__`` attribute that reveals the
``IOError`` as the original cause.
Traceback Attribute
===================
The following example illustrates the '__traceback__' attribute.
The following example illustrates the ``__traceback__`` attribute.
::
def do_logged(file, work):
try:
@ -268,38 +275,40 @@ Traceback Attribute
file.write(... type ... message ... lines ...)
...
In today's Python, the do_logged() function would have to extract
the traceback from sys.exc_traceback or sys.exc_info()[2] and pass
both the value and the traceback to write_exception(). With the
proposed change, write_exception() simply gets one argument and
obtains the exception using the '__traceback__' attribute.
In today's Python, the ``do_logged()`` function would have to extract the
traceback from ``sys.exc_traceback`` or ``sys.exc_info()`` [2]_ and pass both
the value and the traceback to ``write_exception()``. With the proposed
change, ``write_exception()`` simply gets one argument and obtains the
exception using the ``__traceback__`` attribute.
The proposed semantics are as follows:
The proposed semantics are as follows:
1. Whenever an exception is caught, if the exception instance does
not already have a '__traceback__' attribute, the interpreter
sets it to the newly caught traceback.
1. Whenever an exception is caught, if the exception instance does not already
have a ``__traceback__`` attribute, the interpreter sets it to the newly
caught traceback.
Enhanced Reporting
==================
The default exception handler will be modified to report chained
exceptions. The chain of exceptions is traversed by following the
'__cause__' and '__context__' attributes, with '__cause__' taking
priority. In keeping with the chronological order of tracebacks,
the most recently raised exception is displayed last; that is, the
display begins with the description of the innermost exception and
backs up the chain to the outermost exception. The tracebacks are
formatted as usual, with one of the lines:
The default exception handler will be modified to report chained exceptions.
The chain of exceptions is traversed by following the ``__cause__`` and
``__context__`` attributes, with ``__cause__`` taking priority. In keeping
with the chronological order of tracebacks, the most recently raised exception
is displayed last; that is, the display begins with the description of the
innermost exception and backs up the chain to the outermost exception. The
tracebacks are formatted as usual, with one of the lines::
The above exception was the direct cause of the following exception:
or
or
::
During handling of the above exception, another exception occurred:
between tracebacks, depending whether they are linked by __cause__
or __context__ respectively. Here is a sketch of the procedure:
between tracebacks, depending whether they are linked by ``__cause__`` or
``__context__`` respectively. Here is a sketch of the procedure::
def print_chain(exc):
if exc.__cause__:
@ -310,64 +319,68 @@ Enhanced Reporting
print '\nDuring handling of the above exception, ...'
print_exc(exc)
In the 'traceback' module, the format_exception, print_exception,
print_exc, and print_last functions will be updated to accept an
optional 'chain' argument, True by default. When this argument is
True, these functions will format or display the entire chain of
exceptions as just described. When it is False, these functions
will format or display only the outermost exception.
In the ``traceback`` module, the ``format_exception``, ``print_exception``,
``print_exc``, and ``print_last`` functions will be updated to accept an
optional ``chain`` argument, ``True`` by default. When this argument is
``True``, these functions will format or display the entire chain of exceptions
as just described. When it is ``False``, these functions will format or
display only the outermost exception.
The 'cgitb' module should also be updated to display the entire
chain of exceptions.
The ``cgitb`` module should also be updated to display the entire chain of
exceptions.
C API
=====
The PyErr_Set* calls for setting exceptions will not set the
'__context__' attribute on exceptions. PyErr_NormalizeException
will always set the 'traceback' attribute to its 'tb' argument and
the '__context__' and '__cause__' attributes to None.
The ``PyErr_Set*`` calls for setting exceptions will not set the
``__context__`` attribute on exceptions. ``PyErr_NormalizeException`` will
always set the ``traceback`` attribute to its ``tb`` argument and the
``__context__`` and ``__cause__`` attributes to ``None``.
A new API function, PyErr_SetContext(context), will help C
programmers provide chained exception information. This function
will first normalize the current exception so it is an instance,
then set its '__context__' attribute. A similar API function,
PyErr_SetCause(cause), will set the '__cause__' attribute.
A new API function, ``PyErr_SetContext(context)``, will help C programmers
provide chained exception information. This function will first normalize the
current exception so it is an instance, then set its ``__context__`` attribute.
A similar API function, ``PyErr_SetCause(cause)``, will set the ``__cause__``
attribute.
Compatibility
=============
Chained exceptions expose the type of the most recent exception, so
they will still match the same 'except' clauses as they do now.
Chained exceptions expose the type of the most recent exception, so they will
still match the same ``except`` clauses as they do now.
The proposed changes should not break any code unless it sets or
uses attributes named '__context__', '__cause__', or '__traceback__'
on exception instances. As of 2005-05-12, the Python standard
library contains no mention of such attributes.
The proposed changes should not break any code unless it sets or uses
attributes named ``__context__``, ``__cause__``, or ``__traceback__`` on
exception instances. As of 2005-05-12, the Python standard library contains no
mention of such attributes.
Open Issue: Extra Information
==============================
Walter Dörwald [11] expressed a desire to attach extra information
to an exception during its upward propagation without changing its
type. This could be a useful feature, but it is not addressed by
this PEP. It could conceivably be addressed by a separate PEP
establishing conventions for other informational attributes on
exceptions.
Walter Dörwald [11]_ expressed a desire to attach extra information to an
exception during its upward propagation without changing its type. This could
be a useful feature, but it is not addressed by this PEP. It could conceivably
be addressed by a separate PEP establishing conventions for other informational
attributes on exceptions.
Open Issue: Suppressing Context
================================
As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.
As written, this PEP makes it impossible to suppress ``__context__``, since
setting ``exc.__context__`` to ``None`` in an ``except`` or ``finally`` clause
will only result in it being set again when ``exc`` is raised.
Open Issue: Limiting Exception Types
=====================================
To improve encapsulation, library implementors may want to wrap all
implementation-level exceptions with an application-level exception.
One could try to wrap exceptions by writing this:
To improve encapsulation, library implementors may want to wrap all
implementation-level exceptions with an application-level exception. One could
try to wrap exceptions by writing this::
try:
... implementation may raise an exception ...
@ -375,16 +388,16 @@ Open Issue: Limiting Exception Types
import sys
raise ApplicationError from sys.exc_value
or this:
or this::
try:
... implementation may raise an exception ...
except Exception, exc:
raise ApplicationError from exc
but both are somewhat flawed. It would be nice to be able to name
the current exception in a catch-all 'except' clause, but that isn't
addressed here. Such a feature would allow something like this:
but both are somewhat flawed. It would be nice to be able to name the current
exception in a catch-all ``except`` clause, but that isn't addressed here.
Such a feature would allow something like this::
try:
... implementation may raise an exception ...
@ -393,11 +406,12 @@ Open Issue: Limiting Exception Types
Open Issue: yield
==================
The exception context is lost when a 'yield' statement is executed;
resuming the frame after the 'yield' does not restore the context.
Addressing this problem is out of the scope of this PEP; it is not a
new problem, as demonstrated by the following example:
The exception context is lost when a ``yield`` statement is executed; resuming
the frame after the ``yield`` does not restore the context. Addressing this
problem is out of the scope of this PEP; it is not a new problem, as
demonstrated by the following example::
>>> def gen():
... try:
@ -415,140 +429,150 @@ Open Issue: yield
Open Issue: Garbage Collection
===============================
The strongest objection to this proposal has been that it creates
cycles between exceptions and stack frames [12]. Collection of
cyclic garbage (and therefore resource release) can be greatly
delayed.
The strongest objection to this proposal has been that it creates cycles
between exceptions and stack frames [12]_. Collection of cyclic garbage (and
therefore resource release) can be greatly delayed.
::
>>> try:
>>> 1/0
>>> except Exception, err:
>>> pass
will introduce a cycle from err -> traceback -> stack frame -> err,
keeping all locals in the same scope alive until the next GC happens.
will introduce a cycle from err -> traceback -> stack frame -> err, keeping all
locals in the same scope alive until the next GC happens.
Today, these locals would go out of scope. There is lots of code
which assumes that "local" resources -- particularly open files -- will
be closed quickly. If closure has to wait for the next GC, a program
(which runs fine today) may run out of file handles.
Today, these locals would go out of scope. There is lots of code which assumes
that "local" resources -- particularly open files -- will be closed quickly.
If closure has to wait for the next GC, a program (which runs fine today) may
run out of file handles.
Making the __traceback__ attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving
the Exception for later (as unittest does) more awkward, and it would
not allow as much cleanup of the sys module.
Making the ``__traceback__`` attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving the
``Exception`` for later (as ``unittest`` does) more awkward, and it would not
allow as much cleanup of the ``sys`` module.
A possible alternate solution, suggested by Adam Olsen, would be to
instead turn the reference from the stack frame to the 'err' variable
into a weak reference when the variable goes out of scope [13].
A possible alternate solution, suggested by Adam Olsen, would be to instead
turn the reference from the stack frame to the ``err`` variable into a weak
reference when the variable goes out of scope [13]_.
Possible Future Compatible Changes
==================================
These changes are consistent with the appearance of exceptions as
a single object rather than a triple at the interpreter level.
These changes are consistent with the appearance of exceptions as a single
object rather than a triple at the interpreter level.
- If PEP 340 or PEP 343 is accepted, replace the three (type, value,
traceback) arguments to __exit__ with a single exception argument.
- If PEP 340 or PEP 343 is accepted, replace the three (``type``, ``value``,
``traceback``) arguments to ``__exit__`` with a single exception argument.
- Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and
sys.exc_info() in favour of a single member, sys.exception.
- Deprecate ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
``sys.exc_info()`` in favour of a single member, ``sys.exception``.
- Deprecate sys.last_type, sys.last_value, and sys.last_traceback
in favour of a single member, sys.last_exception.
- Deprecate ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``
in favour of a single member, ``sys.last_exception``.
- Deprecate the three-argument form of the 'raise' statement in
favour of the one-argument form.
- Deprecate the three-argument form of the ``raise`` statement in favour of the
one-argument form.
- Upgrade cgitb.html() to accept a single value as its first
argument as an alternative to a (type, value, traceback) tuple.
- Upgrade ``cgitb.html()`` to accept a single value as its first argument as an
alternative to a ``(type, value, traceback)`` tuple.
Possible Future Incompatible Changes
====================================
These changes might be worth considering for Python 3000.
These changes might be worth considering for Python 3000.
- Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and
sys.exc_info().
- Remove ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
``sys.exc_info()``.
- Remove sys.last_type, sys.last_value, and sys.last_traceback.
- Remove ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``.
- Replace the three-argument sys.excepthook with a one-argument
API, and changing the 'cgitb' module to match.
- Replace the three-argument ``sys.excepthook`` with a one-argument API, and
changing the ``cgitb`` module to match.
- Remove the three-argument form of the 'raise' statement.
- Remove the three-argument form of the ``raise`` statement.
- Upgrade traceback.print_exception to accept an 'exception'
argument instead of the type, value, and traceback arguments.
- Upgrade ``traceback.print_exception`` to accept an ``exception`` argument
instead of the ``type``, ``value``, and ``traceback`` arguments.
Implementation
==============
The __traceback__ and __cause__ attributes and the new raise syntax were
implemented in revision 57783 [14].
The ``__traceback__`` and ``__cause__`` attributes and the new raise syntax
were implemented in revision 57783 [14]_.
Acknowledgements
================
Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip
J. Eby, Raymond Hettinger, Walter Dörwald, and others.
Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby,
Raymond Hettinger, Walter Dörwald, and others.
References
==========
[1] Raymond Hettinger, "Idea for avoiding exception masking"
.. [1] Raymond Hettinger, "Idea for avoiding exception masking"
http://mail.python.org/pipermail/python-dev/2003-January/032492.html
[2] Brett Cannon explains chained exceptions
.. [2] Brett Cannon explains chained exceptions
http://mail.python.org/pipermail/python-dev/2003-June/036063.html
[3] Greg Ewing points out masking caused by exceptions during finally
.. [3] Greg Ewing points out masking caused by exceptions during finally
http://mail.python.org/pipermail/python-dev/2003-June/036290.html
[4] Greg Ewing suggests storing the traceback in the exception object
.. [4] Greg Ewing suggests storing the traceback in the exception object
http://mail.python.org/pipermail/python-dev/2003-June/036092.html
[5] Guido van Rossum mentions exceptions having a traceback attribute
.. [5] Guido van Rossum mentions exceptions having a traceback attribute
http://mail.python.org/pipermail/python-dev/2005-April/053060.html
[6] Ka-Ping Yee, "Tidier Exceptions"
.. [6] Ka-Ping Yee, "Tidier Exceptions"
http://mail.python.org/pipermail/python-dev/2005-May/053671.html
[7] Ka-Ping Yee, "Chained Exceptions"
.. [7] Ka-Ping Yee, "Chained Exceptions"
http://mail.python.org/pipermail/python-dev/2005-May/053672.html
[8] Guido van Rossum discusses automatic chaining in PyErr_Set*
.. [8] Guido van Rossum discusses automatic chaining in ``PyErr_Set*``
http://mail.python.org/pipermail/python-dev/2003-June/036180.html
[9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
.. [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
http://dev.perl.org/perl6/rfc/88.html
[10] MSDN .NET Framework Library, "Exception.InnerException Property"
.. [10] MSDN .NET Framework Library, "Exception.InnerException Property"
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp
[11] Walter Dörwald suggests wrapping exceptions to add details
.. [11] Walter Dörwald suggests wrapping exceptions to add details
http://mail.python.org/pipermail/python-dev/2003-June/036148.html
[12] Guido van Rossum restates the objection to cyclic trash
.. [12] Guido van Rossum restates the objection to cyclic trash
http://mail.python.org/pipermail/python-3000/2007-January/005322.html
[13] Adam Olsen suggests using a weakref from stack frame to exception
.. [13] Adam Olsen suggests using a weakref from stack frame to exception
http://mail.python.org/pipermail/python-3000/2007-January/005363.html
[14] Patch to implement the bulk of the PEP
.. [14] Patch to implement the bulk of the PEP
http://svn.python.org/view/python/branches/py3k/Include/?rev=57783&view=rev
Copyright
=========
This document has been placed in the public domain.
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: