iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update with outcome of recent python-dev discussions

This commit is contained in:
Nick Coghlan 2005-10-29 06:08:12 +00:00
parent ad551cb280
commit 781dcd1b39
1 changed files with 209 additions and 107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

316
pep-0343.txt
View File

@ -7,17 +7,17 @@ Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 13-May-2005
Post-History: 2-Jun-2005
Post-History: 2-Jun-2005, 16-Oct-2005, 29-Oct-2005
Abstract
This PEP adds a new statement "with" to the Python language to make
it possible to factor out standard uses of try/finally statements.
The PEP has been approved in principle by the BDFL, but there are
The PEP was approved in principle by the BDFL, but there were
still a couple of implementation details to be worked out (see the
section on Open Issues). It's been reverted to Draft status until
those issues have been settled to Guido's satisfaction.
section on Resolved Issues). It's still at Draft status until
Guido gives a final blessing to the updated PEP.
Author's Note
@ -225,7 +225,7 @@ Specification: The 'with' Statement
The translation of the above statement is:
abc = (EXPR).__with__()
abc = (EXPR).__context__()
exc = (None, None, None)
VAR = abc.__enter__()
try:
@ -241,14 +241,18 @@ Specification: The 'with' Statement
accessible to the user; they will most likely be implemented as
special registers or stack positions.
The call to the __with__() method serves a similar purpose to that
of the __iter__() method of iterator and iterables. An object with
with simple state requirements (such as threading.RLock) may provide
its own __enter__() and __exit__() methods, and simply return
'self' from its __with__ method. On the other hand, an object with
more complex state requirements (such as decimal.Context) may
return a distinct context manager object each time its __with__
method is invoked.
The above translation is fairly literal - if any of the relevant
methods are not found as expected, the interpreter will raise
AttributeError.
The call to the __context__() method serves a similar purpose to
that of the __iter__() method of iterator and iterables. An
object with with simple state requirements (such as
threading.RLock) may provide its own __enter__() and __exit__()
methods, and simply return 'self' from its __context__ method. On
the other hand, an object with more complex state requirements
(such as decimal.Context) may return a distinct context manager
object each time its __context__ method is invoked.
If the "as VAR" part of the syntax is omitted, the "VAR =" part of
the translation is omitted (but abc.__enter__() is still called).
@ -284,12 +288,12 @@ Generator Decorator
that makes it possible to use a generator that yields exactly once
to control a with-statement. Here's a sketch of such a decorator:
class GeneratorContext(object):
class GeneratorContextManager(object):
def __init__(self, gen):
self.gen = gen
def __with__(self):
def __context__(self):
return self
def __enter__(self):
@ -314,14 +318,14 @@ Generator Decorator
else:
raise RuntimeError("generator caught exception")
def context(func):
def contextmanager(func):
def helper(*args, **kwds):
return GeneratorContext(func(*args, **kwds))
return GeneratorContextManager(func(*args, **kwds))
return helper
This decorator could be used as follows:
@context
@contextmanager
def opening(filename):
f = open(filename) # IOError is untouched by GeneratorContext
try:
@ -329,16 +333,18 @@ Generator Decorator
finally:
f.close() # Ditto for errors here (however unlikely)
A robust implementation of this decorator should be made part of
the standard library. Refer to Open Issues regarding its name and
location.
A robust builtin implementation of this decorator will be made
part of the standard library.
Just as generator-iterator functions are very useful for writing
__iter__() methods for iterables, generator-context functions will
be very useful for writing __with__() methods for contexts. It is
proposed that the invocation of the "context" decorator be
considered implicit for generator functions used as __with__()
methods (again, refer to the Open Issues section).
be very useful for writing __context__() methods for contexts.
These methods will still need to be decorated using the
contextmanager decorator. To ensure an obvious error message if the
decorator is left out, generator-iterator objects will NOT be given
a native context - if you want to ensure a generator is closed
promptly, use something similar to the duck-typed "closing" context
manager in the examples.
Optional Extensions
@ -371,6 +377,15 @@ Optional Extensions
second with-statement calls f.__enter__() again. A similar error
can be raised if __enter__ is invoked on a closed file object.
For Python 2.5, the following candidates have been identified for
native context managers:
- file
- decimal.Context
- thread.LockType
- threading.Lock
- threading.RLock
- threading.Condition
Standard Terminology
Discussions about iterators and iterables are aided by the standard
@ -384,7 +399,7 @@ Standard Terminology
This PEP proposes that the protocol used by the with statement be
known as the "context management protocol", and that objects that
implement that protocol be known as "context managers". The term
"context" then encompasses all objects with a __with__() method
"context" then encompasses all objects with a __context__() method
that returns a context manager (this means that all context managers
are contexts, but not all contexts are context managers).
@ -395,50 +410,13 @@ Standard Terminology
In cases where the general term "context" would be ambiguous, it
can be made explicit by expanding it to "manageable context".
Open Issues
Discussion on python-dev revealed some open issues. These are listed
here and will be resolved either by consensus on python-dev or by
BDFL fiat.
1. The name of the decorator used to convert a generator-iterator
function into a generator-context function is still to be
finalised.
The proposal in this PEP is that it be called simply "context"
with the following reasoning:
- A "generator function" is an undecorated function containing
the 'yield' keyword, and the objects produced by
such functions are "generator-iterators". The term
"generator" may refer to either a generator function or a
generator-iterator depending on the situation.
- A "generator context function" is a generator function to
which the "context" decorator is applied and the objects
produced by such functions are "generator-context-managers".
The term "generator context" may refer to either a generator
context function or a generator-context-manager depending on
the situation.
2. Should the decorator to convert a generator function into a
generator context function be a builtin, or located elsewhere in
the standard library? This PEP suggests that it should be a
builtin, as generator context functions are the recommended way
of writing new context managers.
3. Should a generator function used to implement a __with__ method
always be considered to be a generator context function, without
requiring the context decorator? This PEP suggests that it
should, as applying a decorator to a slot just looks strange,
and omitting the decorator would be a source of obscure bugs.
The __new__ slot provides some precedent for special casing of
certain slots when processing slot methods.
Resolved Issues
The following issues were resolved either by BDFL fiat, consensus on
python-dev, or a simple lack of objection to proposals in the
original version of this PEP.
The following issues were resolved either by BDFL approval,
consensus on python-dev, or a simple lack of objection to
proposals in the original version of this PEP.
1. The __exit__() method of the GeneratorContext class
1. The __exit__() method of the GeneratorContextManager class
catches StopIteration and considers it equivalent to re-raising
the exception passed to throw(). Is allowing StopIteration
right here?
@ -458,10 +436,10 @@ Resolved Issues
finally-clause (the one implicit in the with-statement) which
re-raises the original exception anyway.
2. What exception should GeneratorContext raise when the underlying
generator-iterator misbehaves? The following quote is the reason
behind Guido's choice of RuntimeError for both this and for the
generator close() method in PEP 342 (from [8]):
2. What exception should GeneratorContextManager raise when the
underlying generator-iterator misbehaves? The following quote is
the reason behind Guido's choice of RuntimeError for both this
and for the generator close() method in PEP 342 (from [8]):
"I'd rather not introduce a new exception class just for this
purpose, since it's not an exception that I want people to catch:
@ -477,24 +455,27 @@ Resolved Issues
on python-dev [4] settled on the term "context manager" for
objects which provide __enter__ and __exit__ methods, and
"context management protocol" for the protocol itself. With the
addition of the __with__ method to the protocol, a natural
extension is to call all objects which provide a __with__ method
"contexts" (or "manageable contexts" in situations where the
general term "context" would be ambiguous).
addition of the __context__ method to the protocol, a natural
extension is to call all objects which provide a __context__
method "contexts" (or "manageable contexts" in situations where
the general term "context" would be ambiguous).
This is now documented in the "Standard Terminology" section.
4. The originally approved version of this PEP did not include a
__with__ method - the method was only added to the PEP after
__context__ method - the method was only added to the PEP after
Jason Orendorff pointed out the difficulty of writing
appropriate __enter__ and __exit__ methods for decimal.Context
[5]. This approach allows a class to define a native context
manager using generator syntax. It also allows a class to use an
existing independent context manager as its native context
manager by applying the independent context manager to 'self' in
its __with__ method. It even allows a class written in C to use
a generator context manager written in Python.
The __with__ method parallels the __iter__ method which forms
its __context__ method. It even allows a class written in C to
use a generator context manager written in Python.
The __context__ method parallels the __iter__ method which forms
part of the iterator protocol.
An earlier version of this PEP called this the __with__ method.
This was later changed to match the name of the protocol rather
than the keyword for the statement [9].
5. The suggestion was made by Jason Orendorff that the __enter__
and __exit__ methods could be removed from the context
@ -514,18 +495,56 @@ Resolved Issues
works without having to first understand the mechanics of
how generator context managers are implemented.
6. The decorator to make a context manager from a generator will be
a builtin called "contextmanager". The shorter term "context" was
considered too ambiguous and potentially confusing [9].
The different flavours of generators can then be described as:
- A "generator function" is an undecorated function containing
the 'yield' keyword, and the objects produced by
such functions are "generator-iterators". The term
"generator" may refer to either a generator function or a
generator-iterator depending on the situation.
- A "generator context function" is a generator function to
which the "contextmanager" decorator is applied and the
objects produced by such functions are "generator-context-
managers". The term "generator context" may refer to either a
generator context function or a generator-context-manager
depending on the situation.
7. A generator function used to implement a __context__ method will
need to be decorated with the contextmanager decorator in order
to have the correct behaviour. Otherwise, you will get an
AttributeError when using the class in a with statement, as
normal generator-iterators will NOT have __enter__ or __exit__
methods.
Getting deterministic closure of generators will require a
separate context manager such as the closing example below.
As Guido put it, "too much magic is bad for your health" [10].
8. It is fine to raise AttributeError instead of TypeError if the
relevant methods aren't present on a class involved in a with
statement. The fact that the abstract object C API raises
TypeError rather than AttributeError is an accident of history,
rather than a deliberate design decision [11].
Examples
(The generator based examples assume PEP 342 is implemented. Also,
some of the examples are likely to be unnecessary in practice, as
the appropriate objects, such as threading.RLock, will be able to
be used directly in with statements)
The generator based examples rely on PEP 342. Also, some of the
examples are likely to be unnecessary in practice, as the
appropriate objects, such as threading.RLock, will be able to be
used directly in with statements.
The tense used in the names of the example context managers is not
arbitrary. Past tense ("-ed") is used when the name refers to an
action which is done in the __enter__ method and undone in the
__exit__ method. Progressive tense ("-ing") is used when the name
refers to an action which is to be done in the __exit__ method.
1. A template for ensuring that a lock, acquired at the start of a
block, is released when the block is left:
@context
def locking(lock):
@contextmanager
def locked(lock):
lock.acquire()
try:
yield
@ -534,20 +553,20 @@ Examples
Used as follows:
with locking(myLock):
with locked(myLock):
# Code here executes with myLock held. The lock is
# guaranteed to be released when the block is left (even
# if via return or by an uncaught exception).
PEP 319 gives a use case for also having an unlocking()
PEP 319 gives a use case for also having an unlocked()
template; this can be written very similarly (just swap the
acquire() and release() calls).
2. A template for opening a file that ensures the file is closed
when the block is left:
@context
def opening(filename, mode="r"):
@contextmanager
def opened(filename, mode="r"):
f = open(filename, mode)
try:
yield f
@ -556,15 +575,15 @@ Examples
Used as follows:
with opening("/etc/passwd") as f:
with opened("/etc/passwd") as f:
for line in f:
print line.rstrip()
3. A template for committing or rolling back a database
transaction:
@context
def transactional(db):
@contextmanager
def transaction(db):
db.begin()
try:
yield None
@ -575,10 +594,10 @@ Examples
4. Example 1 rewritten without a generator:
class locking:
class locked:
def __init__(self, lock):
self.lock = lock
def __with__(self, lock):
def __context__(self):
return self
def __enter__(self):
self.lock.acquire()
@ -586,13 +605,14 @@ Examples
self.lock.release()
(This example is easily modified to implement the other
examples; it shows that is is easy to avoid the need for a
generator if no special state needs to be preserved.)
relatively stateless examples; it shows that it is easy to avoid
the need for a generator if no special state needs to be
preserved.)
5. Redirect stdout temporarily:
@context
def redirecting_stdout(new_stdout):
@contextmanager
def stdout_redirected(new_stdout):
save_stdout = sys.stdout
sys.stdout = new_stdout
try:
@ -602,18 +622,18 @@ Examples
Used as follows:
with opening(filename, "w") as f:
with redirecting_stdout(f):
with opened(filename, "w") as f:
with stdout_redirected(f):
print "Hello world"
This isn't thread-safe, of course, but neither is doing this
same dance manually. In single-threaded programs (for example,
in scripts) it is a popular way of doing things.
6. A variant on opening() that also returns an error condition:
6. A variant on opened() that also returns an error condition:
@context
def opening_w_error(filename, mode="r"):
@contextmanager
def opened_w_error(filename, mode="r"):
try:
f = open(filename, mode)
except IOError, err:
@ -626,7 +646,7 @@ Examples
Used as follows:
with opening_w_error("/etc/passwd", "a") as (f, err):
with opened_w_error("/etc/passwd", "a") as (f, err):
if err:
print "IOError:", err
else:
@ -637,7 +657,7 @@ Examples
import signal
with signal.blocking():
with signal.blocked():
# code executed without worrying about signals
An optional argument might be a list of signals to be blocked;
@ -679,7 +699,8 @@ Examples
9. Here's a proposed native context manager for decimal.Context:
# This would be a new decimal.Context method
def __with__(self):
@contextmanager
def __context__(self):
# We set the thread context to a copy of this context
# to ensure that changes within the block are kept
# local to the block. This also gives us thread safety
@ -710,7 +731,7 @@ Examples
10. A generic "object-closing" template:
@context
@contextmanager
def closing(obj):
try:
yield obj
@ -737,6 +758,78 @@ Examples
for datum in data:
process(datum)
11. Native contexts for objects with acquire/release methods:
# This would be a new method of e.g., threading.RLock
def __context__(self):
return locked(self)
def released(self):
return unlocked(self)
Sample usage:
with my_lock:
# Operations with the lock held
with my_lock.released():
# Operations without the lock
# e.g. blocking I/O
# Lock is held again here
12. A "nested" context manager that automatically nests the
supplied contexts from left-to-right to avoid excessive
indentation:
class nested(object):
def __init__(*contexts):
self.contexts = contexts
self.entered = None
def __context__(self):
return self
def __enter__(self):
if self.entered is not None:
raise RuntimeError("Context is not reentrant")
self.entered = deque()
vars = []
try:
for context in self.contexts:
mgr = context.__context__()
vars.append(mgr.__enter__())
self.entered.appendleft(mgr)
except:
self.__exit__(*sys.exc_info())
raise
return vars
def __exit__(self, *exc_info):
# Behave like nested with statements
# first in, last out
# New exceptions override old ones
ex = exc_info
for mgr in self.entered:
try:
mgr.__exit__(*ex)
except:
ex = sys.exc_info()
self.entered = None
if ex is not exc_info:
raise ex[0], ex[1], ex[2]
Sample usage:
with nested(a, b, c) as (x, y, z):
# Perform operation
Is equivalent to:
with a as x:
with b as y:
with c as z:
# Perform operation
References
[1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx
@ -760,6 +853,15 @@ References
[8]
http://mail.python.org/pipermail/python-dev/2005-June/054064.html
[9]
http://mail.python.org/pipermail/python-dev/2005-October/057520.html
[10]
http://mail.python.org/pipermail/python-dev/2005-October/057535.html
[11]
http://mail.python.org/pipermail/python-dev/2005-October/057625.html
Copyright
This document has been placed in the public domain.