Remove the __context__ method from PEP 343, and update the terminology section (again). Record a couple of remaining open issues, and try to clarify the resolved issues sections by only giving a very brief overview of all the rejected options that aren't relevant any more.
This commit is contained in:
parent
1fe4adbb71
commit
15948f5fe0
484
pep-0343.txt
484
pep-0343.txt
|
@ -7,18 +7,16 @@ Status: Accepted
|
||||||
Type: Standards Track
|
Type: Standards Track
|
||||||
Content-Type: text/plain
|
Content-Type: text/plain
|
||||||
Created: 13-May-2005
|
Created: 13-May-2005
|
||||||
Post-History: 2-Jun-2005, 16-Oct-2005, 29-Oct-2005, 23-Apr-2006
|
Post-History: 2-Jun-2005, 16-Oct-2005, 29-Oct-2005, 23-Apr-2006, 1-May-2006
|
||||||
|
|
||||||
Abstract
|
Abstract
|
||||||
|
|
||||||
This PEP adds a new statement "with" to the Python language to make
|
This PEP adds a new statement "with" to the Python language to make
|
||||||
it possible to factor out standard uses of try/finally statements.
|
it possible to factor out standard uses of try/finally statements.
|
||||||
|
|
||||||
The PEP was approved in principle by the BDFL, but there were
|
In this PEP, context managers provide __enter__() and __exit__()
|
||||||
still a couple of implementation details to be worked out (see the
|
methods that are invoked on entry to and exit from the managed
|
||||||
section on Resolved Issues). It's still at Draft status until
|
context that forms the body of the with statement.
|
||||||
Guido gives a final blessing to the updated PEP.
|
|
||||||
|
|
||||||
|
|
||||||
Author's Note
|
Author's Note
|
||||||
|
|
||||||
|
@ -29,13 +27,17 @@ Author's Note
|
||||||
|
|
||||||
Python's alpha release cycle revealed terminology problems in this
|
Python's alpha release cycle revealed terminology problems in this
|
||||||
PEP and in the associated documentation and implementation [14].
|
PEP and in the associated documentation and implementation [14].
|
||||||
So while the PEP is already accepted, this refers to the
|
So while the PEP is already accepted in principle, it won't really
|
||||||
implementation rather than the exact terminology.
|
be considered stable until the status becomes Final.
|
||||||
|
|
||||||
The current version of the PEP reflects the implementation and
|
The current version of the PEP reflects the discussions that
|
||||||
documentation as at Python 2.5a2. The PEP will be updated to
|
occurred on python-dev shortly after the release of Python 2.5a2.
|
||||||
reflect any changes made to the terminology prior to the final
|
The PEP will continue to be updated to reflect any changes made to
|
||||||
Python 2.5 release.
|
the details of the feature prior to the final Python 2.5 release.
|
||||||
|
|
||||||
|
Yes, the verb tense is messed up in a few places. We've been
|
||||||
|
working on this PEP for nearly a year now, so things that were
|
||||||
|
originally in the future are now in the past :)
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
|
|
||||||
|
@ -48,11 +50,8 @@ Introduction
|
||||||
[2] and universally approved of. I'm also changing the keyword to
|
[2] and universally approved of. I'm also changing the keyword to
|
||||||
'with'.
|
'with'.
|
||||||
|
|
||||||
On-line discussion of this PEP should take place in the Python
|
Following acceptance of this PEP, the following PEPs have been
|
||||||
Wiki [3].
|
rejected due to overlap:
|
||||||
|
|
||||||
If this PEP is approved, the following PEPs will be rejected due
|
|
||||||
to overlap:
|
|
||||||
|
|
||||||
- PEP 310, Reliable Acquisition/Release Pairs. This is the
|
- PEP 310, Reliable Acquisition/Release Pairs. This is the
|
||||||
original with-statement proposal.
|
original with-statement proposal.
|
||||||
|
@ -66,7 +65,11 @@ Introduction
|
||||||
important; in fact it may be better to always be explicit about
|
important; in fact it may be better to always be explicit about
|
||||||
the mutex being used.
|
the mutex being used.
|
||||||
|
|
||||||
(PEP 340 and PEP 346 have already been withdrawn.)
|
PEP 340 and PEP 346 also overlapped with this PEP, but were
|
||||||
|
voluntarily withdrawn when this PEP was submitted.
|
||||||
|
|
||||||
|
Some discussion of earlier incarnations of this PEP took place on
|
||||||
|
the Python Wiki [3].
|
||||||
|
|
||||||
Motivation and Summary
|
Motivation and Summary
|
||||||
|
|
||||||
|
@ -92,7 +95,7 @@ Motivation and Summary
|
||||||
control flow, in the end, the control flow resumes as if the
|
control flow, in the end, the control flow resumes as if the
|
||||||
finally-suite wasn't there at all.
|
finally-suite wasn't there at all.
|
||||||
|
|
||||||
Remember, PEP 310 proposes rougly this syntax (the "VAR =" part is
|
Remember, PEP 310 proposes roughly this syntax (the "VAR =" part is
|
||||||
optional):
|
optional):
|
||||||
|
|
||||||
with VAR = EXPR:
|
with VAR = EXPR:
|
||||||
|
@ -213,6 +216,9 @@ Motivation and Summary
|
||||||
not make the same guarantee. This applies to Jython, IronPython,
|
not make the same guarantee. This applies to Jython, IronPython,
|
||||||
and probably to Python running on Parrot.
|
and probably to Python running on Parrot.
|
||||||
|
|
||||||
|
(The details of the changes made to generators can now be found in
|
||||||
|
PEP 342 rather than in the current PEP)
|
||||||
|
|
||||||
Use Cases
|
Use Cases
|
||||||
|
|
||||||
See the Examples section near the end.
|
See the Examples section near the end.
|
||||||
|
@ -236,7 +242,7 @@ Specification: The 'with' Statement
|
||||||
|
|
||||||
The translation of the above statement is:
|
The translation of the above statement is:
|
||||||
|
|
||||||
mgr = (EXPR).__context__()
|
mgr = (EXPR)
|
||||||
exit = mgr.__exit__ # Not calling it yet
|
exit = mgr.__exit__ # Not calling it yet
|
||||||
value = mgr.__enter__()
|
value = mgr.__enter__()
|
||||||
exc = True
|
exc = True
|
||||||
|
@ -260,9 +266,9 @@ Specification: The 'with' Statement
|
||||||
implemented as special registers or stack positions.
|
implemented as special registers or stack positions.
|
||||||
|
|
||||||
The details of the above translation are intended to prescribe the
|
The details of the above translation are intended to prescribe the
|
||||||
exact semantics. If any of the relevant methods are not found as
|
exact semantics. If either of the relevant methods are not found
|
||||||
expected, the interpreter will raise AttributeError, in the order
|
as expected, the interpreter will raise AttributeError, in the
|
||||||
that they are tried (__context__, __exit__, __enter__).
|
order that they are tried (__exit__, __enter__).
|
||||||
Similarly, if any of the calls raises an exception, the effect is
|
Similarly, if any of the calls raises an exception, the effect is
|
||||||
exactly as it would be in the above code. Finally, if BLOCK
|
exactly as it would be in the above code. Finally, if BLOCK
|
||||||
contains a break, continue or return statement, the __exit__()
|
contains a break, continue or return statement, the __exit__()
|
||||||
|
@ -270,15 +276,6 @@ Specification: The 'with' Statement
|
||||||
completed normally. (I.e. these "pseudo-exceptions" are not seen
|
completed normally. (I.e. these "pseudo-exceptions" are not seen
|
||||||
as exceptions by __exit__().)
|
as exceptions by __exit__().)
|
||||||
|
|
||||||
The call to the __context__() method serves a similar purpose to
|
|
||||||
that of the __iter__() method of iterator and iterables. A context
|
|
||||||
specifier with simple state requirements (such as
|
|
||||||
threading.RLock) may provide its own __enter__() and __exit__()
|
|
||||||
methods, and simply return 'self' from its __context__ method. On
|
|
||||||
the other hand, a context specifier with more complex state
|
|
||||||
requirements (such as decimal.Context) may return a distinct
|
|
||||||
context manager each time its __context__ method is invoked.
|
|
||||||
|
|
||||||
If the "as VAR" part of the syntax is omitted, the "VAR =" part of
|
If the "as VAR" part of the syntax is omitted, the "VAR =" part of
|
||||||
the translation is omitted (but mgr.__enter__() is still called).
|
the translation is omitted (but mgr.__enter__() is still called).
|
||||||
|
|
||||||
|
@ -324,18 +321,19 @@ Specification: The 'with' Statement
|
||||||
of a database transaction roll-back decision.
|
of a database transaction roll-back decision.
|
||||||
|
|
||||||
To facilitate chaining of contexts in Python code that directly
|
To facilitate chaining of contexts in Python code that directly
|
||||||
manipulates context specifiers and managers, __exit__() methods
|
manipulates context managers, __exit__() methods should *not*
|
||||||
should *not* re-raise the error that is passed in to them, because
|
re-raise the error that is passed in to them. It is always the
|
||||||
it is always the responsibility of the *caller* to do any reraising
|
responsibility of the *caller* of the __exit__() method to do any
|
||||||
in that case.
|
reraising in that case.
|
||||||
|
|
||||||
That way, if the caller needs to tell whether the __exit__()
|
That way, if the caller needs to tell whether the __exit__()
|
||||||
invocation *failed* (as opposed to successfully cleaning up before
|
invocation *failed* (as opposed to successfully cleaning up before
|
||||||
propagating the original error), it can do so.
|
propagating the original error), it can do so.
|
||||||
|
|
||||||
If __exit__() returns without an error, this can then be
|
If __exit__() returns without an error, this can then be
|
||||||
interpreted as success of the __exit__() method itself (whether the
|
interpreted as success of the __exit__() method itself (regardless
|
||||||
original error is to be propagated or suppressed).
|
of whether or not the original error is to be propagated or
|
||||||
|
suppressed).
|
||||||
|
|
||||||
However, if __exit__() propagates an exception to its caller, this
|
However, if __exit__() propagates an exception to its caller, this
|
||||||
means that __exit__() *itself* has failed. Thus, __exit__()
|
means that __exit__() *itself* has failed. Thus, __exit__()
|
||||||
|
@ -343,20 +341,6 @@ Specification: The 'with' Statement
|
||||||
failed. (And allowing the original error to proceed isn't a
|
failed. (And allowing the original error to proceed isn't a
|
||||||
failure.)
|
failure.)
|
||||||
|
|
||||||
Objects returned by __context__() methods should also provide a
|
|
||||||
__context__() method that returns self. This allows a program to
|
|
||||||
retrieve the context manager directly without breaking anything.
|
|
||||||
For example, the following should work just as well as the normal
|
|
||||||
case where the extra variable isn't used:
|
|
||||||
|
|
||||||
mgr = (EXPR).__context__()
|
|
||||||
with mgr as VAR:
|
|
||||||
BLOCK
|
|
||||||
|
|
||||||
The with statement implementation and examples like the nested()
|
|
||||||
function require this behaviour in order to be able to deal
|
|
||||||
transparently with both context specifiers and context managers.
|
|
||||||
|
|
||||||
Transition Plan
|
Transition Plan
|
||||||
|
|
||||||
In Python 2.5, the new syntax will only be recognized if a future
|
In Python 2.5, the new syntax will only be recognized if a future
|
||||||
|
@ -382,9 +366,6 @@ Generator Decorator
|
||||||
def __init__(self, gen):
|
def __init__(self, gen):
|
||||||
self.gen = gen
|
self.gen = gen
|
||||||
|
|
||||||
def __context__(self):
|
|
||||||
return self
|
|
||||||
|
|
||||||
def __enter__(self):
|
def __enter__(self):
|
||||||
try:
|
try:
|
||||||
return self.gen.next()
|
return self.gen.next()
|
||||||
|
@ -435,17 +416,7 @@ Generator Decorator
|
||||||
A robust implementation of this decorator will be made
|
A robust implementation of this decorator will be made
|
||||||
part of the standard library.
|
part of the standard library.
|
||||||
|
|
||||||
Just as generator-iterator functions are very useful for writing
|
Context Managers in the Standard Library
|
||||||
__iter__() methods for iterables, generator context functions will
|
|
||||||
be very useful for writing __context__() methods for context
|
|
||||||
specifiers. These methods will still need to be decorated using the
|
|
||||||
contextmanager decorator. To ensure an obvious error message if the
|
|
||||||
decorator is left out, generator-iterator objects will NOT be given
|
|
||||||
a native context - if you want to ensure a generator is closed
|
|
||||||
promptly, use something similar to the duck-typed "closing" context
|
|
||||||
manager in the examples.
|
|
||||||
|
|
||||||
Optional Extensions
|
|
||||||
|
|
||||||
It would be possible to endow certain objects, like files,
|
It would be possible to endow certain objects, like files,
|
||||||
sockets, and locks, with __enter__() and __exit__() methods so
|
sockets, and locks, with __enter__() and __exit__() methods so
|
||||||
|
@ -476,133 +447,110 @@ Optional Extensions
|
||||||
second with-statement calls f.__enter__() again. A similar error
|
second with-statement calls f.__enter__() again. A similar error
|
||||||
can be raised if __enter__ is invoked on a closed file object.
|
can be raised if __enter__ is invoked on a closed file object.
|
||||||
|
|
||||||
For Python 2.5, the following candidates have been identified for
|
For Python 2.5, the following types have been identified as
|
||||||
native context managers:
|
context managers:
|
||||||
- file
|
- file
|
||||||
- decimal.Context
|
|
||||||
- thread.LockType
|
- thread.LockType
|
||||||
- threading.Lock
|
- threading.Lock
|
||||||
- threading.RLock
|
- threading.RLock
|
||||||
- threading.Condition
|
- threading.Condition
|
||||||
- threading.Semaphore and threading.BoundedSemaphore
|
- threading.Semaphore
|
||||||
|
- threading.BoundedSemaphore
|
||||||
|
|
||||||
|
A context manager will also be added to the decimal module to
|
||||||
|
support using a local decimal arithmetic context within the body
|
||||||
|
of a with statement, automatically restoring the original context
|
||||||
|
when the with statement is exited.
|
||||||
|
|
||||||
Standard Terminology
|
Standard Terminology
|
||||||
|
|
||||||
Discussions about iterators and iterables are aided by the standard
|
|
||||||
terminology used to discuss them. The protocol used by the for
|
|
||||||
statement is called the iterator protocol and an iterator is any
|
|
||||||
object that properly implements that protocol. The term "iterable"
|
|
||||||
then encompasses all objects with an __iter__() method that
|
|
||||||
returns an iterator.
|
|
||||||
|
|
||||||
This PEP proposes that the protocol consisting of the __enter__()
|
This PEP proposes that the protocol consisting of the __enter__()
|
||||||
and __exit__() methods, and a __context__() method that returns
|
and __exit__() methods be known as the "context management protocol",
|
||||||
self be known as the "context management protocol", and that
|
and that objects that implement that protocol be known as "context
|
||||||
objects that implement that protocol be known as "context
|
managers". [4]
|
||||||
managers".
|
|
||||||
|
|
||||||
The term "context specifier" then encompasses all objects with a
|
The code in the body of the with statement is a "managed context".
|
||||||
__context__() method that returns a context manager. The protocol
|
This term refers primarily to the code location, rather than to the
|
||||||
these objects implement is called the "context specification
|
runtime environment established by the context manager.
|
||||||
protocol". This means that all context managers are context
|
|
||||||
specifiers, but not all context specifiers are context managers,
|
|
||||||
just as all iterators are iterables, but not all iterables are
|
|
||||||
iterators.
|
|
||||||
|
|
||||||
These terms are based on the concept that the context specifier
|
The expression immediately following the with keyword in the
|
||||||
defines a context of execution for the code that forms the body of
|
statement is a "context expression" as that expression provides the
|
||||||
the with statement. The role of the context manager is to
|
main clue as to the runtime environment the context manager
|
||||||
translate the context specifier's stored state into an active
|
establishes for the duration of the managed context.
|
||||||
manipulation of the runtime environment to setup and tear down the
|
|
||||||
desired runtime context for the duration of the with statement.
|
The value assigned to the target list after the as keyword is the
|
||||||
For example, a synchronisation lock's context manager acquires the
|
"context entry value", as that value is returned as the result of
|
||||||
lock when entering the with statement, and releases the lock when
|
entering the context.
|
||||||
leaving it. The runtime context established within the body of the
|
|
||||||
with statement is that the synchronisation lock is currently held.
|
These terms are based on the idea that the context expression
|
||||||
|
provides a context manager to appropriately handle entry into the
|
||||||
|
managed context. The context manager may also provide a meaningful
|
||||||
|
context entry value and perform clean up operations on exit from
|
||||||
|
the managed context.
|
||||||
|
|
||||||
The general term "context" is unfortunately ambiguous. If necessary,
|
The general term "context" is unfortunately ambiguous. If necessary,
|
||||||
it can be made more explicit by using the terms "context specifier"
|
it can be made more explicit by using the terms "context manager"
|
||||||
for objects providing a __context__() method and "runtime context"
|
for the concrete object created by the context expression,
|
||||||
for the runtime environment modifications made by the context
|
"managed context" for the code in the body of the with statement,
|
||||||
manager. When solely discussing use of the with statement, the
|
and "runtime context" or (preferebly) "runtime environment" for the
|
||||||
distinction between the two shouldn't matter as the context
|
actual state modifications made by the context manager. When solely
|
||||||
specifier fully defines the changes made to the runtime context.
|
discussing use of the with statement, the distinction between these
|
||||||
The distinction is more important when discussing the process of
|
shouldn't matter too much as the context manager fully defines the
|
||||||
implementing context specifiers and context managers.
|
changes made to the runtime environment, and those changes apply for
|
||||||
|
the duration of the managed context. The distinction is more
|
||||||
|
important when discussing the process of implementing context
|
||||||
|
managers and the mechanics of the with statement itself.
|
||||||
|
|
||||||
|
Caching Context Managers
|
||||||
|
|
||||||
|
Many context managers (such as files and generator-based contexts)
|
||||||
|
will be single-use objects. Once the __exit__() method has been
|
||||||
|
called, the context manager will no longer be in a usable state
|
||||||
|
(e.g. the file has been closed, or the underlying generator has
|
||||||
|
finished execution).
|
||||||
|
|
||||||
|
Requiring a fresh manager object for each with statement is the
|
||||||
|
easiest way to avoid problems with multi-threaded code and nested
|
||||||
|
with statements trying to use the same context manager. It isn't
|
||||||
|
coincidental that all of the standard library context managers
|
||||||
|
that support reuse come from the threading module - they're all
|
||||||
|
already designed to deal with the problems created by threaded
|
||||||
|
and nested usage.
|
||||||
|
|
||||||
|
This means that in order to save a context manager with particular
|
||||||
|
initialisation arguments to be used in multiple with statements, it
|
||||||
|
will typically be necessary to store it in a zero-argument callable
|
||||||
|
that is then called in the context expression of each statement
|
||||||
|
rather than caching the context manager directly.
|
||||||
|
|
||||||
|
When this restriction does not apply, the documentation of the
|
||||||
|
affected context manager should make that clear.
|
||||||
|
|
||||||
|
|
||||||
Open Issues
|
Open Issues
|
||||||
|
|
||||||
1. After this PEP was originally approved, a subsequent discussion
|
1. Greg Ewing raised the question of whether or not the term
|
||||||
on python-dev [4] settled on the term "context manager" for
|
"context manager" was too generic and suggested "context guard"
|
||||||
objects which provide __enter__ and __exit__ methods, and
|
as an alternative name.
|
||||||
"context management protocol" for the protocol itself. With the
|
|
||||||
addition of the __context__ method to the protocol, the natural
|
|
||||||
adjustment is to call all objects which provide a __context__
|
|
||||||
method "context managers", and the objects with __enter__ and
|
|
||||||
__exit__ methods "contexts" (or "manageable contexts" in
|
|
||||||
situations where the general term "context" would be ambiguous).
|
|
||||||
|
|
||||||
As noted above, the Python 2.5 release cycle revealed problems
|
2. In Python 2.5a2, the decorator in contextlib to create a
|
||||||
with the previously agreed terminology. The updated standard
|
context manager from a generator function is called
|
||||||
terminology section has not yet met with consensus on
|
@contextfactory. This made sense when the __context__()
|
||||||
python-dev. It will be refined throughout the Python 2.5 release
|
method existed and the result of the factory function was
|
||||||
cycle based on user feedback on the usability of the
|
a managed context object.
|
||||||
documentation.
|
With the elimination of the __context__() method, the
|
||||||
The first change made as a result of the current discussion is
|
result of the factory function is once again a context
|
||||||
replacement of the term "context object" with
|
manager, suggesting the decorator should be renamed to
|
||||||
"context specifier".
|
either @contextmanager or @managerfactory.
|
||||||
|
The PEP currently uses @contextmanager.
|
||||||
2. The original resolution was for the decorator to make a context
|
|
||||||
manager from a generator to be a builtin called "contextmanager".
|
|
||||||
The shorter term "context" was considered too ambiguous and
|
|
||||||
potentially confusing [9].
|
|
||||||
The different flavours of generators could then be described as:
|
|
||||||
- A "generator function" is an undecorated function containing
|
|
||||||
the 'yield' keyword, and the objects produced by
|
|
||||||
such functions are "generator-iterators". The term
|
|
||||||
"generator" may refer to either a generator function or a
|
|
||||||
generator-iterator depending on the situation.
|
|
||||||
- A "generator context function" is a generator function to
|
|
||||||
which the "contextmanager" decorator is applied and the
|
|
||||||
objects produced by such functions are "generator-context-
|
|
||||||
managers". The term "generator context" may refer to either
|
|
||||||
a generator context function or a generator-context-manager
|
|
||||||
depending on the situation.
|
|
||||||
|
|
||||||
In the Python 2.5 implementation, the decorator is actually part
|
|
||||||
of the standard library module contextlib. The ongoing
|
|
||||||
terminology review may lead to it being renamed
|
|
||||||
"contextlib.context" (with the existence of the underlying context
|
|
||||||
manager being an implementation detail).
|
|
||||||
|
|
||||||
|
|
||||||
Resolved Issues
|
Resolved Issues
|
||||||
|
|
||||||
The following issues were resolved either by BDFL approval,
|
The following issues were resolved by BDFL approval (and a lack
|
||||||
consensus on python-dev, or a simple lack of objection to
|
of any major objections on python-dev).
|
||||||
proposals in the original version of this PEP.
|
|
||||||
|
|
||||||
1. The __exit__() method of the GeneratorContextManager class
|
1. What exception should GeneratorContextManager raise when the
|
||||||
catches StopIteration and considers it equivalent to re-raising
|
|
||||||
the exception passed to throw(). Is allowing StopIteration
|
|
||||||
right here?
|
|
||||||
|
|
||||||
This is so that a generator doing cleanup depending on the
|
|
||||||
exception thrown (like the transactional() example below) can
|
|
||||||
*catch* the exception thrown if it wants to and doesn't have to
|
|
||||||
worry about re-raising it. I find this more convenient for the
|
|
||||||
generator writer. Against this was brought in that the
|
|
||||||
generator *appears* to suppress an exception that it cannot
|
|
||||||
suppress: the transactional() example would be more clear
|
|
||||||
according to this view if it re-raised the original exception
|
|
||||||
after the call to db.rollback(). I personally would find the
|
|
||||||
requirement to re-raise the exception an annoyance in a
|
|
||||||
generator used as a with-template, since all the code after
|
|
||||||
yield is used for is cleanup, and it is invoked from a
|
|
||||||
finally-clause (the one implicit in the with-statement) which
|
|
||||||
re-raises the original exception anyway.
|
|
||||||
|
|
||||||
2. What exception should GeneratorContextManager raise when the
|
|
||||||
underlying generator-iterator misbehaves? The following quote is
|
underlying generator-iterator misbehaves? The following quote is
|
||||||
the reason behind Guido's choice of RuntimeError for both this
|
the reason behind Guido's choice of RuntimeError for both this
|
||||||
and for the generator close() method in PEP 342 (from [8]):
|
and for the generator close() method in PEP 342 (from [8]):
|
||||||
|
@ -617,61 +565,32 @@ Resolved Issues
|
||||||
and for uninitialized objects (and for a variety of
|
and for uninitialized objects (and for a variety of
|
||||||
miscellaneous conditions)."
|
miscellaneous conditions)."
|
||||||
|
|
||||||
3. See item 1 in open issues :)
|
2. It is fine to raise AttributeError instead of TypeError if the
|
||||||
|
|
||||||
|
|
||||||
4. The originally approved version of this PEP did not include a
|
|
||||||
__context__ method - the method was only added to the PEP after
|
|
||||||
Jason Orendorff pointed out the difficulty of writing
|
|
||||||
appropriate __enter__ and __exit__ methods for decimal.Context
|
|
||||||
[5]. This approach allows a class to define a native context
|
|
||||||
manager using generator syntax. It also allows a class to use an
|
|
||||||
existing independent context as its native context object by
|
|
||||||
applying the independent context to 'self' in its __context__
|
|
||||||
method. It even allows a class written in C to
|
|
||||||
use a generator context manager written in Python.
|
|
||||||
The __context__ method parallels the __iter__ method which forms
|
|
||||||
part of the iterator protocol.
|
|
||||||
An earlier version of this PEP called this the __with__ method.
|
|
||||||
This was later changed to match the name of the protocol rather
|
|
||||||
than the keyword for the statement [9].
|
|
||||||
|
|
||||||
5. The suggestion was made by Jason Orendorff that the __enter__
|
|
||||||
and __exit__ methods could be removed from the context
|
|
||||||
management protocol, and the protocol instead defined directly
|
|
||||||
in terms of the enhanced generator interface described in PEP
|
|
||||||
342 [6].
|
|
||||||
Guido rejected this idea [7]. The following are some of benefits
|
|
||||||
of keeping the __enter__ and __exit__ methods:
|
|
||||||
- it makes it easy to implement a simple context in C
|
|
||||||
without having to rely on a separate coroutine builder
|
|
||||||
- it makes it easy to provide a low-overhead implementation
|
|
||||||
for contexts that don't need to maintain any
|
|
||||||
special state between the __enter__ and __exit__ methods
|
|
||||||
(having to use a generator for these would impose
|
|
||||||
unnecessary overhead without any compensating benefit)
|
|
||||||
- it makes it possible to understand how the with statement
|
|
||||||
works without having to first understand the mechanics of
|
|
||||||
how generator context managers are implemented.
|
|
||||||
|
|
||||||
6. See item 2 in open issues :)
|
|
||||||
|
|
||||||
7. A generator function used to implement a __context__ method will
|
|
||||||
need to be decorated with the contextmanager decorator in order
|
|
||||||
to have the correct behaviour. Otherwise, you will get an
|
|
||||||
AttributeError when using the class in a with statement, as
|
|
||||||
normal generator-iterators will NOT have __enter__ or __exit__
|
|
||||||
methods.
|
|
||||||
Getting deterministic closure of generators will require a
|
|
||||||
separate context manager such as the closing example below.
|
|
||||||
As Guido put it, "too much magic is bad for your health" [10].
|
|
||||||
|
|
||||||
8. It is fine to raise AttributeError instead of TypeError if the
|
|
||||||
relevant methods aren't present on a class involved in a with
|
relevant methods aren't present on a class involved in a with
|
||||||
statement. The fact that the abstract object C API raises
|
statement. The fact that the abstract object C API raises
|
||||||
TypeError rather than AttributeError is an accident of history,
|
TypeError rather than AttributeError is an accident of history,
|
||||||
rather than a deliberate design decision [11].
|
rather than a deliberate design decision [11].
|
||||||
|
|
||||||
|
Rejected Options
|
||||||
|
|
||||||
|
For several months, the PEP prohibited suppression of exceptions
|
||||||
|
in order to avoid hidden flow control. Implementation
|
||||||
|
revealed this to be a right royal pain, so Guido restored the
|
||||||
|
ability [13].
|
||||||
|
|
||||||
|
Another aspect of the PEP that caused no end of questions and
|
||||||
|
terminology debates was providing a __context__() method that
|
||||||
|
was analogous to an iterable's __iter__() method [5, 7, 9].
|
||||||
|
The ongoing problems [10, 13] with explaining what it was and why
|
||||||
|
it was and how it was meant to work eventually lead to Guido
|
||||||
|
killing the concept outright [15] (and there was much rejoicing!).
|
||||||
|
|
||||||
|
The notion of using the PEP 342 generator API directly to define
|
||||||
|
the with statement was also briefly entertained [6], but quickly
|
||||||
|
dismissed as making it too difficult to write non-generator
|
||||||
|
based context managers.
|
||||||
|
|
||||||
|
|
||||||
Examples
|
Examples
|
||||||
|
|
||||||
The generator based examples rely on PEP 342. Also, some of the
|
The generator based examples rely on PEP 342. Also, some of the
|
||||||
|
@ -703,10 +622,6 @@ Examples
|
||||||
# guaranteed to be released when the block is left (even
|
# guaranteed to be released when the block is left (even
|
||||||
# if via return or by an uncaught exception).
|
# if via return or by an uncaught exception).
|
||||||
|
|
||||||
PEP 319 gives a use case for also having an unlocked()
|
|
||||||
context; this can be written very similarly (just swap the
|
|
||||||
acquire() and release() calls).
|
|
||||||
|
|
||||||
2. A template for opening a file that ensures the file is closed
|
2. A template for opening a file that ensures the file is closed
|
||||||
when the block is left:
|
when the block is left:
|
||||||
|
|
||||||
|
@ -743,14 +658,10 @@ Examples
|
||||||
class locked:
|
class locked:
|
||||||
def __init__(self, lock):
|
def __init__(self, lock):
|
||||||
self.lock = lock
|
self.lock = lock
|
||||||
def __context__(self):
|
|
||||||
return self
|
|
||||||
def __enter__(self):
|
def __enter__(self):
|
||||||
self.lock.acquire()
|
self.lock.acquire()
|
||||||
def __exit__(self, type, value, tb):
|
def __exit__(self, type, value, tb):
|
||||||
self.lock.release()
|
self.lock.release()
|
||||||
if type is not None:
|
|
||||||
raise type, value, tb
|
|
||||||
|
|
||||||
(This example is easily modified to implement the other
|
(This example is easily modified to implement the other
|
||||||
relatively stateless examples; it shows that it is easy to avoid
|
relatively stateless examples; it shows that it is easy to avoid
|
||||||
|
@ -844,52 +755,59 @@ Examples
|
||||||
# so this must be outside the with-statement:
|
# so this must be outside the with-statement:
|
||||||
return +s
|
return +s
|
||||||
|
|
||||||
9. Here's a proposed native context manager for decimal.Context:
|
9. Here's a proposed context manager for the decimal module:
|
||||||
|
|
||||||
# This would be a new decimal.Context method
|
# This would be a new decimal.Context method
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def __context__(self):
|
def localcontext(ctx=None):
|
||||||
|
"""Set a new local decimal context for the block"""
|
||||||
|
# Default to using the current context
|
||||||
|
if ctx is None:
|
||||||
|
ctx = getcontext()
|
||||||
# We set the thread context to a copy of this context
|
# We set the thread context to a copy of this context
|
||||||
# to ensure that changes within the block are kept
|
# to ensure that changes within the block are kept
|
||||||
# local to the block. This also gives us thread safety
|
# local to the block.
|
||||||
# and supports nested usage of a given context.
|
newctx = ctx.copy()
|
||||||
newctx = self.copy()
|
|
||||||
oldctx = decimal.getcontext()
|
oldctx = decimal.getcontext()
|
||||||
decimal.setcontext(newctx)
|
decimal.setcontext(newctx)
|
||||||
try:
|
try:
|
||||||
yield newctx
|
yield newctx
|
||||||
finally:
|
finally:
|
||||||
|
# Always restore the original context
|
||||||
decimal.setcontext(oldctx)
|
decimal.setcontext(oldctx)
|
||||||
|
|
||||||
Sample usage:
|
Sample usage:
|
||||||
|
|
||||||
|
from decimal import localcontext, ExtendedContext
|
||||||
|
|
||||||
def sin(x):
|
def sin(x):
|
||||||
with decimal.getcontext() as ctx:
|
with localcontext() as ctx:
|
||||||
ctx.prec += 2
|
ctx.prec += 2
|
||||||
# Rest of sin calculation algorithm
|
# Rest of sin calculation algorithm
|
||||||
# uses a precision 2 greater than normal
|
# uses a precision 2 greater than normal
|
||||||
return +s # Convert result to normal precision
|
return +s # Convert result to normal precision
|
||||||
|
|
||||||
def sin(x):
|
def sin(x):
|
||||||
with decimal.ExtendedContext:
|
with localcontext(ExtendedContext):
|
||||||
# Rest of sin calculation algorithm
|
# Rest of sin calculation algorithm
|
||||||
# uses the Extended Context from the
|
# uses the Extended Context from the
|
||||||
# General Decimal Arithmetic Specification
|
# General Decimal Arithmetic Specification
|
||||||
return +s # Convert result to normal context
|
return +s # Convert result to normal context
|
||||||
|
|
||||||
10. A generic "object-closing" template:
|
10. A generic "object-closing" context manager:
|
||||||
|
|
||||||
@contextmanager
|
class closing(object):
|
||||||
def closing(obj):
|
def __init__(self, obj):
|
||||||
try:
|
self.obj = obj
|
||||||
yield obj
|
def __enter__(self):
|
||||||
finally:
|
return self.obj
|
||||||
|
def __exit__(self, *exc_info):
|
||||||
try:
|
try:
|
||||||
close = obj.close
|
close_it = self.obj.close
|
||||||
except AttributeError:
|
except AttributeError:
|
||||||
pass
|
pass
|
||||||
else:
|
else:
|
||||||
close()
|
close_it()
|
||||||
|
|
||||||
This can be used to deterministically close anything with a
|
This can be used to deterministically close anything with a
|
||||||
close method, be it file, generator, or something else. It
|
close method, be it file, generator, or something else. It
|
||||||
|
@ -907,20 +825,27 @@ Examples
|
||||||
for datum in data:
|
for datum in data:
|
||||||
process(datum)
|
process(datum)
|
||||||
|
|
||||||
11. Native contexts for objects with acquire/release methods:
|
(Python 2.5's contextlib module contains a version
|
||||||
|
of this context manager)
|
||||||
|
|
||||||
# This would be a new method of e.g., threading.RLock
|
11. PEP 319 gives a use case for also having a released()
|
||||||
def __context__(self):
|
context to temporarily release a previously acquired lock;
|
||||||
return locked(self)
|
this can be written very similarly to the locked context
|
||||||
|
manager above by swapping the acquire() and release() calls.
|
||||||
|
|
||||||
def released(self):
|
class released:
|
||||||
return unlocked(self)
|
def __init__(self, lock):
|
||||||
|
self.lock = lock
|
||||||
|
def __enter__(self):
|
||||||
|
self.lock.release()
|
||||||
|
def __exit__(self, type, value, tb):
|
||||||
|
self.lock.acquire()
|
||||||
|
|
||||||
Sample usage:
|
Sample usage:
|
||||||
|
|
||||||
with my_lock:
|
with my_lock:
|
||||||
# Operations with the lock held
|
# Operations with the lock held
|
||||||
with my_lock.released():
|
with released(my_lock):
|
||||||
# Operations without the lock
|
# Operations without the lock
|
||||||
# e.g. blocking I/O
|
# e.g. blocking I/O
|
||||||
# Lock is held again here
|
# Lock is held again here
|
||||||
|
@ -973,56 +898,81 @@ Examples
|
||||||
with c as z:
|
with c as z:
|
||||||
# Perform operation
|
# Perform operation
|
||||||
|
|
||||||
|
(Python 2.5's contextlib module contains a version
|
||||||
|
of this context manager)
|
||||||
|
|
||||||
Reference Implementation
|
Reference Implementation
|
||||||
|
|
||||||
This PEP was first accepted by Guido at his EuroPython
|
This PEP was first accepted by Guido at his EuroPython
|
||||||
keynote, 27 June 2005.
|
keynote, 27 June 2005.
|
||||||
It was accepted again later, with the __context__ method added.
|
It was accepted again later, with the __context__ method added.
|
||||||
The PEP was implemented for Python 2.5a1
|
The PEP was implemented in subversion for Python 2.5a1
|
||||||
|
The __context__() method will be removed in Python 2.5a3
|
||||||
|
|
||||||
|
|
||||||
|
Ackowledgements
|
||||||
|
|
||||||
|
Many people contributed to the ideas and concepts in this PEP,
|
||||||
|
including all those mentioned in the acknowledgements for PEP 340
|
||||||
|
and PEP 346.
|
||||||
|
|
||||||
|
Additional thanks goes to (in no meaningful order): Paul Moore,
|
||||||
|
Phillip J. Eby, Greg Ewing, Jason Orendorff, Michael Hudson,
|
||||||
|
Raymond Hettinger, Walter Dörwald, Aahz, Georg Brandl, Terry Reedy,
|
||||||
|
A.M. Kuchling, Brett Cannon, and all those that participated in the
|
||||||
|
discussions on python-dev.
|
||||||
|
|
||||||
|
|
||||||
References
|
References
|
||||||
|
|
||||||
[1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx
|
[1] Raymond Chen's article on hidden flow control
|
||||||
|
http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx
|
||||||
|
|
||||||
[2] http://mail.python.org/pipermail/python-dev/2005-May/053885.html
|
[2] Guido suggests some generator changes that ended up in PEP 342
|
||||||
|
http://mail.python.org/pipermail/python-dev/2005-May/053885.html
|
||||||
|
|
||||||
[3] http://wiki.python.org/moin/WithStatement
|
[3] Wiki discussion of PEP 343
|
||||||
|
http://wiki.python.org/moin/WithStatement
|
||||||
|
|
||||||
[4]
|
[4] Early draft of some documentation for the with statement
|
||||||
http://mail.python.org/pipermail/python-dev/2005-July/054658.html
|
http://mail.python.org/pipermail/python-dev/2005-July/054658.html
|
||||||
|
|
||||||
[5]
|
[5] Proposal to add the __with__ method
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/056947.html
|
http://mail.python.org/pipermail/python-dev/2005-October/056947.html
|
||||||
|
|
||||||
[6]
|
[6] Proposal to use the PEP 342 enhanced generator API directly
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/056969.html
|
http://mail.python.org/pipermail/python-dev/2005-October/056969.html
|
||||||
|
|
||||||
[7]
|
[7] Guido lets me (Nick Coghlan) talk him into a bad idea ;)
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/057018.html
|
http://mail.python.org/pipermail/python-dev/2005-October/057018.html
|
||||||
|
|
||||||
[8]
|
[8] Guido raises some exception handling questions
|
||||||
http://mail.python.org/pipermail/python-dev/2005-June/054064.html
|
http://mail.python.org/pipermail/python-dev/2005-June/054064.html
|
||||||
|
|
||||||
[9]
|
[9] Guido answers some questions about the __context__ method
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/057520.html
|
http://mail.python.org/pipermail/python-dev/2005-October/057520.html
|
||||||
|
|
||||||
[10]
|
[10] Guido answers more questions about the __context__ method
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/057535.html
|
http://mail.python.org/pipermail/python-dev/2005-October/057535.html
|
||||||
|
|
||||||
[11]
|
[11] Guido says AttributeError is fine for missing special methods
|
||||||
http://mail.python.org/pipermail/python-dev/2005-October/057625.html
|
http://mail.python.org/pipermail/python-dev/2005-October/057625.html
|
||||||
|
|
||||||
[12]
|
[12] Original PEP 342 implementation patch
|
||||||
http://sourceforge.net/tracker/index.php?func=detail&aid=1223381&group_id=5470&atid=305470
|
http://sourceforge.net/tracker/index.php?func=detail&aid=1223381&group_id=5470&atid=305470
|
||||||
|
|
||||||
[13]
|
[13] Guido restores the ability to suppress exceptions
|
||||||
http://mail.python.org/pipermail/python-dev/2006-February/061903.html
|
http://mail.python.org/pipermail/python-dev/2006-February/061909.html
|
||||||
|
|
||||||
[14]
|
[14] A simple question kickstarts a thorough review of PEP 343
|
||||||
http://mail.python.org/pipermail/python-dev/2006-April/063859.html
|
http://mail.python.org/pipermail/python-dev/2006-April/063859.html
|
||||||
|
|
||||||
|
[15] Guido kills the __context__() method
|
||||||
|
http://mail.python.org/pipermail/python-dev/2006-April/064632.html
|
||||||
|
|
||||||
|
[16] Greg propose 'context guard' instead of 'context manager'
|
||||||
|
http://mail.python.org/pipermail/python-dev/2006-May/064676.html
|
||||||
|
|
||||||
Copyright
|
Copyright
|
||||||
|
|
||||||
This document has been placed in the public domain.
|
This document has been placed in the public domain.
|
||||||
|
|
Loading…
Reference in New Issue