From db32145d0e1a769ddfd5c09d6a495ba825e95351 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Sun, 23 Apr 2006 05:14:10 +0000 Subject: [PATCH] Clarify the original meanings I intended for various terms, and record the fact that gaining consensus on the terminology is still an open issue --- pep-0343.txt | 165 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 113 insertions(+), 52 deletions(-) diff --git a/pep-0343.txt b/pep-0343.txt index 68bfe0871..86db11565 100644 --- a/pep-0343.txt +++ b/pep-0343.txt @@ -7,7 +7,7 @@ Status: Accepted Type: Standards Track Content-Type: text/plain Created: 13-May-2005 -Post-History: 2-Jun-2005, 16-Oct-2005, 29-Oct-2005 +Post-History: 2-Jun-2005, 16-Oct-2005, 29-Oct-2005, 23-Apr-2006 Abstract @@ -19,6 +19,7 @@ Abstract section on Resolved Issues). It's still at Draft status until Guido gives a final blessing to the updated PEP. + Author's Note This PEP was originally written in first person by Guido, and @@ -26,6 +27,13 @@ Author's Note on python-dev. Any first person references are from Guido's original. + Python's alpha release cycle revealed terminology problems in this + PEP and in the associated documentation and implementation [14]. + The current version of the PEP reflects the implementation and + documentation as at Python 2.5a2. The PEP will be updated to + reflect any changes made to the terminology prior to the final + Python 2.5 release. + Introduction After a lot of discussion about PEP 340 and alternatives, I @@ -225,9 +233,9 @@ Specification: The 'with' Statement The translation of the above statement is: - ctx = (EXPR).__context__() - exit = ctx.__exit__ # Not calling it yet - value = ctx.__enter__() + mgr = (EXPR).__context__() + exit = mgr.__exit__ # Not calling it yet + value = mgr.__enter__() exc = True try: try: @@ -244,7 +252,7 @@ Specification: The 'with' Statement if exc: exit(None, None, None) - Here, the lowercase variables (ctx, exit, value, exc) are internal + Here, the lowercase variables (mgr, exit, value, exc) are internal variables and not accessible to the user; they will most likely be implemented as special registers or stack positions. @@ -260,37 +268,37 @@ Specification: The 'with' Statement as exceptions by __exit__().) The call to the __context__() method serves a similar purpose to - that of the __iter__() method of iterator and iterables. An - object with with simple state requirements (such as + that of the __iter__() method of iterator and iterables. A context + object with simple state requirements (such as threading.RLock) may provide its own __enter__() and __exit__() methods, and simply return 'self' from its __context__ method. On - the other hand, an object with more complex state requirements - (such as decimal.Context) may return a distinct context object + the other hand, a context object with more complex state requirements + (such as decimal.Context) may return a distinct context manager each time its __context__ method is invoked. If the "as VAR" part of the syntax is omitted, the "VAR =" part of - the translation is omitted (but ctx.__enter__() is still called). + the translation is omitted (but mgr.__enter__() is still called). - The calling convention for ctx.__exit__() is as follows. If the + The calling convention for mgr.__exit__() is as follows. If the finally-suite was reached through normal completion of BLOCK or through a non-local goto (a break, continue or return statement in - BLOCK), ctx.__exit__() is called with three None arguments. If + BLOCK), mgr.__exit__() is called with three None arguments. If the finally-suite was reached through an exception raised in - BLOCK, ctx.__exit__() is called with three arguments representing + BLOCK, mgr.__exit__() is called with three arguments representing the exception type, value, and traceback. - IMPORTANT: if ctx.__exit__() returns a "true" value, the exception + IMPORTANT: if mgr.__exit__() returns a "true" value, the exception is "swallowed". That is, if it returns "true", execution continues at the next statement after the with-statement, even if an exception happened inside the with-statement. However, if the with-statement was left via a non-local goto (break, continue or - return), this non-local return is resumed when ctx.__exit__() + return), this non-local return is resumed when mgr.__exit__() returns regardless of the return value. The motivation for this - detail is to make it possible for ctx.__exit__() to swallow + detail is to make it possible for mgr.__exit__() to swallow exceptions, without making it too easy (since the default return value, None, is false and this causes the exception to be re-raised). The main use case for swallowing exceptions is to - make it possible to write the @contextmanager decorator so thatn + make it possible to write the @contextmanager decorator so that a try/except block in a decorated generator behaves exactly as if the body of the generator were expanded in-line at the place of the with-statement. @@ -330,6 +338,17 @@ Specification: The 'with' Statement methods should avoid raising errors unless they have actually failed. (And allowing the original error to proceed isn't a failure.) + + Objects returned by __context__() methods should also provide a + __context__() method that returns self. This allows a program to + retrieve the context manager directly without breaking anything. + For example, the following should work just as well as the normal + case where the extra variable isn't used: + + mgr = (EXPR).__context__() + with mgr as VAR: + BLOCK + Transition Plan @@ -406,13 +425,13 @@ Generator Decorator finally: f.close() # Ditto for errors here (however unlikely) - A robust builtin implementation of this decorator will be made + A robust implementation of this decorator will be made part of the standard library. Just as generator-iterator functions are very useful for writing - __iter__() methods for iterables, generator-context functions will - be very useful for writing __context__() methods for contexts. - These methods will still need to be decorated using the + __iter__() methods for iterables, generator context functions will + be very useful for writing __context__() methods for context + objects. These methods will still need to be decorated using the contextmanager decorator. To ensure an obvious error message if the decorator is left out, generator-iterator objects will NOT be given a native context - if you want to ensure a generator is closed @@ -446,7 +465,7 @@ Optional Extensions is entered). OTOH such mistakes are easily diagnosed; for example, the - generator-context decorator above raises RuntimeError when a + generator context decorator above raises RuntimeError when a second with-statement calls f.__enter__() again. A similar error can be raised if __enter__ is invoked on a closed file object. @@ -470,19 +489,70 @@ Standard Terminology returns an iterator (this means that all iterators are iterables, but not all iterables are iterators). - This PEP proposes that the protocol used by the with statement be - known as the "context management protocol", and that objects that - implement that protocol be known as "context managers". The term - "context manager" then encompasses all objects with a __context__() - method that returns a context object. (This means that all contexts - are context managers, but not all context managers are contexts). + This PEP proposes that the protocol consisting of the __enter__() + and __exit__() methods, and a __context__() method that returns + self be known as the "context management protocol", and that + objects that implement that protocol be known as "context + managers". - The term "context" is based on the concept that the context object - defines a context of execution for the code that forms the body - of the with statement. + The term "context object" then encompasses all objects with a + __context__() method that returns a context manager. The protocol + these objects implement is called the "context protocol". This + means that all context managers are context objects, but not all + context objects are context managers, just as all iterators are + iterables, but not all iterables are iterators. + + These terms are based on the concept that the context object + defines a context of execution for the code that forms the body of + the with statement. The role of the context manager is to + translate the context object's stored state into an active + manipulation of the runtime environment to setup and tear down the + desired runtime context for the duration of the with statement. + For example, a synchronisation lock's context manager acquires the + lock when entering the with statement, and releases the lock when + leaving it. The runtime context established within the body of the + with statement is that the synchronisation lock is currently held. + + The general term "context" is unfortunately ambiguous. If necessary, + it can be made more explicit by using the terms "context objext" for + objects providing a __context__() method and "runtime context" for + the runtime environment modifications made by the context manager. + When solely discussing use of the with statement, the distinction + between the two shouldn't matter as the context object fully + defines the changes made to the runtime context. The distinction is + more important when discussing the process of implementing context + objects and context managers. + +Open Issues + + 1. As noted earlier, the standard terminology section has not yet + met with consensus on python-dev. It will be refined throughout + the Python 2.5 release cycle based on user feedback on the + usability of the documentation. + + 2. The original resolution was for the decorator to make a context + manager from a generator to be a builtin called "contextmanager". + The shorter term "context" was considered too ambiguous and + potentially confusing [9]. + The different flavours of generators could then be described as: + - A "generator function" is an undecorated function containing + the 'yield' keyword, and the objects produced by + such functions are "generator-iterators". The term + "generator" may refer to either a generator function or a + generator-iterator depending on the situation. + - A "generator context function" is a generator function to + which the "contextmanager" decorator is applied and the + objects produced by such functions are "generator-context- + managers". The term "generator context" may refer to either + a generator context function or a generator-context-manager + depending on the situation. + + In the Python 2.5 implementation, the decorator is actually part + of the standard library module contextlib. The ongoing + terminology review may lead to it being renamed + "contextlib.context" (with the existence of the underlying context + manager being an implementation detail). - In cases where the general term "context" would be ambiguous, it - can be made explicit by expanding it to "manageable context". Resolved Issues @@ -570,21 +640,7 @@ Resolved Issues works without having to first understand the mechanics of how generator context managers are implemented. - 6. The decorator to make a context manager from a generator will be - a builtin called "contextmanager". The shorter term "context" was - considered too ambiguous and potentially confusing [9]. - The different flavours of generators can then be described as: - - A "generator function" is an undecorated function containing - the 'yield' keyword, and the objects produced by - such functions are "generator-iterators". The term - "generator" may refer to either a generator function or a - generator-iterator depending on the situation. - - A "generator context function" is a generator function to - which the "contextmanager" decorator is applied and the - objects produced by such functions are "generator-context- - managers". The term "generator context" may refer to either a - generator context function or a generator-context-manager - depending on the situation. + 6. See point 2 in open issues :) 7. A generator function used to implement a __context__ method will need to be decorated with the contextmanager decorator in order @@ -609,7 +665,7 @@ Examples appropriate objects, such as threading.RLock, will be able to be used directly in with statements. - The tense used in the names of the example context managers is not + The tense used in the names of the example contexts is not arbitrary. Past tense ("-ed") is used when the name refers to an action which is done in the __enter__ method and undone in the __exit__ method. Progressive tense ("-ing") is used when the name @@ -634,7 +690,7 @@ Examples # if via return or by an uncaught exception). PEP 319 gives a use case for also having an unlocked() - template; this can be written very similarly (just swap the + context; this can be written very similarly (just swap the acquire() and release() calls). 2. A template for opening a file that ensures the file is closed @@ -903,8 +959,10 @@ Examples Reference Implementation - There is no implementation at this time. This PEP was accepted - by Guido at his EuroPython keynote, 27 June 2005. + This PEP was first accepted by Guido at his EuroPython + keynote, 27 June 2005. + It was accepted again later, with the __context__ method added. + The PEP was implemented for Python 2.5a1 References @@ -945,6 +1003,9 @@ References [13] http://mail.python.org/pipermail/python-dev/2006-February/061903.html + [14] + http://mail.python.org/pipermail/python-dev/2006-April/063859.html + Copyright This document has been placed in the public domain.