python-peps/pep-0340/index.html

671 lines
53 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 340 Anonymous Block Statements | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0340/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 340 Anonymous Block Statements | peps.python.org'>
<meta property="og:description" content="This PEP proposes a new type of compound statement which can be used for resource management purposes. The new statement type is provisionally called the block-statement because the keyword to be used has not yet been chosen.">
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0340/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="This PEP proposes a new type of compound statement which can be used for resource management purposes. The new statement type is provisionally called the block-statement because the keyword to be used has not yet been chosen.">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 340</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 340 Anonymous Block Statements</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Guido van Rossum</dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Formally declined and will not be accepted">Rejected</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Created<span class="colon">:</span></dt>
<dd class="field-even">27-Apr-2005</dd>
<dt class="field-odd">Post-History<span class="colon">:</span></dt>
<dd class="field-odd"><p></p></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#introduction">Introduction</a></li>
<li><a class="reference internal" href="#rejection-notice">Rejection Notice</a></li>
<li><a class="reference internal" href="#motivation-and-summary">Motivation and Summary</a></li>
<li><a class="reference internal" href="#use-cases">Use Cases</a></li>
<li><a class="reference internal" href="#specification-the-exit-method">Specification: the __exit__() Method</a></li>
<li><a class="reference internal" href="#specification-the-anonymous-block-statement">Specification: the Anonymous Block Statement</a></li>
<li><a class="reference internal" href="#specification-generator-exit-handling">Specification: Generator Exit Handling</a></li>
<li><a class="reference internal" href="#alternatives-considered-and-rejected">Alternatives Considered and Rejected</a></li>
<li><a class="reference internal" href="#comparison-to-thunks">Comparison to Thunks</a></li>
<li><a class="reference internal" href="#examples">Examples</a></li>
<li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<section id="introduction">
<h2><a class="toc-backref" href="#introduction" role="doc-backlink">Introduction</a></h2>
<p>This PEP proposes a new type of compound statement which can be
used for resource management purposes. The new statement type
is provisionally called the block-statement because the keyword
to be used has not yet been chosen.</p>
<p>This PEP competes with several other PEPs: <a class="pep reference internal" href="../pep-0288/" title="PEP 288 Generators Attributes and Exceptions">PEP 288</a> (Generators
Attributes and Exceptions; only the second part), <a class="pep reference internal" href="../pep-0310/" title="PEP 310 Reliable Acquisition/Release Pairs">PEP 310</a>
(Reliable Acquisition/Release Pairs), and <a class="pep reference internal" href="../pep-0325/" title="PEP 325 Resource-Release Support for Generators">PEP 325</a>
(Resource-Release Support for Generators).</p>
<p>I should clarify that using a generator to “drive” a block
statement is really a separable proposal; with just the definition
of the block statement from the PEP you could implement all the
examples using a class (similar to example 6, which is easily
turned into a template). But the key idea is using a generator to
drive a block statement; the rest is elaboration, so Id like to
keep these two parts together.</p>
<p>(<a class="pep reference internal" href="../pep-0342/" title="PEP 342 Coroutines via Enhanced Generators">PEP 342</a>, Enhanced Iterators, was originally a part of this PEP;
but the two proposals are really independent and with Steven
Bethards help I have moved it to a separate PEP.)</p>
</section>
<section id="rejection-notice">
<h2><a class="toc-backref" href="#rejection-notice" role="doc-backlink">Rejection Notice</a></h2>
<p>I am rejecting this PEP in favor of <a class="pep reference internal" href="../pep-0343/" title="PEP 343 The “with” Statement">PEP 343</a>. See the motivational
section in that PEP for the reasoning behind this rejection. GvR.</p>
</section>
<section id="motivation-and-summary">
<h2><a class="toc-backref" href="#motivation-and-summary" role="doc-backlink">Motivation and Summary</a></h2>
<p>(Thanks to Shane Hathaway Hi Shane!)</p>
<p>Good programmers move commonly used code into reusable functions.
Sometimes, however, patterns arise in the structure of the
functions rather than the actual sequence of statements. For
example, many functions acquire a lock, execute some code specific
to that function, and unconditionally release the lock. Repeating
the locking code in every function that uses it is error prone and
makes refactoring difficult.</p>
<p>Block statements provide a mechanism for encapsulating patterns of
structure. Code inside the block statement runs under the control
of an object called a block iterator. Simple block iterators
execute code before and after the code inside the block statement.
Block iterators also have the opportunity to execute the
controlled code more than once (or not at all), catch exceptions,
or receive data from the body of the block statement.</p>
<p>A convenient way to write block iterators is to write a generator
(<a class="pep reference internal" href="../pep-0255/" title="PEP 255 Simple Generators">PEP 255</a>). A generator looks a lot like a Python function, but
instead of returning a value immediately, generators pause their
execution at “yield” statements. When a generator is used as a
block iterator, the yield statement tells the Python interpreter
to suspend the block iterator, execute the block statement body,
and resume the block iterator when the body has executed.</p>
<p>The Python interpreter behaves as follows when it encounters a
block statement based on a generator. First, the interpreter
instantiates the generator and begins executing it. The generator
does setup work appropriate to the pattern it encapsulates, such
as acquiring a lock, opening a file, starting a database
transaction, or starting a loop. Then the generator yields
execution to the body of the block statement using a yield
statement. When the block statement body completes, raises an
uncaught exception, or sends data back to the generator using a
continue statement, the generator resumes. At this point, the
generator can either clean up and stop or yield again, causing the
block statement body to execute again. When the generator
finishes, the interpreter leaves the block statement.</p>
</section>
<section id="use-cases">
<h2><a class="toc-backref" href="#use-cases" role="doc-backlink">Use Cases</a></h2>
<p>See the Examples section near the end.</p>
</section>
<section id="specification-the-exit-method">
<h2><a class="toc-backref" href="#specification-the-exit-method" role="doc-backlink">Specification: the __exit__() Method</a></h2>
<p>An optional new method for iterators is proposed, called
<code class="docutils literal notranslate"><span class="pre">__exit__()</span></code>. It takes up to three arguments which correspond to
the three “arguments” to the raise-statement: type, value, and
traceback. If all three arguments are <code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">sys.exc_info()</span></code> may be
consulted to provide suitable default values.</p>
</section>
<section id="specification-the-anonymous-block-statement">
<h2><a class="toc-backref" href="#specification-the-anonymous-block-statement" role="doc-backlink">Specification: the Anonymous Block Statement</a></h2>
<p>A new statement is proposed with the syntax:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">EXPR1</span> <span class="k">as</span> <span class="n">VAR1</span><span class="p">:</span>
<span class="n">BLOCK1</span>
</pre></div>
</div>
<p>Here, block and as are new keywords; <code class="docutils literal notranslate"><span class="pre">EXPR1</span></code> is an arbitrary
expression (but not an expression-list) and <code class="docutils literal notranslate"><span class="pre">VAR1</span></code> is an arbitrary
assignment target (which may be a comma-separated list).</p>
<p>The “as VAR1” part is optional; if omitted, the assignments to
VAR1 in the translation below are omitted (but the expressions
assigned are still evaluated!).</p>
<p>The choice of the block keyword is contentious; many
alternatives have been proposed, including not to use a keyword at
all (which I actually like). <a class="pep reference internal" href="../pep-0310/" title="PEP 310 Reliable Acquisition/Release Pairs">PEP 310</a> uses with for similar
semantics, but I would like to reserve that for a with-statement
similar to the one found in Pascal and VB. (Though I just found
that the C# designers dont like with <a class="footnote-reference brackets" href="#id3" id="id1">[2]</a>, and I have to agree
with their reasoning.) To sidestep this issue momentarily Im
using block until we can agree on the right keyword, if any.</p>
<p>Note that the as keyword is not contentious (it will finally be
elevated to proper keyword status).</p>
<p>Note that it is up to the iterator to decide whether a
block-statement represents a loop with multiple iterations; in the
most common use case <code class="docutils literal notranslate"><span class="pre">BLOCK1</span></code> is executed exactly once. To the
parser, however, it is always a loop; break and continue return
transfer to the blocks iterator (see below for details).</p>
<p>The translation is subtly different from a for-loop: <code class="docutils literal notranslate"><span class="pre">iter()</span></code> is
not called, so <code class="docutils literal notranslate"><span class="pre">EXPR1</span></code> should already be an iterator (not just an
iterable); and the iterator is guaranteed to be notified when
the block-statement is left, regardless if this is due to a
break, return or exception:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">itr</span> <span class="o">=</span> <span class="n">EXPR1</span> <span class="c1"># The iterator</span>
<span class="n">ret</span> <span class="o">=</span> <span class="kc">False</span> <span class="c1"># True if a return statement is active</span>
<span class="n">val</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># Return value, if ret == True</span>
<span class="n">exc</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># sys.exc_info() tuple if an exception is active</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">if</span> <span class="n">exc</span><span class="p">:</span>
<span class="n">ext</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">itr</span><span class="p">,</span> <span class="s2">&quot;__exit__&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">if</span> <span class="n">ext</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">VAR1</span> <span class="o">=</span> <span class="n">ext</span><span class="p">(</span><span class="o">*</span><span class="n">exc</span><span class="p">)</span> <span class="c1"># May re-raise *exc</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">exc</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">exc</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">exc</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">VAR1</span> <span class="o">=</span> <span class="n">itr</span><span class="o">.</span><span class="n">next</span><span class="p">()</span> <span class="c1"># May raise StopIteration</span>
<span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span>
<span class="k">if</span> <span class="n">ret</span><span class="p">:</span>
<span class="k">return</span> <span class="n">val</span>
<span class="k">break</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">ret</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">exc</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">BLOCK1</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">exc</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()</span>
</pre></div>
</div>
<p>(However, the variables itr etc. are not user-visible and the
built-in names used cannot be overridden by the user.)</p>
<p>Inside <code class="docutils literal notranslate"><span class="pre">BLOCK1</span></code>, the following special translations apply:</p>
<ul>
<li>“break” is always legal; it is translated into:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">exc</span> <span class="o">=</span> <span class="p">(</span><span class="ne">StopIteration</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="k">continue</span>
</pre></div>
</div>
</li>
<li>“return EXPR3” is only legal when the block-statement is
contained in a function definition; it is translated into:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">exc</span> <span class="o">=</span> <span class="p">(</span><span class="ne">StopIteration</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="n">ret</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">EXPR3</span>
<span class="k">continue</span>
</pre></div>
</div>
</li>
</ul>
<p>The net effect is that break and return behave much the same as
if the block-statement were a for-loop, except that the iterator
gets a chance at resource cleanup before the block-statement is
left, through the optional <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> method. The iterator also
gets a chance if the block-statement is left through raising an
exception. If the iterator doesnt have an <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> method,
there is no difference with a for-loop (except that a for-loop
calls <code class="docutils literal notranslate"><span class="pre">iter()</span></code> on <code class="docutils literal notranslate"><span class="pre">EXPR1</span></code>).</p>
<p>Note that a yield-statement in a block-statement is not treated
differently. It suspends the function containing the block
<strong>without</strong> notifying the blocks iterator. The blocks iterator is
entirely unaware of this yield, since the local control flow
doesnt actually leave the block. In other words, it is <strong>not</strong>
like a break or return statement. When the loop that was resumed
by the yield calls <code class="docutils literal notranslate"><span class="pre">next()</span></code>, the block is resumed right after the
yield. (See example 7 below.) The generator finalization
semantics described below guarantee (within the limitations of all
finalization semantics) that the block will be resumed eventually.</p>
<p>Unlike the for-loop, the block-statement does not have an
else-clause. I think it would be confusing, and emphasize the
“loopiness” of the block-statement, while I want to emphasize its
<strong>difference</strong> from a for-loop. In addition, there are several
possible semantics for an else-clause, and only a very weak use
case.</p>
</section>
<section id="specification-generator-exit-handling">
<h2><a class="toc-backref" href="#specification-generator-exit-handling" role="doc-backlink">Specification: Generator Exit Handling</a></h2>
<p>Generators will implement the new <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> method API.</p>
<p>Generators will be allowed to have a <code class="docutils literal notranslate"><span class="pre">yield</span></code> statement inside a
try-finally statement.</p>
<p>The expression argument to the yield-statement will become
optional (defaulting to None).</p>
<p>When <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> is called, the generator is resumed but at the
point of the yield-statement the exception represented by the
<code class="docutils literal notranslate"><span class="pre">__exit__</span></code> argument(s) is raised. The generator may re-raise this
exception, raise another exception, or yield another value,
except that if the exception passed in to <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> was
StopIteration, it ought to raise StopIteration (otherwise the
effect would be that a break is turned into continue, which is
unexpected at least). When the <strong>initial</strong> call resuming the
generator is an <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> call instead of a <code class="docutils literal notranslate"><span class="pre">next()</span></code> call, the
generators execution is aborted and the exception is re-raised
without passing control to the generators body.</p>
<p>When a generator that has not yet terminated is garbage-collected
(either through reference counting or by the cyclical garbage
collector), its <code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> method is called once with
StopIteration as its first argument. Together with the
requirement that a generator ought to raise StopIteration when
<code class="docutils literal notranslate"><span class="pre">__exit__()</span></code> is called with StopIteration, this guarantees the
eventual activation of any finally-clauses that were active when
the generator was last suspended. Of course, under certain
circumstances the generator may never be garbage-collected. This
is no different than the guarantees that are made about finalizers
(<code class="docutils literal notranslate"><span class="pre">__del__()</span></code> methods) of other objects.</p>
</section>
<section id="alternatives-considered-and-rejected">
<h2><a class="toc-backref" href="#alternatives-considered-and-rejected" role="doc-backlink">Alternatives Considered and Rejected</a></h2>
<ul>
<li>Many alternatives have been proposed for block. I havent
seen a proposal for another keyword that I like better than
block yet. Alas, block is also not a good choice; it is a
rather popular name for variables, arguments and methods.
Perhaps with is the best choice after all?</li>
<li>Instead of trying to pick the ideal keyword, the block-statement
could simply have the form:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">EXPR1</span> <span class="k">as</span> <span class="n">VAR1</span><span class="p">:</span>
<span class="n">BLOCK1</span>
</pre></div>
</div>
<p>This is at first attractive because, together with a good choice
of function names (like those in the Examples section below)
used in <code class="docutils literal notranslate"><span class="pre">EXPR1</span></code>, it reads well, and feels like a “user-defined
statement”. And yet, it makes me (and many others)
uncomfortable; without a keyword the syntax is very “bland”,
difficult to look up in a manual (remember that as is
optional), and it makes the meaning of break and continue in the
block-statement even more confusing.</p>
</li>
<li>Phillip Eby has proposed to have the block-statement use
an entirely different API than the for-loop, to differentiate
between the two. A generator would have to be wrapped in a
decorator to make it support the block API. IMO this adds more
complexity with very little benefit; and we cant really deny
that the block-statement is conceptually a loop it supports
break and continue, after all.</li>
<li>This keeps getting proposed: “block VAR1 = EXPR1” instead of
“block EXPR1 as VAR1”. That would be very misleading, since
VAR1 does <strong>not</strong> get assigned the value of EXPR1; EXPR1 results
in a generator which is assigned to an internal variable, and
VAR1 is the value returned by successive calls to the <code class="docutils literal notranslate"><span class="pre">__next__()</span></code>
method of that iterator.</li>
<li>Why not change the translation to apply <code class="docutils literal notranslate"><span class="pre">iter(EXPR1)</span></code>? All the
examples would continue to work. But this makes the
block-statement <strong>more</strong> like a for-loop, while the emphasis ought
to be on the <strong>difference</strong> between the two. Not calling <code class="docutils literal notranslate"><span class="pre">iter()</span></code>
catches a bunch of misunderstandings, like using a sequence as
<code class="docutils literal notranslate"><span class="pre">EXPR1</span></code>.</li>
</ul>
</section>
<section id="comparison-to-thunks">
<h2><a class="toc-backref" href="#comparison-to-thunks" role="doc-backlink">Comparison to Thunks</a></h2>
<p>Alternative semantics proposed for the block-statement turn the
block into a thunk (an anonymous function that blends into the
containing scope).</p>
<p>The main advantage of thunks that I can see is that you can save
the thunk for later, like a callback for a button widget (the
thunk then becomes a closure). You cant use a yield-based block
for that (except in Ruby, which uses yield syntax with a
thunk-based implementation). But I have to say that I almost see
this as an advantage: I think Id be slightly uncomfortable seeing
a block and not knowing whether it will be executed in the normal
control flow or later. Defining an explicit nested function for
that purpose doesnt have this problem for me, because I already
know that the def keyword means its body is executed later.</p>
<p>The other problem with thunks is that once we think of them as the
anonymous functions they are, were pretty much forced to say that
a return statement in a thunk returns from the thunk rather than
from the containing function. Doing it any other way would cause
major weirdness when the thunk were to survive its containing
function as a closure (perhaps continuations would help, but Im
not about to go there :-).</p>
<p>But then an IMO important use case for the resource cleanup
template pattern is lost. I routinely write code like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">findSomething</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">elements</span><span class="p">:</span>
<span class="k">if</span> <span class="n">item</span><span class="o">.</span><span class="n">matches</span><span class="p">(</span><span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">return</span> <span class="n">default</span>
<span class="k">finally</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>
</pre></div>
</div>
<p>and Id be bummed if I couldnt write this as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">findSomething</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">block</span> <span class="n">locking</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="p">):</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">elements</span><span class="p">:</span>
<span class="k">if</span> <span class="n">item</span><span class="o">.</span><span class="n">matches</span><span class="p">(</span><span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">return</span> <span class="n">default</span>
</pre></div>
</div>
<p>This particular example can be rewritten using a break:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">findSomething</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">block</span> <span class="n">locking</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="p">):</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">elements</span><span class="p">:</span>
<span class="k">if</span> <span class="n">item</span><span class="o">.</span><span class="n">matches</span><span class="p">(</span><span class="n">key</span><span class="p">):</span>
<span class="k">break</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">item</span> <span class="o">=</span> <span class="n">default</span>
<span class="k">return</span> <span class="n">item</span>
</pre></div>
</div>
<p>but it looks forced and the transformation isnt always that easy;
youd be forced to rewrite your code in a single-return style
which feels too restrictive.</p>
<p>Also note the semantic conundrum of a yield in a thunk the only
reasonable interpretation is that this turns the thunk into a
generator!</p>
<p>Greg Ewing believes that thunks “would be a lot simpler, doing
just what is required without any jiggery pokery with exceptions
and break/continue/return statements. It would be easy to explain
what it does and why its useful.”</p>
<p>But in order to obtain the required local variable sharing between
the thunk and the containing function, every local variable used
or set in the thunk would have to become a cell (our mechanism
for sharing variables between nested scopes). Cells slow down
access compared to regular local variables: access involves an
extra C function call (<code class="docutils literal notranslate"><span class="pre">PyCell_Get()</span></code> or <code class="docutils literal notranslate"><span class="pre">PyCell_Set()</span></code>).</p>
<p>Perhaps not entirely coincidentally, the last example above
(<code class="docutils literal notranslate"><span class="pre">findSomething()</span></code> rewritten to avoid a return inside the block)
shows that, unlike for regular nested functions, well want
variables <strong>assigned to</strong> by the thunk also to be shared with the
containing function, even if they are not assigned to outside the
thunk.</p>
<p>Greg Ewing again: “generators have turned out to be more powerful,
because you can have more than one of them on the go at once. Is
there a use for that capability here?”</p>
<p>I believe there are definitely uses for this; several people have
already shown how to do asynchronous light-weight threads using
generators (e.g. David Mertz quoted in <a class="pep reference internal" href="../pep-0288/" title="PEP 288 Generators Attributes and Exceptions">PEP 288</a>, and Fredrik
Lundh <a class="footnote-reference brackets" href="#id4" id="id2">[3]</a>).</p>
<p>And finally, Greg says: “a thunk implementation has the potential
to easily handle multiple block arguments, if a suitable syntax
could ever be devised. Its hard to see how that could be done in
a general way with the generator implementation.”</p>
<p>However, the use cases for multiple blocks seem elusive.</p>
<p>(Proposals have since been made to change the implementation of
thunks to remove most of these objections, but the resulting
semantics are fairly complex to explain and to implement, so IMO
that defeats the purpose of using thunks in the first place.)</p>
</section>
<section id="examples">
<h2><a class="toc-backref" href="#examples" role="doc-backlink">Examples</a></h2>
<p>(Several of these examples contain “yield None”. If <a class="pep reference internal" href="../pep-0342/" title="PEP 342 Coroutines via Enhanced Generators">PEP 342</a> is
accepted, these can be changed to just “yield” of course.)</p>
<ol class="arabic">
<li>A template for ensuring that a lock, acquired at the start of a
block, is released when the block is left:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">locking</span><span class="p">(</span><span class="n">lock</span><span class="p">):</span>
<span class="n">lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">yield</span> <span class="kc">None</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">locking</span><span class="p">(</span><span class="n">myLock</span><span class="p">):</span>
<span class="c1"># Code here executes with myLock held. The lock is</span>
<span class="c1"># guaranteed to be released when the block is left (even</span>
<span class="c1"># if via return or by an uncaught exception).</span>
</pre></div>
</div>
</li>
<li>A template for opening a file that ensures the file is closed
when the block is left:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">opening</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s2">&quot;r&quot;</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">f</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">opening</span><span class="p">(</span><span class="s2">&quot;/etc/passwd&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">f</span><span class="p">:</span>
<span class="nb">print</span> <span class="n">line</span><span class="o">.</span><span class="n">rstrip</span><span class="p">()</span>
</pre></div>
</div>
</li>
<li>A template for committing or rolling back a database
transaction:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">transactional</span><span class="p">(</span><span class="n">db</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">yield</span> <span class="kc">None</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">db</span><span class="o">.</span><span class="n">rollback</span><span class="p">()</span>
<span class="k">raise</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">db</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span>
</pre></div>
</div>
</li>
<li>A template that tries something up to n times:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">auto_retry</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">exc</span><span class="o">=</span><span class="ne">Exception</span><span class="p">):</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">yield</span> <span class="kc">None</span>
<span class="k">return</span>
<span class="k">except</span> <span class="n">exc</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="c1"># perhaps log exception here</span>
<span class="k">continue</span>
<span class="k">raise</span> <span class="c1"># re-raise the exception we caught earlier</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">auto_retry</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="ne">IOError</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="s2">&quot;https://www.example.com/&quot;</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</pre></div>
</div>
</li>
<li>It is possible to nest blocks and combine templates:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">locking_opening</span><span class="p">(</span><span class="n">lock</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s2">&quot;r&quot;</span><span class="p">):</span>
<span class="n">block</span> <span class="n">locking</span><span class="p">(</span><span class="n">lock</span><span class="p">):</span>
<span class="n">block</span> <span class="n">opening</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">f</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">locking_opening</span><span class="p">(</span><span class="n">myLock</span><span class="p">,</span> <span class="s2">&quot;/etc/passwd&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">f</span><span class="p">:</span>
<span class="nb">print</span> <span class="n">line</span><span class="o">.</span><span class="n">rstrip</span><span class="p">()</span>
</pre></div>
</div>
<p>(If this example confuses you, consider that it is equivalent
to using a for-loop with a yield in its body in a regular
generator which is invoking another iterator or generator
recursively; see for example the source code for <code class="docutils literal notranslate"><span class="pre">os.walk()</span></code>.)</p>
</li>
<li>It is possible to write a regular iterator with the
semantics of example 1:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">locking</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">lock</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span> <span class="o">=</span> <span class="n">lock</span>
<span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arg</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="c1"># ignores arg</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">:</span>
<span class="k">assert</span> <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">==</span> <span class="mi">1</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">raise</span> <span class="ne">StopIteration</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="k">def</span> <span class="fm">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">type</span><span class="p">,</span> <span class="n">value</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">traceback</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="ow">in</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>
<span class="k">raise</span> <span class="nb">type</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">traceback</span>
</pre></div>
</div>
<p>(This example is easily modified to implement the other
examples; it shows how much simpler generators are for the same
purpose.)</p>
</li>
<li>Redirect stdout temporarily:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">redirecting_stdout</span><span class="p">(</span><span class="n">new_stdout</span><span class="p">):</span>
<span class="n">save_stdout</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdout</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span> <span class="o">=</span> <span class="n">new_stdout</span>
<span class="k">yield</span> <span class="kc">None</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span> <span class="o">=</span> <span class="n">save_stdout</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">opening</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s2">&quot;w&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">block</span> <span class="n">redirecting_stdout</span><span class="p">(</span><span class="n">f</span><span class="p">):</span>
<span class="nb">print</span> <span class="s2">&quot;Hello world&quot;</span>
</pre></div>
</div>
</li>
<li>A variant on <code class="docutils literal notranslate"><span class="pre">opening()</span></code> that also returns an error condition:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">opening_w_error</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s2">&quot;r&quot;</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">IOError</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">yield</span> <span class="kc">None</span><span class="p">,</span> <span class="n">err</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">f</span><span class="p">,</span> <span class="kc">None</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>Used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">block</span> <span class="n">opening_w_error</span><span class="p">(</span><span class="s2">&quot;/etc/passwd&quot;</span><span class="p">,</span> <span class="s2">&quot;a&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">if</span> <span class="n">err</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">&quot;IOError:&quot;</span><span class="p">,</span> <span class="n">err</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">&quot;guido::0:0::/:/bin/sh</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">)</span>
</pre></div>
</div>
</li>
</ol>
</section>
<section id="acknowledgements">
<h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2>
<p>In no useful order: Alex Martelli, Barry Warsaw, Bob Ippolito,
Brett Cannon, Brian Sabbey, Chris Ryland, Doug Landauer, Duncan
Booth, Fredrik Lundh, Greg Ewing, Holger Krekel, Jason Diamond,
Jim Jewett, Josiah Carlson, Ka-Ping Yee, Michael Chermside,
Michael Hudson, Neil Schemenauer, Alyssa Coghlan, Paul Moore,
Phillip Eby, Raymond Hettinger, Georg Brandl, Samuele
Pedroni, Shannon Behrens, Skip Montanaro, Steven Bethard, Terry
Reedy, Tim Delaney, Aahz, and others. Thanks all for the valuable
contributions!</p>
</section>
<section id="references">
<h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2>
<p>[1] <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2005-April/052821.html">https://mail.python.org/pipermail/python-dev/2005-April/052821.html</a></p>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="id3" role="doc-footnote">
<dt class="label" id="id3">[<a href="#id1">2</a>]</dt>
<dd><a class="reference external" href="https://web.archive.org/web/20060719195933/http://msdn.microsoft.com/vcsharp/programming/language/ask/withstatement/">https://web.archive.org/web/20060719195933/http://msdn.microsoft.com/vcsharp/programming/language/ask/withstatement/</a></aside>
<aside class="footnote brackets" id="id4" role="doc-footnote">
<dt class="label" id="id4">[<a href="#id2">3</a>]</dt>
<dd><a class="reference external" href="https://web.archive.org/web/20050204062901/http://effbot.org/zone/asyncore-generators.htm">https://web.archive.org/web/20050204062901/http://effbot.org/zone/asyncore-generators.htm</a></aside>
</aside>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document has been placed in the public domain.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0340.rst">https://github.com/python/peps/blob/main/peps/pep-0340.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0340.rst">2023-10-11 12:05:51 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#introduction">Introduction</a></li>
<li><a class="reference internal" href="#rejection-notice">Rejection Notice</a></li>
<li><a class="reference internal" href="#motivation-and-summary">Motivation and Summary</a></li>
<li><a class="reference internal" href="#use-cases">Use Cases</a></li>
<li><a class="reference internal" href="#specification-the-exit-method">Specification: the __exit__() Method</a></li>
<li><a class="reference internal" href="#specification-the-anonymous-block-statement">Specification: the Anonymous Block Statement</a></li>
<li><a class="reference internal" href="#specification-generator-exit-handling">Specification: Generator Exit Handling</a></li>
<li><a class="reference internal" href="#alternatives-considered-and-rejected">Alternatives Considered and Rejected</a></li>
<li><a class="reference internal" href="#comparison-to-thunks">Comparison to Thunks</a></li>
<li><a class="reference internal" href="#examples">Examples</a></li>
<li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0340.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>