Add PEP 403, Statement Local Classes and Functions, a significantly simpler alternative to PEP 3150

pep-0403.txt

@@ -0,0 +1,304 @@
+PEP: 403
+Title: Statement local classes and functions
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan@gmail.com>
+Status: Deferred
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 2011-10-13
+Python-Version: 3.x
+Post-History: 2011-10-13
+Resolution: TBD
+
+
+Abstract
+========
+
+This PEP proposes the addition of ':' as a new class and function prefix
+syntax (analogous to decorators) that permits a statement local function or
+class definition to be appended to any Python statement that currently does
+not have an associated suite.
+
+In addition, the new syntax would allow the '@' symbol to be used to refer
+to the statement local function or class without needing to repeat the name.
+
+When the ':' prefix syntax is used, the associated statement would be executed
+*instead of* the normal local name binding currently implicit in function
+and class definitions.
+
+This PEP is based heavily on many of the ideas in PEP 3150 (Statement Local
+Namespaces) so some elements of the rationale will be familiar to readers of
+that PEP. That PEP has now been withdrawn in favour of this one.
+
+
+PEP Deferral
+============
+
+Like PEP 3150, this PEP currently exists in a deferred state. Unlike PEP 3150,
+this isn't because I suspect it might be a terrible idea or see nasty problems
+lurking in the implementation (aside from one potential parsing issue).
+
+Instead, it's because I think fleshing out the concept, exploring syntax
+variants, creating a reference implementation and generally championing
+the idea is going to require more time than I can give it in the 3.3 time
+frame.
+
+So, it's deferred. If anyone wants to step forward to drive the PEP for 3.3,
+let me know and I can add you as co-author and move it to Draft status.
+
+
+Basic Examples
+==============
+
+Before diving into the long history of this problem and the detailed
+rationale for this specific proposed solution, here are a few simple
+examples of the kind of code it is designed to simplify.
+
+As a trivial example, weakref callbacks could be defined as follows::
+
+    :x = weakref.ref(obj, @)
+    def report_destruction(obj):
+        print("{} is being destroyed".format(obj))
+
+This contrasts with the current repetitive "out of order" syntax for this
+operation::
+
+    def report_destruction(obj):
+        print("{} is being destroyed".format(obj))
+
+    x = weakref.ref(obj, report_destruction)
+
+That structure is OK when you're using the callable multiple times, but
+it's irritating to be forced into it for one-off operations.
+
+Similarly, singleton classes could now be defined as::
+
+  :instance = @()
+  class OnlyOneInstance:
+    pass
+
+Rather than::
+
+  class OnlyOneInstance:
+    pass
+
+  instance = OnlyOneInstance()
+
+And the infamous accumulator example could become::
+
+    def counter():
+        x = 0
+        :return @
+        def increment():
+            nonlocal x
+            x += 1
+            return x
+
+Proposal
+========
+
+This PEP proposes the addition of an optional block prefix clause to the
+syntax for function and class definitions.
+
+This block prefix would be introduced by a leading ``:`` and would be
+allowed to contain any simple statement (including those that don't
+make any sense in that context - while such code would be legal,
+there wouldn't be any point in writing it).
+
+The decorator symbol ``@`` would be repurposed inside the block prefix
+to refer to the function or class being defined.
+
+When a block prefix is provided, it *replaces* the standard local
+name binding otherwise implicit in a class or function definition.
+
+
+Background
+==========
+
+The question of "multi-line lambdas" has been a vexing one for many
+Python users for a very long time, and it took an exploration of Ruby's
+block functionality for me to finally understand why this bugs people
+so much: Python's demand that the function be named and introduced
+before the operation that needs it breaks the developer's flow of thought.
+They get to a point where they go "I need a one-shot operation that does
+<X>", and instead of being able to just *say* that, they instead have to back
+up, name a function to do <X>, then call that function from the operation
+they actually wanted to do in the first place. Lambda expressions can help
+sometimes, but they're no substitute for being able to use a full suite.
+
+Ruby's block syntax also heavily inspired the style of the solution in this
+PEP, by making it clear that even when limited to *one* anonymous function per
+statement, anonymous functions could still be incredibly useful. Consider how
+many constructs Python has where one expression is responsible for the bulk of
+the heavy lifting:
+
+  * comprehensions, generator expressions, map(), filter()
+  * key arguments to sorted(), min(), max()
+  * partial function application
+  * provision of callbacks (e.g. for weak references)
+  * array broadcast operations in NumPy
+
+However, adopting Ruby's block syntax directly won't work for Python, since
+the effectiveness of Ruby's blocks relies heavily on various conventions in
+the way functions are *defined* (specifically, Ruby's ``yield`` syntax to
+call blocks directly and the ``&arg`` mechanism to accept a block as a
+functions final argument.
+
+Since Python has relied on named functions for so long, the signatures of
+APIs that accept callbacks are far more diverse, thus requiring a solution
+that allows anonymous functions to be slotted in at the appropriate location.
+
+
+Relation to PEP 3150
+====================
+
+PEP 3150 (Statement Local Namespaces) described its primary motivation
+as being to elevate ordinary assignment statements to be on par with ``class``
+and ``def`` statements where the name of the item to be defined is presented
+to the reader in advance of the details of how the value of that item is
+calculated. This PEP achieves the same goal in a different way, by allowing
+the simple name binding of a standard function definition to be replaced
+with something else (like assigning the result of the function to a value).
+
+This PEP also achieves most of the other effects described in PEP 3150
+without introducing a new brainbending kind of scope. All of the complex
+scoping rules in PEP 3150 are replaced in this PEP with the simple ``@``
+reference to the statement local function or class definition.
+
+
+Symbol Choice
+==============
+
+The ':' symbol was chosen due to its existing presence in Python and its
+association with 'functions in expressions' via ``lambda`` expressions. The
+past Simple Implicit Lambda proposal (PEP ???) was also a factor.
+
+The proposal definitely requires *some* kind of prefix to avoid parsing
+ambiguity and backwards compatibility problems and ':' at least has the
+virtue of brevity. There's no obious alternative symbol that offers a
+clear improvement.
+
+Introducing a new keyword is another possibility, but I haven't come up
+with one that really has anything to offer over the leading colon.
+
+
+Syntax Change
+=============
+
+Current::
+
+    atom: ('(' [yield_expr|testlist_comp] ')' |
+           '[' [testlist_comp] ']' |
+           '{' [dictorsetmaker] '}' |
+           NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')
+
+Changed::
+
+    atom: ('(' [yield_expr|testlist_comp] ')' |
+           '[' [testlist_comp] ']' |
+           '{' [dictorsetmaker] '}' |
+           NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False' | '@')
+
+New::
+
+    blockprefix: ':' simple_stmt
+    block: blockprefix (decorated | classdef | funcdef)
+
+The above is the general idea, but I suspect that change to the 'atom'
+definition would cause an ambiguity problem in the parser when it comes to
+detecting decorator lines. So the actual implementation would be more complex
+than that.
+
+Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar
+
+
+Possible Implementation Strategy
+================================
+
+This proposal has one titanic advantage over PEP 3150: implementation
+should be relatively straightforward.
+
+Both the class and function definition statements emit code to perform
+the local name binding for their defined name. Implementing this PEP
+should just require intercepting that code generation and replacing
+it with the code in the block prefix.
+
+The one potentially tricky part is working out how to allow the dual
+use of '@' without rewriting half the grammar definition.
+
+More Examples
+=============
+
+Calculating attributes without polluting the local namespace (from os.py)::
+
+  # Current Python (manual namespace cleanup)
+  def _createenviron():
+      ... # 27 line function
+
+  environ = _createenviron()
+  del _createenviron
+
+  # Becomes:
+  :environ = @()
+  def _createenviron():
+      ... # 27 line function
+
+Loop early binding::
+
+  # Current Python (default argument hack)
+  funcs = [(lambda x, i=i: x + i) for i in range(10)]
+
+  # Becomes:
+  :funcs = [@(i) for i in range(10)]
+  def make_incrementor(i):
+    return lambda x: x + i
+
+  # Or even:
+  :funcs = [@(i) for i in range(10)]
+  def make_incrementor(i):
+    :return @
+    def incrementor(x):
+        return x + i
+
+
+Reference Implementation
+========================
+
+None as yet.
+
+
+TO DO
+=====
+
+Sort out links and references to everything :)
+
+
+Acknowledgements
+================
+
+Huge thanks to Gary Bernhardt for being blunt in pointing out that I had no
+idea what I was talking about in criticising Ruby's blocks, kicking off a
+rather enlightening process of investigation.
+
+
+References
+==========
+
+TBD
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End: