From f92d0b58deece7b57eb20a4becd223f5a4ca0330 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Wed, 22 Feb 2012 01:02:52 +1000 Subject: [PATCH] PEP 403: Move this back to Deferred status. I found a couple of syntax possibilities that aren't ugly as sin like the previous version --- pep-0403.txt | 221 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 130 insertions(+), 91 deletions(-) diff --git a/pep-0403.txt b/pep-0403.txt index 8038b074b..314874652 100644 --- a/pep-0403.txt +++ b/pep-0403.txt @@ -1,9 +1,9 @@ PEP: 403 -Title: Prefix syntax for post function definition operations +Title: Forward references to anonymous functions and classes Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan -Status: Withdrawn +Status: Deferred Type: Standards Track Content-Type: text/x-rst Created: 2011-10-13 @@ -15,33 +15,19 @@ Resolution: TBD Abstract ======== -This PEP proposes the addition of ``postdef`` as a new function prefix -syntax (analogous to decorators) that permits the execution of a single simple -statement (potentially including substatements separated by semi-colons) after +This PEP proposes the addition of a new ``in`` statement that allows the use +of the Ellipsis literal (``...``) to make a forward reference to a trailing +anonymous function definition. -In addition, the new syntax would allow the 'def' keyword to be used to refer -to the function being defined without needing to repeat the name. - -When the 'postdef' prefix syntax is used, the associated statement would be -executed *in addition to* the normal local name binding implicit in function -definitions. Any name collision are expected to be minor, analagous to those -encountered with ``for`` loop iteration variables. +This new statement is designed to be used whenever a "one-shot" function is +needed, and the meaning of the function is conveyed clearly by the context +and assigning a name can actually reduce clarity rather than increasing it. This PEP is based heavily on many of the ideas in PEP 3150 (Statement Local Namespaces) so some elements of the rationale will be familiar to readers of that PEP. That PEP has now been withdrawn in favour of this one. -PEP Withdrawal -============== - -The python-ideas thread discussing this PEP [1]_ persuaded me that it was -essentially am unnecessarily cryptic, wholly inferior version of PEP 3150's -statement local namespaces. The discussion also resolved some of my concerns -with PEP 3150, so I am withdrawing this more limited version of the idea in -favour of resurrecting the original concept. - - Basic Examples ============== @@ -51,8 +37,8 @@ examples of the kind of code it is designed to simplify. As a trivial example, weakref callbacks could be defined as follows:: - postdef x = weakref.ref(target, def) - def report_destruction(obj): + in x = weakref.ref(target, ...) + def ...(obj): print("{} is being destroyed".format(obj)) This contrasts with the current repetitive "out of order" syntax for this @@ -69,8 +55,8 @@ it's irritating to be forced into it for one-off operations. Similarly, a sorted operation on a particularly poorly defined type could now be defined as:: - postdef sorted_list = sorted(original, key=def) - def force_sort(item): + in sorted_list = sorted(original, key=...) + def ...(item): try: return item.calc_sort_order() except NotSortableError: @@ -88,32 +74,40 @@ Rather than:: And early binding semantics in a list comprehension could be attained via:: - postdef funcs = [def(i) for i in range(10)] - def make_incrementor(i): - postdef return def - def incrementor(x): - return x + i + in funcs = [...(i) for i in range(10)] + def ...(i): + return lambda x: x + i Proposal ======== -This PEP proposes the addition of an optional block prefix clause to the -syntax for function and class definitions. +This PEP proposes the addition of a new ``in`` statement that is a variant +of the existing class and function definition syntax. -This block prefix would be introduced by a leading ``postdef`` and would be -allowed to contain any simple statement (including those that don't -make any sense in that context - while such code would be legal, +The new ``in`` clause replaces the decorator lines, and allows forward +references to the trailing function or class definition with the ``...`` +literal syntax. + +The trailing function or class definition is always anonymous - the provide +a visual link with the forward reference, the ``...`` literal is always +given as the "name" of the class or function. + +The ``in`` clause is allowed to contain any simple statement (including those +that don't make any sense in that context - while such code would be legal, there wouldn't be any point in writing it). This permissive structure is easier to define and easier to explain, but a more restrictive approach that only permits operations that "make sense" would also be possible (see PEP 3150 for a list of possible candidates) -The function definition keyword ``def`` would be repurposed inside the block prefix -to refer to the function being defined. +The Ellipsis literal ``...`` would be repurposed inside the ``in`` clause +to refer to the anonymous function or class being defined. The Ellipsis +builtin itself can still be accessed by name from an ``in`` clause if +necessary. -When a block prefix is provided, the standard local name binding implicit -in the function definition still takes place. +As functions or classes defined for an ``in`` statement are always +anonymous, local name binding takes place only if the ``in`` clause +includes an assignment. Background @@ -166,8 +160,8 @@ with something else (like assigning the result of the function to a value). This PEP also achieves most of the other effects described in PEP 3150 without introducing a new brainbending kind of scope. All of the complex -scoping rules in PEP 3150 are replaced in this PEP with the simple ``def`` -reference to the associated function definition. +scoping rules in PEP 3150 are replaced in this PEP with a simple forward +reference to the associated function or class definition. Keyword Choice @@ -178,51 +172,49 @@ ambiguity and backwards compatibility problems with existing constructs. It also needs to be clearly highlighted to readers, since it declares that the following piece of code is going to be executed out of order. -The 'postdef' keyword was chosen as a literal explanation of exactly what -the new clause does: execute the specified statement *after* the associated -function definition, even though it is physically written *before* the -definition in the source code. +The ``in`` keyword was chosen as an existing keyword that can be used to +denote the concept of a forward reference. + +For functions, the construct is intended to be read as "in define "..." as this function". + +The mapping to English prose isn't as clean for the class definition case, +but the concept remains the same. -Requirement to Name Functions -============================= +Better Debugging Support for Anonymous Functions and Classes +============================================================ One of the objections to widespread use of lambda expressions is that they -have an atrocious effect on traceback intelligibility and other aspects of -introspection. Accordingly, this PEP requires that even throwaway functions -be given some kind of name. +have a negative effect on traceback intelligibility and other aspects of +introspection. -To help encourage the use of meaningful names without users having to repeat -themselves, the PEP suggests the provision of the ``def`` shorthand reference -to the current function from the ``postdef`` clause. +However, the introduction of qualified names in PEP 3155 means that +anonymous functions in different scopes will now have different +representations. For example:: + >>> def f(): + ... return lambda: y + ... + >>> f() + . at 0x7f6f46faeae0> + +Anonymous function within the *same* scope will still share representations +(aside from the object ID), but this is still a major improvement over the +historical situation where everything *except* the object ID was identical. + +The anonymous functions and classes created by the new statement will use +the metaname ````. Syntax Change ============= -Current:: - - atom: ('(' [yield_expr|testlist_comp] ')' | - '[' [testlist_comp] ']' | - '{' [dictorsetmaker] '}' | - NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False') - -Changed:: - - atom: ('(' [yield_expr|testlist_comp] ')' | - '[' [testlist_comp] ']' | - '{' [dictorsetmaker] '}' | - NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False' | 'def') - New:: - blockprefix: 'postdef' simple_stmt - block: blockprefix funcdef - -The above is the general idea, but I suspect that the change to the 'atom' -definition may cause an ambiguity problem in the parser when it comes to -detecting function definitions. So the actual implementation may need to be -more complex than that. + in_stmt: in_prefix (in_classdef|in_funcdef) + in_prefix: 'in' simple_stmt + in_funcdef: 'def' '...' parameters ['->' test] ':' suite + in_classdef: 'class' '...' ['(' [arglist] ')'] ':' suite Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar @@ -230,14 +222,17 @@ Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar Possible Implementation Strategy ================================ -This proposal has one titanic advantage over PEP 3150: implementation -should be relatively straightforward. +This proposal has at least one titanic advantage over PEP 3150: +implementation should be relatively straightforward. -The post definition statement can be incorporated into the AST for the -function node and simply visited out of sequence. +The AST for the ``in`` statement will include both the function or class +definition and the statement that references it, so it should just be a +matter of emitting the two operations out of order and using a hidden +variable to link up any references. -The one potentially tricky part is working out how to allow the dual -use of 'def' without rewriting half the grammar definition. +The one potentially tricky part is changing the meaning of the Ellipsis +literal notation while within the scope of the ``in`` clause, but that +shouldn't be too hard to address within the compiler. More Examples @@ -253,8 +248,8 @@ Calculating attributes without polluting the local namespace (from os.py):: del _createenviron # Becomes: - postdef environ = def() - def _createenviron(): + in environ = ...() + def ...(): ... # 27 line function Loop early binding:: @@ -263,17 +258,56 @@ Loop early binding:: funcs = [(lambda x, i=i: x + i) for i in range(10)] # Becomes: - postdef funcs = [def(i) for i in range(10)] - def make_incrementor(i): + in funcs = [...(i) for i in range(10)] + def ...(i): return lambda x: x + i # Or even: - postdef funcs = [def(i) for i in range(10)] - def make_incrementor(i): - postdef return def - def incrementor(x): + in funcs = [...(i) for i in range(10)] + def ...(i): + in return ... + def ...(x): return x + i +Statement local namespace: + + # OK, this definitely looks weird and needs further thought... + in c = math.sqrt(....a*....a + ....b*....b) + class ...: + a = calculate_a() + b = calculate_b() + + +Alternative Idea +================ + +As the statement local namespace example shows, using ```...`` for the +forward reference doesn't play nicely with attribute references on the +anonymous object. The doubly nested example also shows that overuse can +lead to readability disappearing in a mass of dots. + +An alternative approach would be to use a similar hidden variable +implementation strategy to implement a *single* statement local variable +for use as the forward reference. Getting the scoping right could be +challenging, but it should still be feasible. + +Then the two problematic examples could be written as:: + + in funcs = [f(i) for i in range(10)] + def f(i): + in return incr + def incr(x): + return x + i + + in c = math.sqrt(x.a*x.a + x.b*x.b) + class x: + a = calculate_a() + b = calculate_b() + +With the name not actually being bound in the local scope, it isn't +necessary to worry about name collisions, but meaningful names can still be +used to improve readability. + Reference Implementation ======================== @@ -288,8 +322,13 @@ Huge thanks to Gary Bernhardt for being blunt in pointing out that I had no idea what I was talking about in criticising Ruby's blocks, kicking off a rather enlightening process of investigation. -Even though this PEP has been withdrawn, the process of writing and arguing -in its favour has been quite influential on the future direction of PEP 3150. + +Rejected Concepts +================= + +A previous incarnation of this PEP (see [1]) proposed a much uglier syntax +that (quite rightly) was not well received. The current proposal is +significantly easier both to read and write. References