From 05da20448116b84cf2f3e2c171a6a22054f781b9 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Thu, 23 Feb 2012 01:22:14 +1000 Subject: [PATCH] Switch back to named functions, since the Ellipsis version degenerated badly for more complex cases. Will revisit this in the 3.4 timeframe --- pep-0403.txt | 209 ++++++++++++++++++++++++++------------------------- 1 file changed, 106 insertions(+), 103 deletions(-) diff --git a/pep-0403.txt b/pep-0403.txt index c08f7fddb..7011b9ba7 100644 --- a/pep-0403.txt +++ b/pep-0403.txt @@ -7,7 +7,7 @@ Status: Deferred Type: Standards Track Content-Type: text/x-rst Created: 2011-10-13 -Python-Version: 3.x +Python-Version: 3.4 Post-History: 2011-10-13 Resolution: TBD @@ -18,12 +18,14 @@ Abstract This PEP proposes the addition of a new ``in`` statement that accepts a statement local function or class definition. -The statement allows the use of the Ellipsis literal (``...``) to make a -forward reference to a trailing anonymous function definition. +The statement accepts a single simple statement that can make a forward +reference to a trailing function or class definition. -This new statement is designed to be used whenever a "one-shot" function is -needed, and the meaning of the function is conveyed clearly by the context -and assigning a name can actually reduce clarity rather than increasing it. +This new statement is designed to be used whenever a "one-shot" function or +class is needed, and placing the function or class definition before the +statement that uses it actually makes the code harder to read. It also +avoids any name shadowing concerns by making sure the new name is visible +only to the statement in the ``in`` clause. This PEP is based heavily on many of the ideas in PEP 3150 (Statement Local Namespaces) so some elements of the rationale will be familiar to readers of @@ -37,14 +39,14 @@ Before diving into the long history of this problem and the detailed rationale for this specific proposed solution, here are a few simple examples of the kind of code it is designed to simplify. -As a trivial example, weakref callbacks could be defined as follows:: +As a trivial example, a weakref callback could be defined as follows:: - in x = weakref.ref(target, ...) - def ...(obj): + in x = weakref.ref(target, report_destruction) + def report_destruction(obj): print("{} is being destroyed".format(obj)) -This contrasts with the current repetitive "out of order" syntax for this -operation:: +This contrasts with the current (conceptually) "out of order" syntax for +this operation:: def report_destruction(obj): print("{} is being destroyed".format(obj)) @@ -54,11 +56,19 @@ operation:: That structure is OK when you're using the callable multiple times, but it's irritating to be forced into it for one-off operations. +If the repetition of the name seems especially annoying, then a throwaway +name like ``f`` can be used instead:: + + in x = weakref.ref(target, f) + def f(obj): + print("{} is being destroyed".format(obj)) + + Similarly, a sorted operation on a particularly poorly defined type could now be defined as:: - in sorted_list = sorted(original, key=...) - def ...(item): + in sorted_list = sorted(original, key=f) + def f(item): try: return item.calc_sort_order() except NotSortableError: @@ -76,8 +86,8 @@ Rather than:: And early binding semantics in a list comprehension could be attained via:: - in funcs = [...(i) for i in range(10)] - def ...(i): + in funcs = [adder(i) for i in range(10)] + def adder(i): return lambda x: x + i @@ -88,28 +98,29 @@ This PEP proposes the addition of a new ``in`` statement that is a variant of the existing class and function definition syntax. The new ``in`` clause replaces the decorator lines, and allows forward -references to the trailing function or class definition with the ``...`` -literal syntax. +references to the trailing function or class definition. -The trailing function or class definition is always anonymous - the provide -a visual link with the forward reference, the ``...`` literal is always -given as the "name" of the class or function. +The trailing function or class definition is always named - the name of +the trailing definition is then used to make the forward reference from the +preceding statement. The ``in`` clause is allowed to contain any simple statement (including those -that don't make any sense in that context - while such code would be legal, -there wouldn't be any point in writing it). This permissive structure is -easier to define and easier to explain, but a more restrictive approach that -only permits operations that "make sense" would also be possible (see PEP -3150 for a list of possible candidates) +that don't make any sense in that context, such as ``pass`` - while such code +would be legal, there wouldn't be any point in writing it). This permissive +structure is easier to define and easier to explain, but a more restrictive +approach that only permits operations that "make sense" would also be +possible (see PEP 3150 for a list of possible candidates). -The Ellipsis literal ``...`` would be repurposed inside the ``in`` clause -to refer to the anonymous function or class being defined. The Ellipsis -builtin itself can still be accessed by name from an ``in`` clause if -necessary. +The ``in`` statement will not create a new scope - all name binding +operations aside from the trailing function or class definition will affect +the containing scope. -As functions or classes defined for an ``in`` statement are always -anonymous, local name binding takes place only if the ``in`` clause -includes an assignment. +The name used in the trailing function or class definition is only visible +from the associated ``in`` clause, and behaves as if it was an ordinary +variable defined in that scope. If any nested scopes are created in either +the ``in`` clause or the trailing function or class definition, those scopes +will see the trailing function or class definition rather than any other +bindings for that name in the containing scope. Background @@ -121,10 +132,11 @@ block functionality for me to finally understand why this bugs people so much: Python's demand that the function be named and introduced before the operation that needs it breaks the developer's flow of thought. They get to a point where they go "I need a one-shot operation that does -", and instead of being able to just *say* that, they instead have to back -up, name a function to do , then call that function from the operation -they actually wanted to do in the first place. Lambda expressions can help -sometimes, but they're no substitute for being able to use a full suite. +", and instead of being able to just *say* that directly, they instead +have to back up, name a function to do , then call that function from +the operation they actually wanted to do in the first place. Lambda +expressions can help sometimes, but they're no substitute for being able to +use a full suite. Ruby's block syntax also heavily inspired the style of the solution in this PEP, by making it clear that even when limited to *one* anonymous function per @@ -140,13 +152,19 @@ the heavy lifting: However, adopting Ruby's block syntax directly won't work for Python, since the effectiveness of Ruby's blocks relies heavily on various conventions in -the way functions are *defined* (specifically, Ruby's ``yield`` syntax to -call blocks directly and the ``&arg`` mechanism to accept a block as a +the way functions are *defined* (specifically, using Ruby's ``yield`` syntax +to call blocks directly and the ``&arg`` mechanism to accept a block as a function's final argument). Since Python has relied on named functions for so long, the signatures of APIs that accept callbacks are far more diverse, thus requiring a solution -that allows anonymous functions to be slotted in at the appropriate location. +that allows one-shot functions to be slotted in at the appropriate location. + +The approach taken in this PEP is to retain the requirement to name the +function explicitly, but allow the relative order of the definition and the +statement that references it to be changed to match the developer's flow of +thought. The rationale is essentially the same as that used when introducing +decorators, but covering a broader set of applications. Relation to PEP 3150 @@ -162,8 +180,9 @@ with something else (like assigning the result of the function to a value). This PEP also achieves most of the other effects described in PEP 3150 without introducing a new brainbending kind of scope. All of the complex -scoping rules in PEP 3150 are replaced in this PEP with a simple forward -reference to the associated function or class definition. +scoping rules in PEP 3150 are replaced in this PEP with allowing a forward +reference to the associated function or class definition without creating an +actual name binding in the current scope. Keyword Choice @@ -172,28 +191,31 @@ Keyword Choice The proposal definitely requires *some* kind of prefix to avoid parsing ambiguity and backwards compatibility problems with existing constructs. It also needs to be clearly highlighted to readers, since it declares that -the following piece of code is going to be executed out of order. +the following piece of code is going to be executed only after the trailing +function or class definition has been executed. The ``in`` keyword was chosen as an existing keyword that can be used to denote the concept of a forward reference. For functions, the construct is intended to be read as "in define "..." as this function". +that references NAME> define NAME as a function that does ". -The mapping to English prose isn't as clean for the class definition case, +The mapping to English prose isn't as obvious for the class definition case, but the concept remains the same. -Better Debugging Support for Anonymous Functions and Classes -============================================================ +Better Debugging Support for Functions and Classes with Short Names +=================================================================== One of the objections to widespread use of lambda expressions is that they have a negative effect on traceback intelligibility and other aspects of -introspection. +introspection. Similarly objections are raised regarding constructs that +promote short, cryptic function names (including this one, which requires +that the name of the trailing definition be supplied at least twice) -However, the introduction of qualified names in PEP 3155 means that -anonymous functions in different scopes will now have different -representations. For example:: +However, the introduction of qualified names in PEP 3155 means that even +anonymous classes and functions will now have different representations if +they occur in different scopes. For example:: >>> def f(): ... return lambda: y @@ -201,22 +223,17 @@ representations. For example:: >>> f() . at 0x7f6f46faeae0> -Anonymous function within the *same* scope will still share representations -(aside from the object ID), but this is still a major improvement over the -historical situation where everything *except* the object ID was identical. - -The anonymous functions and classes created by the new statement will use -the metaname ````. +Anonymous functions (or functions that share a name) within the *same* scope +will still share representations (aside from the object ID), but this is +still a major improvement over the historical situation where everything +*except* the object ID was identical. Syntax Change ============= New:: - in_stmt: in_prefix (in_classdef|in_funcdef) - in_prefix: 'in' simple_stmt - in_funcdef: 'def' '...' parameters ['->' test] ':' suite - in_classdef: 'class' '...' ['(' [arglist] ')'] ':' suite + in_stmt: 'in' simple_stmt (classdef|funcdef) Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar @@ -232,9 +249,10 @@ definition and the statement that references it, so it should just be a matter of emitting the two operations out of order and using a hidden variable to link up any references. -The one potentially tricky part is changing the meaning of the Ellipsis -literal notation while within the scope of the ``in`` clause, but that -shouldn't be too hard to address within the compiler. +The one potentially tricky part is changing the meaning of the references to +the statement local function or namespace while within the scope of the +``in`` statement, but that shouldn't be too hard to address by maintaining +some additional state within the compiler. More Examples @@ -250,8 +268,8 @@ Calculating attributes without polluting the local namespace (from os.py):: del _createenviron # Becomes: - in environ = ...() - def ...(): + in environ = _createenviron() + def _createenviron(): ... # 27 line function Loop early binding:: @@ -260,56 +278,25 @@ Loop early binding:: funcs = [(lambda x, i=i: x + i) for i in range(10)] # Becomes: - in funcs = [...(i) for i in range(10)] - def ...(i): + in funcs = [adder(i) for i in range(10)] + def adder(i): return lambda x: x + i # Or even: - in funcs = [...(i) for i in range(10)] - def ...(i): - in return ... - def ...(x): - return x + i - -Statement local namespace:: - - # OK, this definitely looks weird and needs further thought... - in c = math.sqrt(....a*....a + ....b*....b) - class ...: - a = calculate_a() - b = calculate_b() - - -Alternative Idea -================ - -As the statement local namespace example shows, using ```...`` for the -forward reference doesn't play nicely with attribute references on the -anonymous object. The doubly nested example also shows that overuse can -lead to readability disappearing in a mass of dots. - -An alternative approach would be to use a similar hidden variable -implementation strategy to implement a *single* statement local variable -for use as the forward reference. Getting the scoping right could be -challenging, but it should still be feasible. - -Then the two problematic examples could be written as:: - - in funcs = [f(i) for i in range(10)] - def f(i): + in funcs = [adder(i) for i in range(10)] + def adder(i): in return incr def incr(x): return x + i +A trailing class can be used as a statement local namespace:: + + # Evaluate subexpressions only once in c = math.sqrt(x.a*x.a + x.b*x.b) class x: a = calculate_a() b = calculate_b() -With the name not actually being bound in the local scope, it isn't -necessary to worry about name collisions, but meaningful names can still be -used to improve readability. - Reference Implementation ======================== @@ -332,6 +319,22 @@ A previous incarnation of this PEP (see [1]) proposed a much uglier syntax that (quite rightly) was not well received. The current proposal is significantly easier both to read and write. +A more recent variant always used ``...`` for forward references, along +with genuinely anonymous function and class definitions. However, this +degenerated quickly into a mass of unintelligible dots in more complex +cases:: + + in funcs = [...(i) for i in range(10)] + def ...(i): + in return ... + def ...(x): + return x + i + + in c = math.sqrt(....a*....a + ....b*....b) + class ...: + a = calculate_a() + b = calculate_b() + References ==========