Some new ideas for PEP 3150

pep-3150.txt

@@ -19,9 +19,11 @@ This PEP proposes the addition of an optional ``given`` clause to several
 Python statements that do not currently have an associated code suite. This
 clause will create a statement local namespace for additional names that are
 accessible in the associated statement, but do not become part of the
-containing namespace. To permit a sane implementation strategy, forward
+containing namespace.
-references to names from the ``given`` clause will need to be marked
+
-explicitly.
+Adoption of a new symbol, ``?``, is proposed to denote a forward reference
+to the namespace created by running the associated code suite. It will be
+a reference to a ``types.SimpleNamespace`` object.
 
 The primary motivation is to enable a more declarative style of programming,
 where the operation to be performed is presented to the reader first, and the
@@ -72,12 +74,16 @@ The ``given`` clause would allow subexpressions to be referenced by
 name in the header line, with the actual definitions following in
 the indented clause. As a simple example::
 
-   sorted_data = sorted(data, key=.sort_key) given:
+   sorted_data = sorted(data, key=?.sort_key) given:
        def sort_key(item):
            return item.attr1, item.attr2
 
-The leading ``.`` on ``.sort_key`` indicates to the compiler that this
+The new symbol ``?`` is used to refer to the given namespace. It would be a
-is a forward reference to a name defined in the ``given`` clause.
+``types.SimpleNamespace`` instance, so ``?.sort_key`` functions as
+a forward reference to a name defined in the ``given`` clause.
+
+A docstring would be permitted in the given clause, and would be attached
+to the result namespace as its ``__doc__`` attribute.
 
 The ``pass`` statement is included to provide a consistent way to skip
 inclusion of a meaningful expression in the header line. While this is not
@@ -94,7 +100,7 @@ binding operations in the header line::
    # Explicit early binding via given clause
    seq = []
    for i in range(10):
-       seq.append(.f) given i=i:
+       seq.append(.f) given i=i in:
            def f():
                return i
    assert [f() for f in seq] == list(range(10))
@@ -105,7 +111,7 @@ Semantics
 
 The following statement::
 
-   op(.f, .g) given bound_a=a, bound_b=b:
+   op(?.f, ?.g) given bound_a=a, bound_b=b in:
        def f():
            return bound_a + bound_b
        def g():
@@ -121,9 +127,10 @@ hidden compiler variable or simply an entry on the interpreter stack)::
            return bound_a + bound_b
        def g():
            return bound_a - bound_b
-      return f, g
+      return types.SimpleNamespace(**locals())
-   __ref1, __ref2 = __scope(__arg1)
+   __ref = __scope(__arg1, __arg2)
-   op(__ref1, __ref2)
+   __ref.__doc__ = __scope.__doc__
+   op(__ref.f, __ref.g)
 
 A ``given`` clause is essentially a nested function which is created and
 then immediately executed. Unless explicitly passed in, names are looked
@@ -158,7 +165,7 @@ New::
    yield_stmt: yield_expr [given_clause]
    raise_stmt: 'raise' [test ['from' test]] [given_clause]
    assert_stmt: 'assert' test [',' test] [given_clause]
-   given_clause: "given" (NAME '=' test)* ":" suite
+   given_clause: "given" [(NAME '=' test)+ "in"]":" suite
 
 (Note that ``expr_stmt`` in the grammar is a slight misnomer, as it covers
 assignment and augmented assignment in addition to simple expression
@@ -207,7 +214,7 @@ For reference, here are the current definitions at that level::
    flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
 
 In addition to the above changes, the definition of ``atom`` would be changed
-to also allow ``"." NAME``. The restriction of this usage to statements with
+to also allow ``?``. The restriction of this usage to statements with
 an associated ``given`` clause would be handled by a later stage of the
 compilation process (likely AST construction, which already enforces
 other restrictions where the grammar is overly permissive in order to
@@ -277,13 +284,14 @@ without reading the body of the suite.
 However, while they are the initial motivating use case, limiting this
 feature solely to simple assignments would be overly restrictive. Once the
 feature is defined at all, it would be quite arbitrary to prevent its use
-for augmented assignments, return statements, yield expressions and
+for augmented assignments, return statements, yield expressions,
-arbitrary expressions that may modify the application state.
+comprehensions and arbitrary expressions that may modify the
+application state.
 
 The ``given`` clause may also function as a more readable
 alternative to some uses of lambda expressions and similar
 constructs when passing one-off functions to operations
-like ``sorted()``.
+like ``sorted()`` or in callback based event-driven programming.
 
 In module and class level code, the ``given`` clause will serve as a
 clear and reliable replacement for usage of the ``del`` statement to keep
@@ -350,7 +358,7 @@ container comprehensions::
 
     # would be equivalent to
 
-    seq2 = .result given seq=seq:
+    seq2 = ?.result given seq=seq:
         result = []
         for y in seq:
             if p(y):
@@ -367,7 +375,7 @@ Not that, unlike PEP 403, the current version of this PEP *cannot*
 provide a precisely equivalent expansion for a generator expression. The
 closest it can get is to define an additional level of scoping::
 
-    seq2 = .g(seq) given:
+    seq2 = ?.g(seq) given:
         def g(seq):
             for y in seq:
                 if p(y):
@@ -375,6 +383,22 @@ closest it can get is to define an additional level of scoping::
                         if q(x):
                             yield x
 
+This limitation could be remedied by permitting the given clause to be
+a generator function, in which case ? would refer to a generator-iterator
+object rather than a simple namespace::
+
+    seq2 = ? given seq=seq in:
+        for y in seq:
+            if p(y):
+                for x in y:
+                    if q(x):
+                        yield x
+
+However, this would make the meaning of "?" quite ambiguous, even more so
+than is already the case for the meaning of ``def`` statements (which will
+usually have a docstring indicating whether or not a function definition is
+actually a generator)
+
 Explaining Decorator Clause Evaluation and Application
 ------------------------------------------------------
 
@@ -477,14 +501,19 @@ what the language allows them to express.
 I believe the proposal in this PEP would finally let Python get close to the
 "executable pseudocode" bar for the kind of thought expressed above::
 
-   sorted_list = sorted(original, key=.sort_key) given:
+   sorted_list = sorted(original, key=?.key) given:
-       def sort_key(item):
+       def key(item):
            return item.attr1, item.attr2
 
-Everything is in the same order as it was in the user's original thought, the
+Everything is in the same order as it was in the user's original thought, and
-only addition they have to make is to give the sorting criteria a name so that
+they don't even need to come up with a name for the sorting criteria: it is
-the usage can be linked up to the subsequent definition.
+possible to reuse the keyword argument name directly.
-   
+
+A possible enhancement to those proposal would be to provide a convenient
+shorthand syntax to say "use the given clause contents as keyword
+arguments". Even without dedicated syntax, that can be written simply as
+``**vars(?)``.
+
 
 Harmful to Introspection
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -516,7 +545,7 @@ world code is genuinely enhanced.
 This is more of a deficiency in the PEP rather than the idea, though. If
 it wasn't a real world problem, we wouldn't get so many complaints about
 the lack of multi-line lambda support and Ruby's block construct
-probaly wouldn't be quite so popular.
+probably wouldn't be quite so popular.
 
 
 Open Questions
@@ -525,9 +554,12 @@ Open Questions
 Syntax for Forward References
 -----------------------------
 
-The leading ``.`` arguably fails the "syntax shall not look like grit on
+The ``?`` symbol is proposed for forward references to the given namespace
-Uncle Tim's monitor" test. However, it does have the advantages of being
+as it is short, currently unused and suggests "there's something missing
-easy to type and already having an association with namespaces.
+here that will be filled in later".
+
+The proposal in the PEP doesn't neatly parallel any existing Python feature,
+so reusing an already used symbol has been deliberately avoided.
 
 
 Handling of ``nonlocal`` and ``global``
@@ -541,8 +573,8 @@ Alternatively, they could be defined as operating as if the anonymous
 functions were defined as in the expansion above.
 
 
-Detailed Semantics #3: Handling of ``break`` and ``continue``
+Handling of ``break`` and ``continue``
--------------------------------------------------------------
+--------------------------------------
 
 ``break`` and ``continue`` will operate as if the anonymous functions were
 defined as in the expansion above. They will be syntax errors if they occur
@@ -561,6 +593,25 @@ they appear within a ``def`` statement within that suite.
 Examples
 ========
 
+Defining callbacks for event driven programming::
+
+  # Current Python (definition before use)
+  def cb(sock):
+      # Do something with socket
+  def eb(exc):
+      logging.exception(
+          "Failed connecting to %s:%s", host, port)
+  loop.create_connection((host, port), cb, eb) given:
+
+  # Becomes:
+  loop.create_connection((host, port), ?.cb, ?.eb) given:
+      def cb(sock):
+          # Do something with socket
+      def eb(exc):
+          logging.exception(
+              "Failed connecting to %s:%s", host, port)
+
+
 Defining "one-off" classes which typically only have a single instance::
 
   # Current Python (instantiation after definition)
@@ -579,7 +630,7 @@ Defining "one-off" classes which typically only have a single instance::
     ... # However many lines
 
   # Becomes:
-  public_name = .MeaningfulClassName(*params) given:
+  public_name = ?.MeaningfulClassName(*params) given:
     class MeaningfulClassName():
       ... # Should trawl the stdlib for an example of doing this
 
@@ -593,7 +644,7 @@ Calculating attributes without polluting the local namespace (from os.py)::
   del _createenviron
 
   # Becomes:
-  environ = ._createenviron() given:
+  environ = ?._createenviron() given:
       def _createenviron():
         ... # 27 line function
 
@@ -606,7 +657,7 @@ Replacing default argument hack (from functools.lru_cache)::
   return decorating_function
 
   # Becomes:
-  return .decorating_function given:
+  return ?.decorating_function given:
     # Cell variables rather than locals, but should give similar speedup
     tuple, sorted, len, KeyError = tuple, sorted, len, KeyError
     def decorating_function(user_function):
@@ -701,6 +752,9 @@ References
 .. [9] Possible PEP 3150 style guidelines (#2):
    http://mail.python.org/pipermail/python-ideas/2011-October/012341.html
 
+.. [10] Multi-line lambdas (again!)
+   http://mail.python.org/pipermail/python-ideas/2013-August/022526.html
+
 Copyright
 =========