PEP 572: TargetScopeError->SyntaxError, other SyntaxError adjustments (#1140)

- switch from the new `TargetScopeError` to a plain `SyntaxError`
  as per referenced python-dev discussion
- clarify rationale for those extra SyntaxError cases (we don't
  want current CPython implementation details to implicitly leak into
  the "do what CPython does" de facto language specification)
- broaden one of the error cases to handle the fact that CPython's
  symbol table analysis for a comprehension involves two different
  scopes and hence makes it difficult to detect when the target of
  a named expression in the iterable expression gets re-used as an
  iteration variable in the comprehension
- note that even dead code affects the symbol analysis pass

pep-0572.rst

@@ -8,7 +8,7 @@ Content-Type: text/x-rst
 Created: 28-Feb-2018
 Python-Version: 3.8
 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018,
-              25-Apr-2018, 09-Jul-2018
+              25-Apr-2018, 09-Jul-2018, 05-Aug-2019
 Resolution: https://mail.python.org/pipermail/python-dev/2018-July/154601.html
 
 
@@ -16,9 +16,18 @@ Abstract
 ========
 
 This is a proposal for creating a way to assign to variables within an
-expression using the notation ``NAME := expr``. A new exception,
+expression using the notation ``NAME := expr``.
-``TargetScopeError`` is added, and there is one change to evaluation
+
-order.
+As part of this change, there is also an update to dictionary comprehension
+evaluation order to ensure key expressions are executed before value
+expressions (allowing the key to be bound to a name and then re-used as part of
+calculating the corresponding value).
+
+During discussion of this PEP, the operator became informally known as
+"the walrus operator". The construct's formal name is "Assignment Expressions"
+(as per the PEP title), but they may also be referred to as "Named Expressions"
+(e.g. the CPython reference implementation uses that name internally).
+
 
 Rationale
 =========
@@ -275,16 +284,50 @@ i`` part establishes that ``i`` is local to the comprehension, but the
 ``i :=`` part insists that ``i`` is not local to the comprehension.
 The same reason makes these examples invalid too::
 
-    [[(j := j) for i in range(5)] for j in range(5)]
+    [[(j := j) for i in range(5)] for j in range(5)] # INVALID
-    [i := 0 for i, j in stuff]
+    [i := 0 for i, j in stuff]                       # INVALID
-    [i+1 for i in i := stuff]
+    [i+1 for i in (i := stuff)]                      # INVALID
+
+While it's technically possible to assign consistent semantics to these cases,
+it's difficult to determine whether those semantics actually make *sense* in the
+absence of real use cases. Accordingly, the reference implementation will ensure
+that such cases raise ``SyntaxError``, rather than executing with implementation
+defined behaviour.
+
+This restriction applies even if the assignment expression is never executed::
+
+    [False and (i := 0) for i, j in stuff]     # INVALID
+    [i for i, j in stuff if True or (j := 1)]  # INVALID
+
+For the comprehension body (the part before the first "for" keyword) and the
+filter expression (the part after "if" and before any nested "for"), this
+restriction applies solely to target names that are also used as iteration
+variables in the comprehension. Lambda expressions appearing in these
+positions introduce a new explicit function scope, and hence may use assignment
+expressions with no additional restrictions.
+
+Due to design constraints in the reference implementation (the symbol table
+analyser cannot easily detect when names are re-used between the leftmost
+comprehension iterable expression and the rest of the comprehension), named
+expressions are disallowed entirely as part of comprehension iterable
+expressions (the part after each "in", and before any subsequent "if" or
+"for" keyword)::
+
+    [i+1 for i in (j := stuff)]                    # INVALID
+    [i+1 for i in range(2) for j in (k := stuff)]  # INVALID
+    [i+1 for i in [j for j in (k := stuff)]]       # INVALID
+    [i+1 for i in (lambda: (j := stuff))()]        # INVALID
 
 A further exception applies when an assignment expression occurs in a
 comprehension whose containing scope is a class scope.  If the rules
 above were to result in the target being assigned in that class's
-scope, the assignment expression is expressly invalid.
+scope, the assignment expression is expressly invalid. This case also raises
+``SyntaxError``::
 
-(The reason for the latter exception is the implicit function created
+    class Example:
+        [(j := i) for i in range(5)]  # INVALID
+
+(The reason for the latter exception is the implicit function scope created
 for comprehensions -- there is currently no runtime mechanism for a
 function to refer to a variable in the containing class scope, and we
 do not want to add such a mechanism.  If this issue ever gets resolved
@@ -295,8 +338,6 @@ variable defined in the class scope from a comprehension.)
 See Appendix B for some examples of how the rules for targets in
 comprehensions translate to equivalent code.
 
-The two invalid cases listed above raise ``TargetScopeError``, a
-new subclass of ``SyntaxError`` (with the same signature).
 
 Relative precedence of ``:=``
 -----------------------------
@@ -413,6 +454,26 @@ found in assignment statements:
     total += tax  # Equivalent: (total := total + tax)
 
 
+Specification changes during implementation
+===========================================
+
+The following changes have been made based on implementation experience and
+additional review after the PEP was first accepted and before Python 3.8 was
+released:
+
+* for consistency with other similar exceptions, and to avoid locking in an
+  exception name that is not necessarily going to improve clarity for end users,
+  the originally proposed ``TargetScopeError`` subclass of ``SyntaxError`` was
+  dropped in favour of just raising ``SyntaxError`` directly. [3]_
+* due to a limitation in CPython's symbol table analysis process, the reference
+  implementation raises ``SyntaxError`` for all uses of named expressions inside
+  comprehension iterable expressions, rather than only raising them when the
+  named expression target conflicts with one of the iteration variables in the
+  comprehension. This could be revisited given sufficiently compelling examples,
+  but the extra complexity needed to implement the more selective restriction
+  doesn't seem worthwhile for purely hypothetical use cases.
+
+
 Examples
 ========
 
@@ -1257,6 +1318,8 @@ References
    (https://github.com/Rosuav/cpython/tree/assignment-expressions)
 .. [2] Pivotal post regarding inline assignment semantics
    (https://mail.python.org/pipermail/python-ideas/2018-March/049409.html)
+.. [3] Discussion of PEP 572 TargetScopeError
+   (https://mail.python.org/archives/list/python-dev@python.org/thread/FXVSYCTQOTT7JCFACKPGPXKULBCGEPQY/)
 
 
 Copyright