834 lines
33 KiB
ReStructuredText
834 lines
33 KiB
ReStructuredText
PEP: 577
|
||
Title: Augmented Assignment Expressions
|
||
Author: Nick Coghlan <ncoghlan@gmail.com>
|
||
Status: Withdrawn
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 14-May-2018
|
||
Python-Version: 3.8
|
||
Post-History: 22-May-2018
|
||
|
||
|
||
PEP Withdrawal
|
||
==============
|
||
|
||
While working on this PEP, I realised that it didn't really address what was
|
||
actually bothering me about PEP 572's proposed scoping rules for previously
|
||
unreferenced assignment targets, and also had some significant undesirable
|
||
consequences (most notably, allowing ``>>=` and ``<<=`` as inline augmented
|
||
assignment operators that meant something entirely different from the ``>=``
|
||
and ``<=`` comparison operators).
|
||
|
||
I also realised that even without dedicated syntax of their own, PEP 572
|
||
technically allows inline augmented assignments to be written using the
|
||
``operator`` module::
|
||
|
||
from operator import iadd
|
||
if (target := iadd(target, value)) < limit:
|
||
...
|
||
|
||
The restriction to simple names as inline assignment targets means that the
|
||
target expression can always be repeated without side effects, and thus avoids
|
||
the ambiguity that would arise from allowing actual embedded augmented
|
||
assignments (it's still a bad idea, since it would almost certainly be hard
|
||
for humans to read, this note is just about the theoretical limits of language
|
||
level expressiveness).
|
||
|
||
Accordingly, I withdrew this PEP without submitting it for pronouncement. At
|
||
the time I also started writing a replacement PEP that focused specifically on
|
||
the handling of assignment targets which hadn't already been declared as local
|
||
variables in the current scope (for both regular block scopes, and for scoped
|
||
expressions), but that draft never even reached a stage where *I* liked it
|
||
better than the ultimately accepted proposal in PEP 572, so it was never
|
||
posted anywhere, nor assigned a PEP number.
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
This is a proposal to allow augmented assignments such as ``x += 1`` to be
|
||
used as expressions, not just statements.
|
||
|
||
As part of this, ``NAME := EXPR`` is proposed as an inline assignment expression
|
||
that uses the new augmented assignment scoping rules, rather than implicitly
|
||
defining a new local variable name the way that existing name binding
|
||
statements do. The question of allowing expression level local variable
|
||
declarations at function scope is deliberately separated from the question of
|
||
allowing expression level name bindings, and deferred to a later PEP.
|
||
|
||
This PEP is a direct competitor to PEP 572 (although it borrows heavily from that
|
||
PEP's motivation, and even shares the proposed syntax for inline assignments).
|
||
See `Relationship with PEP 572`_ for more details on the connections between
|
||
the two PEPs.
|
||
|
||
To improve the usability of the new expressions, a semantic split is proposed
|
||
between the handling of augmented assignments in regular block scopes (modules,
|
||
classes, and functions), and the handling of augmented assignments in scoped
|
||
expressions (lambda expressions, generator expressions, and comprehensions),
|
||
such that all inline assignments default to targeting the nearest containing
|
||
block scope.
|
||
|
||
A new compile time ``TargetNameError`` is added as a subclass of ``SyntaxError``
|
||
to handle cases where it is deemed to be currently unclear which target is
|
||
expected to be rebound by an inline assignment, or else the target scope
|
||
for the inline assignment is considered invalid for another reason.
|
||
|
||
|
||
Syntax and semantics
|
||
====================
|
||
|
||
Augmented assignment expressions
|
||
--------------------------------
|
||
|
||
The language grammar would be adjusted to allow augmented assignments to
|
||
appear as expressions, where the result of the augmented assignment
|
||
expression is the same post-calculation reference as is being bound to the
|
||
given target.
|
||
|
||
For example::
|
||
|
||
>>> n = 0
|
||
>>> n += 5
|
||
5
|
||
>>> n -= 2
|
||
3
|
||
>>> n *= 3
|
||
9
|
||
>>> n
|
||
9
|
||
|
||
For mutable targets, this means the result is always just the original object::
|
||
|
||
>>> seq = []
|
||
>>> seq_id = id(seq)
|
||
>>> seq += range(3)
|
||
[0, 1, 2]
|
||
>>> seq_id == id(seq)
|
||
True
|
||
|
||
Augmented assignments to attributes and container subscripts will be permitted,
|
||
with the result being the post-calculation reference being bound to the target,
|
||
just as it is for simple name targets::
|
||
|
||
def increment(self, step=1):
|
||
return self._value += step
|
||
|
||
In these cases, ``__getitem__`` and ``__getattribute__`` will *not* be called
|
||
after the assignment has already taken place (they will only be called as
|
||
needed to evaluate the in-place operation).
|
||
|
||
|
||
Adding an inline assignment operator
|
||
------------------------------------
|
||
|
||
Given only the addition of augmented assignment expressions, it would be
|
||
possible to abuse a symbol like ``|=`` as a general purpose assignment
|
||
operator by defining a ``Target`` wrapper type that worked as follows::
|
||
|
||
>>> class Target:
|
||
... def __init__(self, value):
|
||
... self.value = value
|
||
... def __or__(self, other):
|
||
... return Target(other)
|
||
...
|
||
>>> x = Target(10)
|
||
>>> x.value
|
||
10
|
||
>>> x |= 42
|
||
<__main__.Target object at 0x7f608caa8048>
|
||
>>> x.value
|
||
42
|
||
|
||
This is similar to the way that storing a single reference in a list was long
|
||
used as a workaround for the lack of a ``nonlocal`` keyword, and can still be
|
||
used today (in combination with ``operator.itemsetter``) to work around the
|
||
lack of expression level assignments.
|
||
|
||
Rather than requiring such workarounds, this PEP instead proposes that
|
||
PEP 572's "NAME := EXPR" syntax be adopted as a new inline assignment
|
||
expression that uses the augmented assignment scoping rules described below.
|
||
|
||
This cleanly handles cases where only the new value is of interest, and the
|
||
previously bound value (if any) can just be discarded completely.
|
||
|
||
Note that for both simple names and complex assignment targets, the inline
|
||
assignment operator does *not* read the previous reference before assigning
|
||
the new one. However, when used at function scope (either directly or inside
|
||
a scoped expression), it does *not* implicitly define a new local variable,
|
||
and will instead raise ``TargetNameError`` (as described for augmented
|
||
assignments below).
|
||
|
||
|
||
Assignment operator precedence
|
||
------------------------------
|
||
|
||
To preserve the existing semantics of augmented assignment statements,
|
||
inline assignment operators will be defined as being of lower precedence
|
||
than all other operators, include the comma pseudo-operator. This ensures
|
||
that when used as a top level expression the entire right hand side of the
|
||
expression is still interpreted as the value to be processed (even when that
|
||
value is a tuple without parentheses).
|
||
|
||
The difference this introduces relative to PEP 572 is that where
|
||
``(n := first, second)`` sets ``n = first`` in PEP 572, in this PEP it would set
|
||
``n = (first, second)`, and getting the first meaning would require an extra
|
||
set of parentheses (``((n := first), second)``).
|
||
|
||
PEP 572 quite reasonably notes that this results in ambiguity when assignment
|
||
expressions are used as function call arguments. This PEP resolves that concern
|
||
a different way by requiring that assignment expressions be parenthesised
|
||
when used as arguments to a function call (unless they're the sole argument).
|
||
|
||
This is a more relaxed version of the restriction placed on generator
|
||
expressions (which always require parentheses, except when they're the sole
|
||
argument to a function call).
|
||
|
||
|
||
Augmented assignment to names in block scopes
|
||
---------------------------------------------
|
||
|
||
No target name binding changes are proposed for augmented assignments at module
|
||
or class scope (this also includes code executed using "exec" or "eval"). These
|
||
will continue to implicitly declare a new local variable as the binding target
|
||
as they do today, and (if necessary) will be able to resolve the name from an
|
||
outer scope before binding it locally.
|
||
|
||
At function scope, augmented assignments will be changed to require that there
|
||
be either a preceding name binding or variable declaration to explicitly
|
||
establish the target name as being local to the function, or else an explicit
|
||
``global`` or ``nonlocal`` declaration. ``TargetNameError``, a new
|
||
``SyntaxError`` subclass, will be raised at compile time if no such binding or
|
||
declaration is present.
|
||
|
||
For example, the following code would compile and run as it does today::
|
||
|
||
x = 0
|
||
x += 1 # Sets global "x" to 1
|
||
|
||
class C:
|
||
x += 1 # Sets local "x" to 2, leaves global "x" alone
|
||
|
||
def local_target():
|
||
x = 0
|
||
x += 1 # Sets local "x" to 1, leaves global "x" alone
|
||
|
||
def global_target():
|
||
global x
|
||
x += 1 # Increments global "x" each time this runs
|
||
|
||
def nonlocal_target():
|
||
x = 0
|
||
def g():
|
||
nonlocal x
|
||
x += 1 # Increments "x" in outer scope each time this runs
|
||
return x
|
||
return g
|
||
|
||
The follow examples would all still compile and then raise an error at runtime
|
||
as they do today::
|
||
|
||
n += 1 # Raises NameError at runtime
|
||
|
||
class C:
|
||
n += 1 # Raises NameError at runtime
|
||
|
||
def missing_global():
|
||
global n
|
||
n += 1 # Raises NameError at runtime
|
||
|
||
def delayed_nonlocal_initialisation():
|
||
def f():
|
||
nonlocal n
|
||
n += 1
|
||
f() # Raises NameError at runtime
|
||
n = 0
|
||
|
||
def skipped_conditional_initialisation():
|
||
if False:
|
||
n = 0
|
||
n += 1 # Raises UnboundLocalError at runtime
|
||
|
||
def local_declaration_without_initial_assignment():
|
||
n: typing.Any
|
||
n += 1 # Raises UnboundLocalError at runtime
|
||
|
||
Whereas the following would raise a compile time ``DeprecationWarning``
|
||
initially, and eventually change to report a compile time ``TargetNameError``::
|
||
|
||
def missing_target():
|
||
x += 1 # Compile time TargetNameError due to ambiguous target scope
|
||
# Is there a missing initialisation of "x" here? Or a missing
|
||
# global or nonlocal declaration?
|
||
|
||
As a conservative implementation approach, the compile time function name
|
||
resolution change would be introduced as a ``DeprecationWarning`` in Python
|
||
3.8, and then converted to ``TargetNameError`` in Python 3.9. This avoids
|
||
potential problems in cases where an unused function would currently raise
|
||
``UnboundLocalError`` if it was ever actually called, but the code is actually
|
||
unused - converting that latent runtime defect to a compile time error qualifies
|
||
as a backwards incompatible change that requires a deprecation period.
|
||
|
||
When augmented assignments are used as expressions in function scope (rather
|
||
than as standalone statements), there aren't any backwards compatibility
|
||
concerns, so the compile time name binding checks would be enforced immediately
|
||
in Python 3.8.
|
||
|
||
Similarly, the new inline assignment expressions would always require explicit
|
||
predeclaration of their target scope when used as part of a function, at least
|
||
for Python 3.8. (See the design discussion section for notes on potentially
|
||
revisiting that restriction in the future).
|
||
|
||
|
||
Augmented assignment to names in scoped expressions
|
||
---------------------------------------------------
|
||
|
||
Scoped expressions is a new collective term being proposed for expressions that
|
||
introduce a new nested scope of execution, either as an intrinsic part of their
|
||
operation (lambda expressions, generator expressions), or else as a way of
|
||
hiding name binding operations from the containing scope (container
|
||
comprehensions).
|
||
|
||
Unlike regular functions, these scoped expressions can't include explicit
|
||
``global`` or ``nonlocal`` declarations to rebind names directly in an outer
|
||
scope.
|
||
|
||
Instead, their name binding semantics for augmented assignment expressions would
|
||
be defined as follows:
|
||
|
||
* augmented assignment targets used in scoped expressions are expected to either
|
||
be already bound in the containing block scope, or else have their scope
|
||
explicitly declared in the containing block scope. If no suitable name
|
||
binding or declaration can be found in that scope, then ``TargetNameError``
|
||
will be raised at compile time (rather than creating a new binding within
|
||
the scoped expression).
|
||
* if the containing block scope is a function scope, and the target name is
|
||
explicitly declared as ``global`` or ``nonlocal``, then it will be use the
|
||
same scope declaration in the body of the scoped expression
|
||
* if the containing block scope is a function scope, and the target name is
|
||
a local variable in that function, then it will be implicitly declared as
|
||
``nonlocal`` in the body of the scoped expression
|
||
* if the containing block scope is a class scope, than ``TargetNameError`` will
|
||
always be raised, with a dedicated message indicating that combining class
|
||
scopes with augmented assignments in scoped expressions is not currently
|
||
permitted.
|
||
* if a name is declared as a formal parameter (lambda expressions), or as an
|
||
iteration variable (generator expressions, comprehensions), then that name
|
||
is considered local to that scoped expression, and attempting to use it as
|
||
the target of an augmented assignment operation in that scope, or any nested
|
||
scoped expression, will raise ``TargetNameError`` (this is a restriction that
|
||
could potentially be lifted later, but is being proposed for now to simplify
|
||
the initial set of compile time and runtime semantics that needs to be
|
||
covered in the language reference and handled by the compiler and interpreter)
|
||
|
||
For example, the following code would work as shown::
|
||
|
||
>>> global_target = 0
|
||
>>> incr_global_target = lambda: global_target += 1
|
||
>>> incr_global_target()
|
||
1
|
||
>>> incr_global_target()
|
||
2
|
||
>>> global_target
|
||
2
|
||
>>> def cumulative_sums(data, start=0)
|
||
... total = start
|
||
... yield from (total += value for value in data)
|
||
... return total
|
||
...
|
||
>>> print(list(cumulative_sums(range(5))))
|
||
[0, 1, 3, 6, 10]
|
||
|
||
While the following examples would all raise ``TargetNameError``::
|
||
|
||
class C:
|
||
cls_target = 0
|
||
incr_cls_target = lambda: cls_target += 1 # Error due to class scope
|
||
|
||
def missing_target():
|
||
incr_x = lambda: x += 1 # Error due to missing target "x"
|
||
|
||
def late_target():
|
||
incr_x = lambda: x += 1 # Error due to "x" being declared after use
|
||
x = 1
|
||
|
||
lambda arg: arg += 1 # Error due to attempt to target formal parameter
|
||
|
||
[x += 1 for x in data] # Error due to attempt to target iteration variable
|
||
|
||
|
||
As augmented assignments currently can't appear inside scoped expressions, the
|
||
above compile time name resolution exceptions would be included as part of the
|
||
initial implementation rather than needing to be phased in as a potentially
|
||
backwards incompatible change.
|
||
|
||
|
||
Design discussion
|
||
=================
|
||
|
||
Allowing complex assignment targets
|
||
-----------------------------------
|
||
|
||
The initial drafts of this PEP kept PEP 572's restriction to single name targets
|
||
when augmented assignments were used as expressions, allowing attribute and
|
||
subscript targets solely for the statement form.
|
||
|
||
However, enforcing that required varying the permitted targets based on whether
|
||
or not the augmented assignment was a top level expression or not, as well as
|
||
explaining why ``n += 1``, ``(n += 1)``, and ``self.n += 1`` were all legal,
|
||
but ``(self.n += 1)`` was prohibited, so the proposal was simplified to allow
|
||
all existing augmented assignment targets for the expression form as well.
|
||
|
||
Since this PEP defines ``TARGET := EXPR`` as a variant on augmented assignment,
|
||
that also gained support for assignment and subscript targets.
|
||
|
||
|
||
Augmented assignment or name binding only?
|
||
------------------------------------------
|
||
|
||
PEP 572 makes a reasonable case that the potential use cases for inline
|
||
augmented assignment are notably weaker than those for inline assignment in
|
||
general, so it's acceptable to require that they be spelled as ``x := x + 1``,
|
||
bypassing any in-place augmented assignment methods.
|
||
|
||
While this is at least arguably true for the builtin types (where potential
|
||
counterexamples would probably need to focus on set manipulation use cases
|
||
that the PEP author doesn't personally have), it would also rule out more
|
||
memory intensive use cases like manipulation of NumPy arrays, where the data
|
||
copying involved in out-of-place operations can make them impractical as
|
||
alternatives to their in-place counterparts.
|
||
|
||
That said, this PEP mainly exists because the PEP author found the inline
|
||
assignment proposal much easier to grasp as "It's like ``+=``, only skipping
|
||
the addition step", and also liked the way that that framing provides an
|
||
actual semantic difference between ``NAME = EXPR`` and ``NAME := EXPR`` at
|
||
function scope.
|
||
|
||
That difference in target scoping behaviour means that the ``NAME := EXPR``
|
||
syntax would be expected to have two primary use cases:
|
||
|
||
* as a way of allowing assignments to be embedded as an expression in an ``if``
|
||
or ``while`` statement, or as part of a scoped expression
|
||
* as a way of requesting a compile time check that the target name be previously
|
||
declared or bound in the current function scope
|
||
|
||
At module or class scope, ``NAME = EXPR`` and ``NAME := EXPR`` would be
|
||
semantically equivalent due to the compiler's lack of visibility into the set
|
||
of names that will be resolvable at runtime, but code linters and static
|
||
type checkers would be encouraged to enforce the same "declaration or assignment
|
||
required before use" behaviour for ``NAME := EXPR`` as the compiler would
|
||
enforce at function scope.
|
||
|
||
|
||
Postponing a decision on expression level target declarations
|
||
-------------------------------------------------------------
|
||
|
||
At least for Python 3.8, usage of inline assignments (whether augmented or not)
|
||
at function scope would always require a preceding name binding or scope
|
||
declaration to avoid getting ``TargetNameError``, even when used outside a
|
||
scoped expression.
|
||
|
||
The intent behind this requirement is to clearly separate the following two
|
||
language design questions:
|
||
|
||
1. Can an expression rebind a name in the current scope?
|
||
2. Can an expression declare a new name in the current scope?
|
||
|
||
For module global scopes, the answer to both of those questions is unequivocally
|
||
"Yes", because it's a language level guarantee that mutating the ``globals()``
|
||
dict will immediately impact the runtime module scope, and ``global NAME``
|
||
declarations inside a function can have the same effect (as can importing the
|
||
currently executing module and modifying its attributes).
|
||
|
||
For class scopes, the answer to both questions is also "Yes" in practice,
|
||
although less unequivocally so, since the semantics of ``locals()`` are
|
||
currently formally unspecified. However, if the current behaviour of ``locals()``
|
||
at class scope is taken as normative (as PEP 558 proposes), then this is
|
||
essentially the same scenario as manipulating the module globals, just using
|
||
``locals()`` instead.
|
||
|
||
For function scopes, however, the current answers to these two questions are
|
||
respectively "Yes" and "No". Expression level rebinding of function locals is
|
||
already possible thanks to lexically nested scopes and explicit ``nonlocal NAME``
|
||
expressions. While this PEP will likely make expression level rebinding more
|
||
common than it is today, it isn't a fundamentally new concept for the language.
|
||
|
||
By contrast, declaring a *new* function local variable is currently a statement
|
||
level action, involving one of:
|
||
|
||
* an assignment statement (``NAME = EXPR``, ``OTHER_TARGET = NAME = EXPR``, etc)
|
||
* a variable declaration (``NAME : EXPR``)
|
||
* a nested function definition
|
||
* a nested class definition
|
||
* a ``for`` loop
|
||
* a ``with`` statement
|
||
* an ``except`` clause (with limited scope of access)
|
||
|
||
The historical trend for the language has actually been to *remove* support for
|
||
expression level declarations of function local names, first with the
|
||
introduction of "fast locals" semantics (which made the introduction of names
|
||
via ``locals()`` unsupported for function scopes), and again with the hiding
|
||
of comprehension iteration variables in Python 3.0.
|
||
|
||
Now, it may be that in Python 3.9, we decide to revisit this question based on
|
||
our experience with expression level name binding in Python 3.8, and decide that
|
||
we really do want expression level function local variable declarations as well,
|
||
and that we want ``NAME := EXPR`` to be the way we spell that (rather than,
|
||
for example, spelling inline declarations more explicitly as
|
||
``NAME := EXPR given NAME``, which would permit them to carry type annotations,
|
||
and also permit them to declare new local variables in scoped expressions,
|
||
rather than having to pollute the namespace in their containing scope).
|
||
|
||
But the proposal in this PEP is that we explicitly give ourselves a full
|
||
release to decide how much we want that feature, and exactly where we find
|
||
its absence irritating. Python has survived happily without expression level
|
||
name bindings *or* declarations for decades, so we can afford to give ourselves
|
||
a couple of years to decide if we really want *both* of those, or if expression
|
||
level bindings are sufficient.
|
||
|
||
|
||
Ignoring scoped expressions when determining augmented assignment targets
|
||
-------------------------------------------------------------------------
|
||
|
||
When discussing possible binding semantics for PEP 572's assignment expressions,
|
||
Tim Peters made a plausible case [1_,2_,3_] for assignment expressions targeting
|
||
the containing block scope, essentially ignoring any intervening scoped
|
||
expressions.
|
||
|
||
This approach allows use cases like cumulative sums, or extracting the final
|
||
value from a generator expression to be written in a relatively straightforward
|
||
way::
|
||
|
||
total = 0
|
||
partial_sums = [total := total + value for value in data]
|
||
|
||
factor = 1
|
||
while any(n % (factor := p) == 0 for p in small_primes):
|
||
n //= factor
|
||
|
||
Guido also expressed his approval for this general approach [4_].
|
||
|
||
The proposal in this PEP differs from Tim's original proposal in three main
|
||
areas:
|
||
|
||
- it applies the proposal to all augmented assignment operators, not just a
|
||
single new name binding operator
|
||
- as far as is practical, it extends the augmented assignment requirement that
|
||
the name already be defined to the new name binding operator (raising
|
||
``TargetNameError`` rather than implicitly declaring new local variables at
|
||
function scope)
|
||
- it includes lambda expressions in the set of scopes that get ignored for
|
||
target name binding purposes, making this transparency to assignments common
|
||
to all of the scoped expressions rather than being specific to comprehensions
|
||
and generator expressions
|
||
|
||
With scoped expressions being ignored when calculating binding targets, it's
|
||
once again difficult to detect the scoping difference between the outermost
|
||
iterable expressions in generator expressions and comprehensions (you have to
|
||
mess about with either class scopes or attempting to rebind iteration Variables
|
||
to detect it), so there's also no need to tinker with that.
|
||
|
||
|
||
Treating inline assignment as an augmented assignment variant
|
||
-------------------------------------------------------------
|
||
|
||
One of the challenges with PEP 572 is the fact that ``NAME = EXPR`` and
|
||
``NAME := EXPR`` are entirely semantically equivalent at every scope. This
|
||
makes the two forms hard to teach, since there's no inherent nudge towards
|
||
choosing one over the other at the statement level, so you end up having to
|
||
resort to "``NAME = EXPR`` is preferred because it's been around longer"
|
||
(and PEP 572 proposes to enfore that historical idiosyncrasy at the compiler
|
||
level).
|
||
|
||
That semantic equivalence is difficult to avoid at module and class scope while
|
||
still having ``if NAME := EXPR:`` and ``while NAME := EXPR:`` work sensibly, but
|
||
at function scope the compiler's comprehensive view of all local names makes
|
||
it possible to require that the name be assigned or declared before use,
|
||
providing a reasonable incentive to continue to default to using the
|
||
``NAME = EXPR`` form when possible, while also enabling the use of the
|
||
``NAME := EXPR`` as a kind of simple compile time assertion (i.e. explicitly
|
||
indicating that the targeted name has already been bound or declared and hence
|
||
should already be known to the compiler).
|
||
|
||
If Guido were to declare that support for inline declarations was a hard
|
||
design requirement, then this PEP would be updated to propose that
|
||
``EXPR given NAME`` also be introduced as a way to support inline name declarations
|
||
after arbitrary expressions (this would allow the inline name declarations to be
|
||
deferred until the end of a complex expression rather than needing to be
|
||
embedded in the middle of it, and PEP 8 would gain a recommendation encouraging
|
||
that style).
|
||
|
||
|
||
Disallowing augmented assignments in class level scoped expressions
|
||
-------------------------------------------------------------------
|
||
|
||
While modern classes do define an implicit closure that's visible to method
|
||
implementations (in order to make ``__class__`` available for use in zero-arg
|
||
``super()`` calls), there's no way for user level code to explicitly add
|
||
additional names to that scope.
|
||
|
||
Meanwhile, attributes defined in a class body are ignored for the purpose of
|
||
defining a method's lexical closure, which means adding them there wouldn't
|
||
work at an implementation level.
|
||
|
||
Rather than trying to resolve that inherent ambiguity, this PEP simply
|
||
prohibits such usage, and requires that any affected logic be written somewhere
|
||
other than directly inline in the class body (e.g. in a separate helper
|
||
function).
|
||
|
||
|
||
Comparison operators vs assignment operators
|
||
--------------------------------------------
|
||
|
||
The ``OP=`` construct as an expression currently indicates a comparison
|
||
operation::
|
||
|
||
x == y # Equals
|
||
x >= y # Greater-than-or-equal-to
|
||
x <= y # Less-than-or-equal-to
|
||
|
||
Both this PEP and PEP 572 propose adding at least one operator that's somewhat
|
||
similar in appearance, but defines an assignment instead::
|
||
|
||
x := y # Becomes
|
||
|
||
This PEP then goes much further and allows all *13* augmented assignment symbols
|
||
to be uses as binary operators::
|
||
|
||
x += y # In-place add
|
||
x -= y # In-place minus
|
||
x *= y # In-place multiply
|
||
x @= y # In-place matrix multiply
|
||
x /= y # In-place division
|
||
x //= y # In-place int division
|
||
x %= y # In-place mod
|
||
x &= y # In-place bitwise and
|
||
x |= y # In-place bitwise or
|
||
x ^= y # In-place bitwise xor
|
||
x <<= y # In-place left shift
|
||
x >>= y # In-place right shift
|
||
x **= y # In-place power
|
||
|
||
Of those additional binary operators, the most questionable would be the
|
||
bitshift assignment operators, since they're each only one doubled character
|
||
away from one of the inclusive ordered comparison operators.
|
||
|
||
|
||
Examples
|
||
========
|
||
|
||
Simplifying retry loops
|
||
-----------------------
|
||
|
||
There are currently a few different options for writing retry loops, including::
|
||
|
||
# Post-decrementing a counter
|
||
remaining_attempts = MAX_ATTEMPTS
|
||
while remaining_attempts:
|
||
remaining_attempts -= 1
|
||
try:
|
||
result = attempt_operation()
|
||
except Exception as exc:
|
||
continue # Failed, so try again
|
||
log.debug(f"Succeeded after {attempts} attempts")
|
||
break # Success!
|
||
else:
|
||
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
||
|
||
# Loop-and-a-half with a pre-incremented counter
|
||
attempt = 0
|
||
while True:
|
||
attempts += 1
|
||
if attempts > MAX_ATTEMPTS:
|
||
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
||
try:
|
||
result = attempt_operation()
|
||
except Exception as exc:
|
||
continue # Failed, so try again
|
||
log.debug(f"Succeeded after {attempts} attempts")
|
||
break # Success!
|
||
|
||
Each of the available options hides some aspect of the intended loop structure
|
||
inside the loop body, whether that's the state modification, the exit condition,
|
||
or both.
|
||
|
||
The proposal in this PEP allows both the state modification and the exit
|
||
condition to be included directly in the loop header::
|
||
|
||
attempt = 0
|
||
while (attempt += 1) <= MAX_ATTEMPTS:
|
||
try:
|
||
result = attempt_operation()
|
||
except Exception as exc:
|
||
continue # Failed, so try again
|
||
log.debug(f"Succeeded after {attempts} attempts")
|
||
break # Success!
|
||
else:
|
||
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
||
|
||
|
||
Simplifying if-elif chains
|
||
--------------------------
|
||
|
||
if-elif chains that need to rebind the checked condition currently need to
|
||
be written using nested if-else statements::
|
||
|
||
|
||
m = pattern.match(data)
|
||
if m:
|
||
...
|
||
else:
|
||
m = other_pattern.match(data)
|
||
if m:
|
||
...
|
||
else:
|
||
m = yet_another_pattern.match(data)
|
||
if m:
|
||
...
|
||
else:
|
||
...
|
||
|
||
As with PEP 572, this PEP allows the else/if portions of that chain to be
|
||
condensed, making their consistent and mutually exclusive structure more
|
||
readily apparent::
|
||
|
||
m = pattern.match(data)
|
||
if m:
|
||
...
|
||
elif m := other_pattern.match(data):
|
||
...
|
||
elif m := yet_another_pattern.match(data):
|
||
...
|
||
else:
|
||
...
|
||
|
||
Unlike PEP 572, this PEP requires that the assignment target be explicitly
|
||
indicated as local before the first use as a ``:=`` target, either by
|
||
binding it to a value (as shown above), or else by including an appropriate
|
||
explicit type declaration::
|
||
|
||
m: typing.re.Match
|
||
if m := pattern.match(data):
|
||
...
|
||
elif m := other_pattern.match(data):
|
||
...
|
||
elif m := yet_another_pattern.match(data):
|
||
...
|
||
else:
|
||
...
|
||
|
||
|
||
Capturing intermediate values from comprehensions
|
||
-------------------------------------------------
|
||
|
||
The proposal in this PEP makes it straightforward to capture and reuse
|
||
intermediate values in comprehensions and generator expressions by
|
||
exporting them to the containing block scope::
|
||
|
||
factor: int
|
||
while any(n % (factor := p) == 0 for p in small_primes):
|
||
n //= factor
|
||
|
||
total = 0
|
||
partial_sums = [total += value for value in data]
|
||
|
||
|
||
Allowing lambda expressions to act more like re-usable code thunks
|
||
------------------------------------------------------------------
|
||
|
||
This PEP allows the classic closure usage example::
|
||
|
||
def make_counter(start=0):
|
||
x = start
|
||
def counter(step=1):
|
||
nonlocal x
|
||
x += step
|
||
return x
|
||
return counter
|
||
|
||
To be abbreviated as::
|
||
|
||
def make_counter(start=0):
|
||
x = start
|
||
return lambda step=1: x += step
|
||
|
||
While the latter form is still a conceptually dense piece of code, it can be
|
||
reasonably argued that the lack of boilerplate (where the "def", "nonlocal",
|
||
and "return" keywords and two additional repetitions of the "x" variable name
|
||
have been replaced with the "lambda" keyword) may make it easier to read in
|
||
practice.
|
||
|
||
|
||
Relationship with PEP 572
|
||
=========================
|
||
|
||
The case for allowing inline assignments at all is made in PEP 572. This
|
||
competing PEP was initially going to propose an alternate surface syntax
|
||
(``EXPR given NAME = EXPR``), while retaining the expression semantics from
|
||
PEP 572, but that changed when discussing one of the initial motivating use
|
||
cases for allowing embedded assignments at all: making it possible to easily
|
||
calculate cumulative sums in comprehensions and generator expressions.
|
||
|
||
As a result of that, and unlike PEP 572, this PEP focuses primarily on use
|
||
cases for inline augmented assignment. It also has the effect of converting
|
||
cases that currently inevitably raise ``UnboundLocalError`` at function call
|
||
time to report a new compile time ``TargetNameError``.
|
||
|
||
New syntax for a name rebinding expression (``NAME := TARGET``) is then added
|
||
not only to handle the same use cases as are identified in PEP 572, but also
|
||
as a lower level primitive to help illustrate, implement and explain
|
||
the new augmented assignment semantics, rather than being the sole change being
|
||
proposed.
|
||
|
||
The author of this PEP believes that this approach makes the value of the new
|
||
flexibility in name rebinding clearer, while also mitigating many of the
|
||
potential concerns raised with PEP 572 around explaining when to use
|
||
``NAME = EXPR`` over ``NAME := EXPR`` (and vice-versa), without resorting to
|
||
prohibiting the bare statement form of ``NAME := EXPR`` outright (such
|
||
that ``NAME := EXPR`` is a compile error, but ``(NAME := EXPR)`` is permitted).
|
||
|
||
|
||
Acknowledgements
|
||
================
|
||
|
||
The PEP author wishes to thank Chris Angelico for his work on PEP 572, and his
|
||
efforts to create a coherent summary of the great many sprawling discussions
|
||
that spawned on both python-ideas and python-dev, as well as Tim Peters for
|
||
the in-depth discussion of parent local scoping that prompted the above
|
||
scoping proposal for augmented assignments inside scoped expressions.
|
||
|
||
Eric Snow's feedback on a pre-release version of this PEP helped make it
|
||
significantly more readable.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] The beginning of Tim's genexp & comprehension scoping thread
|
||
(https://mail.python.org/pipermail/python-ideas/2018-May/050367.html)
|
||
|
||
.. [2] Reintroducing the original cumulative sums use case
|
||
(https://mail.python.org/pipermail/python-ideas/2018-May/050544.html)
|
||
|
||
.. [3] Tim's language reference level explanation of his proposed scoping semantics
|
||
(https://mail.python.org/pipermail/python-ideas/2018-May/050729.html)
|
||
|
||
.. [4] Guido's endorsement of Tim's proposed genexp & comprehension scoping
|
||
(https://mail.python.org/pipermail/python-ideas/2018-May/050411.html)
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|