823 lines
33 KiB
ReStructuredText
823 lines
33 KiB
ReStructuredText
PEP: 577
|
|
Title: Augmented Assignment Expressions
|
|
Author: Nick Coghlan <ncoghlan@gmail.com>
|
|
Status: Withdrawn
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 14-May-2018
|
|
Python-Version: 3.8
|
|
Post-History: 22-May-2018
|
|
|
|
|
|
PEP Withdrawal
|
|
==============
|
|
|
|
While working on this PEP, I realised that it didn't really address what was
|
|
actually bothering me about :pep:`572`'s proposed scoping rules for previously
|
|
unreferenced assignment targets, and also had some significant undesirable
|
|
consequences (most notably, allowing ``>>=`` and ``<<=`` as inline augmented
|
|
assignment operators that meant something entirely different from the ``>=``
|
|
and ``<=`` comparison operators).
|
|
|
|
I also realised that even without dedicated syntax of their own, :pep:`572`
|
|
technically allows inline augmented assignments to be written using the
|
|
``operator`` module::
|
|
|
|
from operator import iadd
|
|
if (target := iadd(target, value)) < limit:
|
|
...
|
|
|
|
The restriction to simple names as inline assignment targets means that the
|
|
target expression can always be repeated without side effects, and thus avoids
|
|
the ambiguity that would arise from allowing actual embedded augmented
|
|
assignments (it's still a bad idea, since it would almost certainly be hard
|
|
for humans to read, this note is just about the theoretical limits of language
|
|
level expressiveness).
|
|
|
|
Accordingly, I withdrew this PEP without submitting it for pronouncement. At
|
|
the time I also started writing a replacement PEP that focused specifically on
|
|
the handling of assignment targets which hadn't already been declared as local
|
|
variables in the current scope (for both regular block scopes, and for scoped
|
|
expressions), but that draft never even reached a stage where *I* liked it
|
|
better than the ultimately accepted proposal in :pep:`572`, so it was never
|
|
posted anywhere, nor assigned a PEP number.
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This is a proposal to allow augmented assignments such as ``x += 1`` to be
|
|
used as expressions, not just statements.
|
|
|
|
As part of this, ``NAME := EXPR`` is proposed as an inline assignment expression
|
|
that uses the new augmented assignment scoping rules, rather than implicitly
|
|
defining a new local variable name the way that existing name binding
|
|
statements do. The question of allowing expression level local variable
|
|
declarations at function scope is deliberately separated from the question of
|
|
allowing expression level name bindings, and deferred to a later PEP.
|
|
|
|
This PEP is a direct competitor to :pep:`572` (although it borrows heavily from that
|
|
PEP's motivation, and even shares the proposed syntax for inline assignments).
|
|
See `Relationship with PEP 572`_ for more details on the connections between
|
|
the two PEPs.
|
|
|
|
To improve the usability of the new expressions, a semantic split is proposed
|
|
between the handling of augmented assignments in regular block scopes (modules,
|
|
classes, and functions), and the handling of augmented assignments in scoped
|
|
expressions (lambda expressions, generator expressions, and comprehensions),
|
|
such that all inline assignments default to targeting the nearest containing
|
|
block scope.
|
|
|
|
A new compile time ``TargetNameError`` is added as a subclass of ``SyntaxError``
|
|
to handle cases where it is deemed to be currently unclear which target is
|
|
expected to be rebound by an inline assignment, or else the target scope
|
|
for the inline assignment is considered invalid for another reason.
|
|
|
|
|
|
Syntax and semantics
|
|
====================
|
|
|
|
Augmented assignment expressions
|
|
--------------------------------
|
|
|
|
The language grammar would be adjusted to allow augmented assignments to
|
|
appear as expressions, where the result of the augmented assignment
|
|
expression is the same post-calculation reference as is being bound to the
|
|
given target.
|
|
|
|
For example::
|
|
|
|
>>> n = 0
|
|
>>> n += 5
|
|
5
|
|
>>> n -= 2
|
|
3
|
|
>>> n *= 3
|
|
9
|
|
>>> n
|
|
9
|
|
|
|
For mutable targets, this means the result is always just the original object::
|
|
|
|
>>> seq = []
|
|
>>> seq_id = id(seq)
|
|
>>> seq += range(3)
|
|
[0, 1, 2]
|
|
>>> seq_id == id(seq)
|
|
True
|
|
|
|
Augmented assignments to attributes and container subscripts will be permitted,
|
|
with the result being the post-calculation reference being bound to the target,
|
|
just as it is for simple name targets::
|
|
|
|
def increment(self, step=1):
|
|
return self._value += step
|
|
|
|
In these cases, ``__getitem__`` and ``__getattribute__`` will *not* be called
|
|
after the assignment has already taken place (they will only be called as
|
|
needed to evaluate the in-place operation).
|
|
|
|
|
|
Adding an inline assignment operator
|
|
------------------------------------
|
|
|
|
Given only the addition of augmented assignment expressions, it would be
|
|
possible to abuse a symbol like ``|=`` as a general purpose assignment
|
|
operator by defining a ``Target`` wrapper type that worked as follows::
|
|
|
|
>>> class Target:
|
|
... def __init__(self, value):
|
|
... self.value = value
|
|
... def __or__(self, other):
|
|
... return Target(other)
|
|
...
|
|
>>> x = Target(10)
|
|
>>> x.value
|
|
10
|
|
>>> x |= 42
|
|
<__main__.Target object at 0x7f608caa8048>
|
|
>>> x.value
|
|
42
|
|
|
|
This is similar to the way that storing a single reference in a list was long
|
|
used as a workaround for the lack of a ``nonlocal`` keyword, and can still be
|
|
used today (in combination with ``operator.itemsetter``) to work around the
|
|
lack of expression level assignments.
|
|
|
|
Rather than requiring such workarounds, this PEP instead proposes that
|
|
:pep:`572`'s "NAME := EXPR" syntax be adopted as a new inline assignment
|
|
expression that uses the augmented assignment scoping rules described below.
|
|
|
|
This cleanly handles cases where only the new value is of interest, and the
|
|
previously bound value (if any) can just be discarded completely.
|
|
|
|
Note that for both simple names and complex assignment targets, the inline
|
|
assignment operator does *not* read the previous reference before assigning
|
|
the new one. However, when used at function scope (either directly or inside
|
|
a scoped expression), it does *not* implicitly define a new local variable,
|
|
and will instead raise ``TargetNameError`` (as described for augmented
|
|
assignments below).
|
|
|
|
|
|
Assignment operator precedence
|
|
------------------------------
|
|
|
|
To preserve the existing semantics of augmented assignment statements,
|
|
inline assignment operators will be defined as being of lower precedence
|
|
than all other operators, include the comma pseudo-operator. This ensures
|
|
that when used as a top level expression the entire right hand side of the
|
|
expression is still interpreted as the value to be processed (even when that
|
|
value is a tuple without parentheses).
|
|
|
|
The difference this introduces relative to :pep:`572` is that where
|
|
``(n := first, second)`` sets ``n = first`` in :pep:`572`, in this PEP it would set
|
|
``n = (first, second)``, and getting the first meaning would require an extra
|
|
set of parentheses (``((n := first), second)``).
|
|
|
|
:pep:`572` quite reasonably notes that this results in ambiguity when assignment
|
|
expressions are used as function call arguments. This PEP resolves that concern
|
|
a different way by requiring that assignment expressions be parenthesised
|
|
when used as arguments to a function call (unless they're the sole argument).
|
|
|
|
This is a more relaxed version of the restriction placed on generator
|
|
expressions (which always require parentheses, except when they're the sole
|
|
argument to a function call).
|
|
|
|
|
|
Augmented assignment to names in block scopes
|
|
---------------------------------------------
|
|
|
|
No target name binding changes are proposed for augmented assignments at module
|
|
or class scope (this also includes code executed using "exec" or "eval"). These
|
|
will continue to implicitly declare a new local variable as the binding target
|
|
as they do today, and (if necessary) will be able to resolve the name from an
|
|
outer scope before binding it locally.
|
|
|
|
At function scope, augmented assignments will be changed to require that there
|
|
be either a preceding name binding or variable declaration to explicitly
|
|
establish the target name as being local to the function, or else an explicit
|
|
``global`` or ``nonlocal`` declaration. ``TargetNameError``, a new
|
|
``SyntaxError`` subclass, will be raised at compile time if no such binding or
|
|
declaration is present.
|
|
|
|
For example, the following code would compile and run as it does today::
|
|
|
|
x = 0
|
|
x += 1 # Sets global "x" to 1
|
|
|
|
class C:
|
|
x += 1 # Sets local "x" to 2, leaves global "x" alone
|
|
|
|
def local_target():
|
|
x = 0
|
|
x += 1 # Sets local "x" to 1, leaves global "x" alone
|
|
|
|
def global_target():
|
|
global x
|
|
x += 1 # Increments global "x" each time this runs
|
|
|
|
def nonlocal_target():
|
|
x = 0
|
|
def g():
|
|
nonlocal x
|
|
x += 1 # Increments "x" in outer scope each time this runs
|
|
return x
|
|
return g
|
|
|
|
The follow examples would all still compile and then raise an error at runtime
|
|
as they do today::
|
|
|
|
n += 1 # Raises NameError at runtime
|
|
|
|
class C:
|
|
n += 1 # Raises NameError at runtime
|
|
|
|
def missing_global():
|
|
global n
|
|
n += 1 # Raises NameError at runtime
|
|
|
|
def delayed_nonlocal_initialisation():
|
|
def f():
|
|
nonlocal n
|
|
n += 1
|
|
f() # Raises NameError at runtime
|
|
n = 0
|
|
|
|
def skipped_conditional_initialisation():
|
|
if False:
|
|
n = 0
|
|
n += 1 # Raises UnboundLocalError at runtime
|
|
|
|
def local_declaration_without_initial_assignment():
|
|
n: typing.Any
|
|
n += 1 # Raises UnboundLocalError at runtime
|
|
|
|
Whereas the following would raise a compile time ``DeprecationWarning``
|
|
initially, and eventually change to report a compile time ``TargetNameError``::
|
|
|
|
def missing_target():
|
|
x += 1 # Compile time TargetNameError due to ambiguous target scope
|
|
# Is there a missing initialisation of "x" here? Or a missing
|
|
# global or nonlocal declaration?
|
|
|
|
As a conservative implementation approach, the compile time function name
|
|
resolution change would be introduced as a ``DeprecationWarning`` in Python
|
|
3.8, and then converted to ``TargetNameError`` in Python 3.9. This avoids
|
|
potential problems in cases where an unused function would currently raise
|
|
``UnboundLocalError`` if it was ever actually called, but the code is actually
|
|
unused - converting that latent runtime defect to a compile time error qualifies
|
|
as a backwards incompatible change that requires a deprecation period.
|
|
|
|
When augmented assignments are used as expressions in function scope (rather
|
|
than as standalone statements), there aren't any backwards compatibility
|
|
concerns, so the compile time name binding checks would be enforced immediately
|
|
in Python 3.8.
|
|
|
|
Similarly, the new inline assignment expressions would always require explicit
|
|
predeclaration of their target scope when used as part of a function, at least
|
|
for Python 3.8. (See the design discussion section for notes on potentially
|
|
revisiting that restriction in the future).
|
|
|
|
|
|
Augmented assignment to names in scoped expressions
|
|
---------------------------------------------------
|
|
|
|
Scoped expressions is a new collective term being proposed for expressions that
|
|
introduce a new nested scope of execution, either as an intrinsic part of their
|
|
operation (lambda expressions, generator expressions), or else as a way of
|
|
hiding name binding operations from the containing scope (container
|
|
comprehensions).
|
|
|
|
Unlike regular functions, these scoped expressions can't include explicit
|
|
``global`` or ``nonlocal`` declarations to rebind names directly in an outer
|
|
scope.
|
|
|
|
Instead, their name binding semantics for augmented assignment expressions would
|
|
be defined as follows:
|
|
|
|
* augmented assignment targets used in scoped expressions are expected to either
|
|
be already bound in the containing block scope, or else have their scope
|
|
explicitly declared in the containing block scope. If no suitable name
|
|
binding or declaration can be found in that scope, then ``TargetNameError``
|
|
will be raised at compile time (rather than creating a new binding within
|
|
the scoped expression).
|
|
* if the containing block scope is a function scope, and the target name is
|
|
explicitly declared as ``global`` or ``nonlocal``, then it will be use the
|
|
same scope declaration in the body of the scoped expression
|
|
* if the containing block scope is a function scope, and the target name is
|
|
a local variable in that function, then it will be implicitly declared as
|
|
``nonlocal`` in the body of the scoped expression
|
|
* if the containing block scope is a class scope, than ``TargetNameError`` will
|
|
always be raised, with a dedicated message indicating that combining class
|
|
scopes with augmented assignments in scoped expressions is not currently
|
|
permitted.
|
|
* if a name is declared as a formal parameter (lambda expressions), or as an
|
|
iteration variable (generator expressions, comprehensions), then that name
|
|
is considered local to that scoped expression, and attempting to use it as
|
|
the target of an augmented assignment operation in that scope, or any nested
|
|
scoped expression, will raise ``TargetNameError`` (this is a restriction that
|
|
could potentially be lifted later, but is being proposed for now to simplify
|
|
the initial set of compile time and runtime semantics that needs to be
|
|
covered in the language reference and handled by the compiler and interpreter)
|
|
|
|
For example, the following code would work as shown::
|
|
|
|
>>> global_target = 0
|
|
>>> incr_global_target = lambda: global_target += 1
|
|
>>> incr_global_target()
|
|
1
|
|
>>> incr_global_target()
|
|
2
|
|
>>> global_target
|
|
2
|
|
>>> def cumulative_sums(data, start=0)
|
|
... total = start
|
|
... yield from (total += value for value in data)
|
|
... return total
|
|
...
|
|
>>> print(list(cumulative_sums(range(5))))
|
|
[0, 1, 3, 6, 10]
|
|
|
|
While the following examples would all raise ``TargetNameError``::
|
|
|
|
class C:
|
|
cls_target = 0
|
|
incr_cls_target = lambda: cls_target += 1 # Error due to class scope
|
|
|
|
def missing_target():
|
|
incr_x = lambda: x += 1 # Error due to missing target "x"
|
|
|
|
def late_target():
|
|
incr_x = lambda: x += 1 # Error due to "x" being declared after use
|
|
x = 1
|
|
|
|
lambda arg: arg += 1 # Error due to attempt to target formal parameter
|
|
|
|
[x += 1 for x in data] # Error due to attempt to target iteration variable
|
|
|
|
|
|
As augmented assignments currently can't appear inside scoped expressions, the
|
|
above compile time name resolution exceptions would be included as part of the
|
|
initial implementation rather than needing to be phased in as a potentially
|
|
backwards incompatible change.
|
|
|
|
|
|
Design discussion
|
|
=================
|
|
|
|
Allowing complex assignment targets
|
|
-----------------------------------
|
|
|
|
The initial drafts of this PEP kept :pep:`572`'s restriction to single name targets
|
|
when augmented assignments were used as expressions, allowing attribute and
|
|
subscript targets solely for the statement form.
|
|
|
|
However, enforcing that required varying the permitted targets based on whether
|
|
or not the augmented assignment was a top level expression or not, as well as
|
|
explaining why ``n += 1``, ``(n += 1)``, and ``self.n += 1`` were all legal,
|
|
but ``(self.n += 1)`` was prohibited, so the proposal was simplified to allow
|
|
all existing augmented assignment targets for the expression form as well.
|
|
|
|
Since this PEP defines ``TARGET := EXPR`` as a variant on augmented assignment,
|
|
that also gained support for assignment and subscript targets.
|
|
|
|
|
|
Augmented assignment or name binding only?
|
|
------------------------------------------
|
|
|
|
:pep:`572` makes a reasonable case that the potential use cases for inline
|
|
augmented assignment are notably weaker than those for inline assignment in
|
|
general, so it's acceptable to require that they be spelled as ``x := x + 1``,
|
|
bypassing any in-place augmented assignment methods.
|
|
|
|
While this is at least arguably true for the builtin types (where potential
|
|
counterexamples would probably need to focus on set manipulation use cases
|
|
that the PEP author doesn't personally have), it would also rule out more
|
|
memory intensive use cases like manipulation of NumPy arrays, where the data
|
|
copying involved in out-of-place operations can make them impractical as
|
|
alternatives to their in-place counterparts.
|
|
|
|
That said, this PEP mainly exists because the PEP author found the inline
|
|
assignment proposal much easier to grasp as "It's like ``+=``, only skipping
|
|
the addition step", and also liked the way that that framing provides an
|
|
actual semantic difference between ``NAME = EXPR`` and ``NAME := EXPR`` at
|
|
function scope.
|
|
|
|
That difference in target scoping behaviour means that the ``NAME := EXPR``
|
|
syntax would be expected to have two primary use cases:
|
|
|
|
* as a way of allowing assignments to be embedded as an expression in an ``if``
|
|
or ``while`` statement, or as part of a scoped expression
|
|
* as a way of requesting a compile time check that the target name be previously
|
|
declared or bound in the current function scope
|
|
|
|
At module or class scope, ``NAME = EXPR`` and ``NAME := EXPR`` would be
|
|
semantically equivalent due to the compiler's lack of visibility into the set
|
|
of names that will be resolvable at runtime, but code linters and static
|
|
type checkers would be encouraged to enforce the same "declaration or assignment
|
|
required before use" behaviour for ``NAME := EXPR`` as the compiler would
|
|
enforce at function scope.
|
|
|
|
|
|
Postponing a decision on expression level target declarations
|
|
-------------------------------------------------------------
|
|
|
|
At least for Python 3.8, usage of inline assignments (whether augmented or not)
|
|
at function scope would always require a preceding name binding or scope
|
|
declaration to avoid getting ``TargetNameError``, even when used outside a
|
|
scoped expression.
|
|
|
|
The intent behind this requirement is to clearly separate the following two
|
|
language design questions:
|
|
|
|
1. Can an expression rebind a name in the current scope?
|
|
2. Can an expression declare a new name in the current scope?
|
|
|
|
For module global scopes, the answer to both of those questions is unequivocally
|
|
"Yes", because it's a language level guarantee that mutating the ``globals()``
|
|
dict will immediately impact the runtime module scope, and ``global NAME``
|
|
declarations inside a function can have the same effect (as can importing the
|
|
currently executing module and modifying its attributes).
|
|
|
|
For class scopes, the answer to both questions is also "Yes" in practice,
|
|
although less unequivocally so, since the semantics of ``locals()`` are
|
|
currently formally unspecified. However, if the current behaviour of ``locals()``
|
|
at class scope is taken as normative (as :pep:`558` proposes), then this is
|
|
essentially the same scenario as manipulating the module globals, just using
|
|
``locals()`` instead.
|
|
|
|
For function scopes, however, the current answers to these two questions are
|
|
respectively "Yes" and "No". Expression level rebinding of function locals is
|
|
already possible thanks to lexically nested scopes and explicit ``nonlocal NAME``
|
|
expressions. While this PEP will likely make expression level rebinding more
|
|
common than it is today, it isn't a fundamentally new concept for the language.
|
|
|
|
By contrast, declaring a *new* function local variable is currently a statement
|
|
level action, involving one of:
|
|
|
|
* an assignment statement (``NAME = EXPR``, ``OTHER_TARGET = NAME = EXPR``, etc)
|
|
* a variable declaration (``NAME : EXPR``)
|
|
* a nested function definition
|
|
* a nested class definition
|
|
* a ``for`` loop
|
|
* a ``with`` statement
|
|
* an ``except`` clause (with limited scope of access)
|
|
|
|
The historical trend for the language has actually been to *remove* support for
|
|
expression level declarations of function local names, first with the
|
|
introduction of "fast locals" semantics (which made the introduction of names
|
|
via ``locals()`` unsupported for function scopes), and again with the hiding
|
|
of comprehension iteration variables in Python 3.0.
|
|
|
|
Now, it may be that in Python 3.9, we decide to revisit this question based on
|
|
our experience with expression level name binding in Python 3.8, and decide that
|
|
we really do want expression level function local variable declarations as well,
|
|
and that we want ``NAME := EXPR`` to be the way we spell that (rather than,
|
|
for example, spelling inline declarations more explicitly as
|
|
``NAME := EXPR given NAME``, which would permit them to carry type annotations,
|
|
and also permit them to declare new local variables in scoped expressions,
|
|
rather than having to pollute the namespace in their containing scope).
|
|
|
|
But the proposal in this PEP is that we explicitly give ourselves a full
|
|
release to decide how much we want that feature, and exactly where we find
|
|
its absence irritating. Python has survived happily without expression level
|
|
name bindings *or* declarations for decades, so we can afford to give ourselves
|
|
a couple of years to decide if we really want *both* of those, or if expression
|
|
level bindings are sufficient.
|
|
|
|
|
|
Ignoring scoped expressions when determining augmented assignment targets
|
|
-------------------------------------------------------------------------
|
|
|
|
When discussing possible binding semantics for :pep:`572`'s assignment expressions,
|
|
Tim Peters made a plausible case [1]_, [2]_, [3]_ for assignment expressions targeting
|
|
the containing block scope, essentially ignoring any intervening scoped
|
|
expressions.
|
|
|
|
This approach allows use cases like cumulative sums, or extracting the final
|
|
value from a generator expression to be written in a relatively straightforward
|
|
way::
|
|
|
|
total = 0
|
|
partial_sums = [total := total + value for value in data]
|
|
|
|
factor = 1
|
|
while any(n % (factor := p) == 0 for p in small_primes):
|
|
n //= factor
|
|
|
|
Guido also expressed his approval for this general approach [4]_.
|
|
|
|
The proposal in this PEP differs from Tim's original proposal in three main
|
|
areas:
|
|
|
|
- it applies the proposal to all augmented assignment operators, not just a
|
|
single new name binding operator
|
|
- as far as is practical, it extends the augmented assignment requirement that
|
|
the name already be defined to the new name binding operator (raising
|
|
``TargetNameError`` rather than implicitly declaring new local variables at
|
|
function scope)
|
|
- it includes lambda expressions in the set of scopes that get ignored for
|
|
target name binding purposes, making this transparency to assignments common
|
|
to all of the scoped expressions rather than being specific to comprehensions
|
|
and generator expressions
|
|
|
|
With scoped expressions being ignored when calculating binding targets, it's
|
|
once again difficult to detect the scoping difference between the outermost
|
|
iterable expressions in generator expressions and comprehensions (you have to
|
|
mess about with either class scopes or attempting to rebind iteration Variables
|
|
to detect it), so there's also no need to tinker with that.
|
|
|
|
|
|
Treating inline assignment as an augmented assignment variant
|
|
-------------------------------------------------------------
|
|
|
|
One of the challenges with :pep:`572` is the fact that ``NAME = EXPR`` and
|
|
``NAME := EXPR`` are entirely semantically equivalent at every scope. This
|
|
makes the two forms hard to teach, since there's no inherent nudge towards
|
|
choosing one over the other at the statement level, so you end up having to
|
|
resort to "``NAME = EXPR`` is preferred because it's been around longer"
|
|
(and :pep:`572` proposes to enforce that historical idiosyncrasy at the compiler
|
|
level).
|
|
|
|
That semantic equivalence is difficult to avoid at module and class scope while
|
|
still having ``if NAME := EXPR:`` and ``while NAME := EXPR:`` work sensibly, but
|
|
at function scope the compiler's comprehensive view of all local names makes
|
|
it possible to require that the name be assigned or declared before use,
|
|
providing a reasonable incentive to continue to default to using the
|
|
``NAME = EXPR`` form when possible, while also enabling the use of the
|
|
``NAME := EXPR`` as a kind of simple compile time assertion (i.e. explicitly
|
|
indicating that the targeted name has already been bound or declared and hence
|
|
should already be known to the compiler).
|
|
|
|
If Guido were to declare that support for inline declarations was a hard
|
|
design requirement, then this PEP would be updated to propose that
|
|
``EXPR given NAME`` also be introduced as a way to support inline name declarations
|
|
after arbitrary expressions (this would allow the inline name declarations to be
|
|
deferred until the end of a complex expression rather than needing to be
|
|
embedded in the middle of it, and :pep:`8` would gain a recommendation encouraging
|
|
that style).
|
|
|
|
|
|
Disallowing augmented assignments in class level scoped expressions
|
|
-------------------------------------------------------------------
|
|
|
|
While modern classes do define an implicit closure that's visible to method
|
|
implementations (in order to make ``__class__`` available for use in zero-arg
|
|
``super()`` calls), there's no way for user level code to explicitly add
|
|
additional names to that scope.
|
|
|
|
Meanwhile, attributes defined in a class body are ignored for the purpose of
|
|
defining a method's lexical closure, which means adding them there wouldn't
|
|
work at an implementation level.
|
|
|
|
Rather than trying to resolve that inherent ambiguity, this PEP simply
|
|
prohibits such usage, and requires that any affected logic be written somewhere
|
|
other than directly inline in the class body (e.g. in a separate helper
|
|
function).
|
|
|
|
|
|
Comparison operators vs assignment operators
|
|
--------------------------------------------
|
|
|
|
The ``OP=`` construct as an expression currently indicates a comparison
|
|
operation::
|
|
|
|
x == y # Equals
|
|
x >= y # Greater-than-or-equal-to
|
|
x <= y # Less-than-or-equal-to
|
|
|
|
Both this PEP and :pep:`572` propose adding at least one operator that's somewhat
|
|
similar in appearance, but defines an assignment instead::
|
|
|
|
x := y # Becomes
|
|
|
|
This PEP then goes much further and allows all *13* augmented assignment symbols
|
|
to be uses as binary operators::
|
|
|
|
x += y # In-place add
|
|
x -= y # In-place minus
|
|
x *= y # In-place multiply
|
|
x @= y # In-place matrix multiply
|
|
x /= y # In-place division
|
|
x //= y # In-place int division
|
|
x %= y # In-place mod
|
|
x &= y # In-place bitwise and
|
|
x |= y # In-place bitwise or
|
|
x ^= y # In-place bitwise xor
|
|
x <<= y # In-place left shift
|
|
x >>= y # In-place right shift
|
|
x **= y # In-place power
|
|
|
|
Of those additional binary operators, the most questionable would be the
|
|
bitshift assignment operators, since they're each only one doubled character
|
|
away from one of the inclusive ordered comparison operators.
|
|
|
|
|
|
Examples
|
|
========
|
|
|
|
Simplifying retry loops
|
|
-----------------------
|
|
|
|
There are currently a few different options for writing retry loops, including::
|
|
|
|
# Post-decrementing a counter
|
|
remaining_attempts = MAX_ATTEMPTS
|
|
while remaining_attempts:
|
|
remaining_attempts -= 1
|
|
try:
|
|
result = attempt_operation()
|
|
except Exception as exc:
|
|
continue # Failed, so try again
|
|
log.debug(f"Succeeded after {attempts} attempts")
|
|
break # Success!
|
|
else:
|
|
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
|
|
|
# Loop-and-a-half with a pre-incremented counter
|
|
attempt = 0
|
|
while True:
|
|
attempts += 1
|
|
if attempts > MAX_ATTEMPTS:
|
|
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
|
try:
|
|
result = attempt_operation()
|
|
except Exception as exc:
|
|
continue # Failed, so try again
|
|
log.debug(f"Succeeded after {attempts} attempts")
|
|
break # Success!
|
|
|
|
Each of the available options hides some aspect of the intended loop structure
|
|
inside the loop body, whether that's the state modification, the exit condition,
|
|
or both.
|
|
|
|
The proposal in this PEP allows both the state modification and the exit
|
|
condition to be included directly in the loop header::
|
|
|
|
attempt = 0
|
|
while (attempt += 1) <= MAX_ATTEMPTS:
|
|
try:
|
|
result = attempt_operation()
|
|
except Exception as exc:
|
|
continue # Failed, so try again
|
|
log.debug(f"Succeeded after {attempts} attempts")
|
|
break # Success!
|
|
else:
|
|
raise OperationFailed(f"Failed after {MAX_ATTEMPTS} attempts") from exc
|
|
|
|
|
|
Simplifying if-elif chains
|
|
--------------------------
|
|
|
|
if-elif chains that need to rebind the checked condition currently need to
|
|
be written using nested if-else statements::
|
|
|
|
|
|
m = pattern.match(data)
|
|
if m:
|
|
...
|
|
else:
|
|
m = other_pattern.match(data)
|
|
if m:
|
|
...
|
|
else:
|
|
m = yet_another_pattern.match(data)
|
|
if m:
|
|
...
|
|
else:
|
|
...
|
|
|
|
As with :pep:`572`, this PEP allows the else/if portions of that chain to be
|
|
condensed, making their consistent and mutually exclusive structure more
|
|
readily apparent::
|
|
|
|
m = pattern.match(data)
|
|
if m:
|
|
...
|
|
elif m := other_pattern.match(data):
|
|
...
|
|
elif m := yet_another_pattern.match(data):
|
|
...
|
|
else:
|
|
...
|
|
|
|
Unlike :pep:`572`, this PEP requires that the assignment target be explicitly
|
|
indicated as local before the first use as a ``:=`` target, either by
|
|
binding it to a value (as shown above), or else by including an appropriate
|
|
explicit type declaration::
|
|
|
|
m: typing.re.Match
|
|
if m := pattern.match(data):
|
|
...
|
|
elif m := other_pattern.match(data):
|
|
...
|
|
elif m := yet_another_pattern.match(data):
|
|
...
|
|
else:
|
|
...
|
|
|
|
|
|
Capturing intermediate values from comprehensions
|
|
-------------------------------------------------
|
|
|
|
The proposal in this PEP makes it straightforward to capture and reuse
|
|
intermediate values in comprehensions and generator expressions by
|
|
exporting them to the containing block scope::
|
|
|
|
factor: int
|
|
while any(n % (factor := p) == 0 for p in small_primes):
|
|
n //= factor
|
|
|
|
total = 0
|
|
partial_sums = [total += value for value in data]
|
|
|
|
|
|
Allowing lambda expressions to act more like re-usable code thunks
|
|
------------------------------------------------------------------
|
|
|
|
This PEP allows the classic closure usage example::
|
|
|
|
def make_counter(start=0):
|
|
x = start
|
|
def counter(step=1):
|
|
nonlocal x
|
|
x += step
|
|
return x
|
|
return counter
|
|
|
|
To be abbreviated as::
|
|
|
|
def make_counter(start=0):
|
|
x = start
|
|
return lambda step=1: x += step
|
|
|
|
While the latter form is still a conceptually dense piece of code, it can be
|
|
reasonably argued that the lack of boilerplate (where the "def", "nonlocal",
|
|
and "return" keywords and two additional repetitions of the "x" variable name
|
|
have been replaced with the "lambda" keyword) may make it easier to read in
|
|
practice.
|
|
|
|
|
|
Relationship with PEP 572
|
|
=========================
|
|
|
|
The case for allowing inline assignments at all is made in :pep:`572`. This
|
|
competing PEP was initially going to propose an alternate surface syntax
|
|
(``EXPR given NAME = EXPR``), while retaining the expression semantics from
|
|
:pep:`572`, but that changed when discussing one of the initial motivating use
|
|
cases for allowing embedded assignments at all: making it possible to easily
|
|
calculate cumulative sums in comprehensions and generator expressions.
|
|
|
|
As a result of that, and unlike :pep:`572`, this PEP focuses primarily on use
|
|
cases for inline augmented assignment. It also has the effect of converting
|
|
cases that currently inevitably raise ``UnboundLocalError`` at function call
|
|
time to report a new compile time ``TargetNameError``.
|
|
|
|
New syntax for a name rebinding expression (``NAME := TARGET``) is then added
|
|
not only to handle the same use cases as are identified in :pep:`572`, but also
|
|
as a lower level primitive to help illustrate, implement and explain
|
|
the new augmented assignment semantics, rather than being the sole change being
|
|
proposed.
|
|
|
|
The author of this PEP believes that this approach makes the value of the new
|
|
flexibility in name rebinding clearer, while also mitigating many of the
|
|
potential concerns raised with :pep:`572` around explaining when to use
|
|
``NAME = EXPR`` over ``NAME := EXPR`` (and vice-versa), without resorting to
|
|
prohibiting the bare statement form of ``NAME := EXPR`` outright (such
|
|
that ``NAME := EXPR`` is a compile error, but ``(NAME := EXPR)`` is permitted).
|
|
|
|
|
|
Acknowledgements
|
|
================
|
|
|
|
The PEP author wishes to thank Chris Angelico for his work on :pep:`572`, and his
|
|
efforts to create a coherent summary of the great many sprawling discussions
|
|
that spawned on both python-ideas and python-dev, as well as Tim Peters for
|
|
the in-depth discussion of parent local scoping that prompted the above
|
|
scoping proposal for augmented assignments inside scoped expressions.
|
|
|
|
Eric Snow's feedback on a pre-release version of this PEP helped make it
|
|
significantly more readable.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] The beginning of Tim's genexp & comprehension scoping thread
|
|
(https://mail.python.org/pipermail/python-ideas/2018-May/050367.html)
|
|
|
|
.. [2] Reintroducing the original cumulative sums use case
|
|
(https://mail.python.org/pipermail/python-ideas/2018-May/050544.html)
|
|
|
|
.. [3] Tim's language reference level explanation of his proposed scoping semantics
|
|
(https://mail.python.org/pipermail/python-ideas/2018-May/050729.html)
|
|
|
|
.. [4] Guido's endorsement of Tim's proposed genexp & comprehension scoping
|
|
(https://mail.python.org/pipermail/python-ideas/2018-May/050411.html)
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|