python-peps/peps/pep-0736.rst

PEP: 736
Title: Shorthand syntax for keyword arguments at invocation
Author: Joshua Bambrick <jbambrick@google.com>,
        Chris Angelico <rosuav@gmail.com>
Sponsor: Guido van Rossum <guido@python.org>
Discussions-To: https://discuss.python.org/t/pep-736-shorthand-syntax-for-keyword-arguments-at-invocation/43432
Status: Draft
Type: Standards Track
Created: 28-Nov-2023
Python-Version: 3.13
Post-History: `14-Oct-2023 <https://discuss.python.org/t/syntactic-sugar-to-encourage-use-of-named-arguments/36217>`__,
              `17-Jan-2024 <https://discuss.python.org/t/pep-736-shorthand-syntax-for-keyword-arguments-at-invocation/43432>`__,

Abstract
========

This PEP proposes introducing syntactic sugar ``f(x=)`` for the common
pattern where a named argument is the same as the name of the variable
corresponding to its value ``f(x=x)``.

Motivation
==========

Keyword argument syntax can become needlessly repetitive and verbose.

Consider the following call:
::

    my_function(
      my_first_variable=my_first_variable,
      my_second_variable=my_second_variable,
      my_third_variable=my_third_variable,
    )

The case of a keyword argument name matching the variable name of its value is
prevalent among all major Python libraries. This verbosity and redundancy
discourages use of named arguments and reduces readability by increasing visual
noise.

Rationale
=========

There are two ways to invoke a function with arguments: by position and by
keyword. Keyword arguments confer many benefits by being explicit, thus
increasing readability and minimising the risk of inadvertent transposition. On
the flipside, positional arguments are often used simply to minimise verbosity
and visual noise.

We contend that a simple syntactic sugar used to simplify this common pattern
which would confer numerous benefits:

Encourages use of named variables
---------------------------------

This syntax would encourage the use of named variables, thereby increasing
readability (*explicit is better than implicit*) and reducing bugs from argument
transposition.

Reduces verbosity
-----------------

By minimising visual noise and in some cases lines of code, we can increase
readability (*readability counts*).

Encourages consistent variable names
------------------------------------

A common problem is that semantically identical variables have different names
depending on their contexts. This syntax would encourage authors to use the same
variable name when calling a function as the argument name, which would increase
consistency of variable names used and hence also *readability*.

Applicability to dictionary construction
----------------------------------------

This syntax can be applied to dictionary construction where a similar
pattern frequently occurs (where dictionary keys are identical the names of the
variables assigned as their values), ``{"x": x, "y": y}`` or ``dict(x=x, y=y)``.
With this feature, this can now also be trivially written as ``dict(x=, y=)``.
Whether to further support similar syntax in dictionary literals is an open
question out of the scope of this PEP.

Specification
=============

We propose to introduce syntactic sugar such that, if the value of a keyword
argument is omitted from a function invocation, the argument's value is inferred
to be the variable matching that name at the invocation scope.

For example, the function invocation:
::

    my_function(my_first_variable=, my_second_variable=, my_third_variable=)

Will be interpreted exactly equivalently to following in existing syntax:
::

    my_function(
      my_first_variable=my_first_variable,
      my_second_variable=my_second_variable,
      my_third_variable=my_third_variable,
    )

If no variable matches that name in the invocation scope, a ``NameError`` is
raised in an identical manner as would be with the established expanded syntax.

This proposal only pertains to function invocations; function defintions are
unaffected by the syntax change. All existing valid syntax is unchanged.

Backwards Compatibility
=======================

Only new syntax is added which was previously syntactically erroreous. No
existing valid syntax is modified. As such, the changes proposed are fully
backwards compatible.

Security Implications
=====================

There are no security implications for this change.

How to Teach This
=================

Programmers may learn about this feature as an optional abbreviated syntax where
keyword arguments are taught. The
`Python Glossary <https://docs.python.org/3/glossary.html#term-argument>`__ and
`Tutorial <https://docs.python.org/3/tutorial/controlflow.html#keyword-arguments>`__
may be updated accordingly.

Prior Art
=========

Python already possesses a very similar feature in f-string interpolation where
``f'{x=}'`` is effectively expanded to ``f'x={x}'`` [1]_.

Several modern languages provide similar features during function invocation,
sometimes referred to as 'punning'. For example:

* In Ruby,  ``f(x:, y:)`` is syntactic sugar for ``f(x: x, y: y)`` [2]_.
* In ReasonML, ``f(~x, ~y)`` is syntactic sugar for ``f(~x=x, ~y=y)`` [3]_.
* In SystemVerilog, ``(.mult, .mop1, .data);`` is syntactic sugar for
  ``(.mult(mult), .mop1(mop1),  .data(data));`` [4]_.

Beyond function invocation specifically, more languages offer similar features:

* In OCaml, ``let+ x in …`` is syntactic sugar for ``let+ x = x in …`` [5]_.
* In JavaScript, ``{ x, y }`` is syntactic sugar for ``{x: x, y: y}`` [6]_.
* In Rust, ``User { x, y }`` is shorthand for ``User {x: x, y: y}`` [7]_.

Applicability
=============

We analysed popular Python libraries using
`this script <https://gist.github.com/joshuabambrick/a850d0e0050129b9252c748fa06c48b2>`__
to compute:

* The number of keyword arguments were of the form ``f(x=x)`` at invocation.
* The percentage of keyword arguments which had the form ``f(x=x)`` at
  invocation.
* The number of lines of code which could be saved by using this syntactic sugar
  to reduce the need for line wraps.

===================================================================== ================ ============== =============== =====================
Statistic                                                             `cpython <a_>`__ `numpy <b_>`__ `pandas <c_>`__ `scikit-learn <d_>`__
===================================================================== ================ ============== =============== =====================
Number of keyword arguments of the form ``f(x=x)`` at invocation      4,225            2,768          13,235          8,342
Percentage of keyword arguments of the form ``f(x=x)`` at invocation  11.06%           13.17%         17.24%          18.64%
Lines saved                                                           290              247            935             794
===================================================================== ================ ============== =============== =====================

.. _a: https://github.com/python/cpython/pull/111423/
.. _b: https://github.com/numpy/numpy/pull/25021/
.. _c: https://github.com/pandas-dev/pandas/pull/55744/
.. _d: https://github.com/scikit-learn/scikit-learn/pull/27680/

Based on this, we note that  the ``f(x=x)`` keyword argument pattern is
widespread, accounting for 10-20% of all keyword argument uses.

Proposed Syntax
===============

While this feature has been proposed on numerous occasions with several
different forms [8]_ [9]_ [10]_ [11]_ [12]_, [13]_ we have opted to advocate
for the ``f(x=)`` form for the following reasons:

* This feature has been proposed frequently over a ten year period with the
  ``f(x=)`` or ``f(=x)`` being by far the most common syntax  [8]_ [9]_ [13]_.
  This is a strong indicator that it is the obvious notation.
* The proposed syntax closely matches the f-string debug ``f'{var=}'`` syntax
  (established Pythonic style) and serves an almost identical purpose.
* The proposed syntax is exactly analogous to the Ruby keyword argument
  syntactic sugar [2]_.
* The syntax is easy to implement as it is simple syntactic sugar.
* When compared to the prefix form (see `Rejected Ideas`_), this syntax
  communicates "here is a parameter, go find its argument" which is more
  appropriate given the semantics of named arguments.
* `A poll of Python developers <https://discuss.python.org/t/syntactic-sugar-to-encourage-use-of-named-arguments/36217/130>`__
  indicates that this is the most popular syntax among those proposed.

Rejected Ideas
==============

Many alternative syntaxes have been proposed however no syntax other than
``f(=x)`` or ``f(x=)`` has garnered significant support. We here enumerate some
of the most popular proposed alternatives and why we ultimately reject them.

``f(=x)``
----------

In favour of this form:

* The prefix operator is more similar to the established ``*args`` and
  ``**kwargs`` syntax for function calls.
* It draws more attention to itself when arguments are arranged vertically. In
  particular, if the arguments are of different lengths it is harder to find the
  equal sign at the end. Moreover, since Python is read left to right, the use
  of this feature is clearer to the reader earlier on.

On the contrary:

* While the prefix version is visually louder, in practice, there is no need for
  this feature to shout its presence any more than a typical named argument. By
  the time we read to the ``=`` it is clear that the value is filled in
  automatically just as the value is clear in the typical keyword argument case.
* Semantically, this form communicates 'here is a value, fill in the parameter'.
* which is not what we want to convey.
* Less similar to f-string syntax.
* Less obvious that arbitrary expressions are invalid, e.g. ``f(=a+b)``.

``f(%x)`` or ``f(:x)`` or ``f(.x)``
-----------------------------------

Several flavours of this syntax have been proposed with the prefix form
substituting another character for ``=``. However, no such form has gained
traction and the choice of symbol seems arbitrary compared to ``=``.
Additionally, there is less precedent in terms of existing language features
(such as f-string) or other languages (such as Ruby).

``f(a, b, *, x)``
-----------------

On a few occasions the idea has been floated to borrow the syntax from
keyword-only function definitions. This is less arbitrary than ``f(%x)`` or
variants, but no less so than ``f(x=)``. 

However, we object that:

* For any given argument, it is less clear from local context whether it is
  positional or named. The ``*`` could easily be missed in a long argument list
  and named arguments may be read as positional or vice versa.
* It is unclear whether keyword arguments for which the value was not elided may
  follow the ``*``. If so, then their relative position will be inconsistent but
  if not, then an arbitrary grouping is enforced between different types of
  keyword arguments.

Objections
==========

There are only a few hard objections to the introduction of this syntactic
sugar. Most of those not in favour of this feature are in the camp of 'I
wouldn't use it'. However, over the extensive conversations about this feature,
the following objections were the most common:

The syntax is ugly
------------------

This objection is by far the most common. On the contrary, we argue that:

* This objection is is subjective and many community members disagree.
* A nearly-identical syntax is already established for f-strings.
* Programmers will, as ever, adjust over time.

The feature is confusing
------------------------

We argue that:

* Introducing new features typically has this impact temporarily.
* The syntax is very similar to the established ``f'{x=}'`` syntax.
* The feature and syntax are familiar from other popular modern languages.
* The expansion of ``x=`` to ``x=x`` is in fact a trivial feature and inherently
  significantly less complex than ``*arg`` and ``**kwarg`` expansion.
* This particular syntactic form has been independently proposed on numerous
  occasions, indicating that it is the most obvious [8]_ [9]_ [13]_.

The feature is not explicit
---------------------------

This is based on a misunderstanding of the Zen of Python. Keyword arguments are
fundamentally more explicit than positional ones where argument assignment is
only visible at the function definition. On the contrary, the proposed syntactic
sugar contains all the information as is conveyed by the established keyword
argument syntax but without the redundancy. Moreover, the introduction of this
syntactic sugar incentivises use of keyword arguments, making typical Python
codebases more explicit.

The feature adds another way of doing things
--------------------------------------------

The same argument can be made against all syntax changes. This is a simple
syntactic sugar, much as ``x += 1`` is sugar for ``x = x + 1`` when ``x`` is an
integer. This isn't tantamount to a 'new way' of passing arguments but a more
readable notation for the same way.

Renaming the variable in the calling context will break the code
----------------------------------------------------------------

A ``NameError`` would make the mistake abundantly clear. Moreover, text editors
could highlight this based on static analysis ‒ ``f(x=)`` is exactly equivalent
to writing ``f(x=x)``. If ``x`` does not exist, modern editors have no problem
highlighting the issue.

Recommendations
===============

As with any other language feature, the programmer should exercise their own
judgement about whether to use it in any given context. We do not recommend
enforcing a rule to use the feature in all cases where it may be applicable.

Reference Implementation
========================

`A proposed implementation <https://github.com/Hels15/cpython/tree/last-build>`_
for cpython has been provided by @Hels15.

References
==========

.. [1] Issue 36817: Add = to f-strings for easier debugging. - Python tracker
   https://bugs.python.org/issue36817
.. [2] Ruby keyword argument syntactic sugar
   https://www.ruby-lang.org/en/news/2021/12/25/ruby-3-1-0-released/#:~:text=Other%20Notable%20New%20Features
.. [3] ReasonML named argument punning
   https://reasonml.github.io/docs/en/function#:~:text=Named%20argument%20punning
.. [4] SystemVerilog Implicit Port Connections
   http://www.sunburst-design.com/papers/CummingsDesignCon2005_SystemVerilog_ImplicitPorts.pdf
.. [5] OCaml Short notation for variable bindings (let-punning)
   https://v2.ocaml.org/manual/bindingops.html#ss:letops-punning
.. [6] JavaScript Object Initializer
   https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer
.. [7] Rust Using the Field Init Shorthand
   https://doc.rust-lang.org/book/ch05-01-defining-structs.html#using-the-field-init-shorthand-when-variables-and-fields-have-the-same-name
.. [8] Short form for keyword arguments and dicts (2013)
   https://mail.python.org/archives/list/python-ideas@python.org/thread/SQKZ273MYAY5WNIQRGEDLYTKVORVKNEZ/#LXMU22F63VPCF7CMQ4OQRH2CG6H7WCQ6
.. [9] Keyword arguments self-assignment (2020)
   https://mail.python.org/archives/list/python-ideas@python.org/thread/SIMIOC7OW6QKLJOTHJJVNNBDSXDE2SGV/
.. [10] Shorthand notation of dict literal and function call (2020)
   https://discuss.python.org/t/shorthand-notation-of-dict-literal-and-function-call/5697/1
.. [11] Allow identifiers as keyword arguments at function call site (extension
   of PEP 3102?) (2023)
   https://discuss.python.org/t/allow-identifiers-as-keyword-arguments-at-function-call-site-extension-of-pep-3102/31677
.. [12] Shorten Keyword Arguments with Implicit Notation: foo(a=a, b=b) to foo(.a, .b) (2023)
   https://discuss.python.org/t/shorten-keyword-arguments-with-implicit-notation-foo-a-a-b-b-to-foo-a-b/33080
.. [13] Syntactic sugar to encourage use of named arguments (2023)
   https://discuss.python.org/t/syntactic-sugar-to-encourage-use-of-named-arguments/36217

Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.