diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 4ee38a037..a90a99d75 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -615,7 +615,7 @@ peps/pep-0732.rst @Mariatta peps/pep-0733.rst @encukou @vstinner @zooba @iritkatriel peps/pep-0734.rst @ericsnowcurrently peps/pep-0735.rst @brettcannon -peps/pep-0736.rst @gvanrossum @Rosuav +peps/pep-0736.rst @Rosuav peps/pep-0737.rst @vstinner peps/pep-0738.rst @encukou peps/pep-0740.rst @dstufft diff --git a/peps/pep-0736.rst b/peps/pep-0736.rst index ee3075719..65c7c0e11 100644 --- a/peps/pep-0736.rst +++ b/peps/pep-0736.rst @@ -2,21 +2,20 @@ PEP: 736 Title: Shorthand syntax for keyword arguments at invocation Author: Joshua Bambrick , Chris Angelico -Sponsor: Guido van Rossum Discussions-To: https://discuss.python.org/t/pep-736-shorthand-syntax-for-keyword-arguments-at-invocation/43432 Status: Draft Type: Standards Track Created: 28-Nov-2023 -Python-Version: 3.13 +Python-Version: 3.14 Post-History: `14-Oct-2023 `__, `17-Jan-2024 `__, Abstract ======== -This PEP proposes introducing syntactic sugar ``f(x=)`` for the common -pattern where a named argument is the same as the name of the variable -corresponding to its value ``f(x=x)``. +This PEP proposes introducing syntactic sugar ``f(x=)`` for the pattern +where a named argument is the same as the name of the variable corresponding to +its value ``f(x=x)``. Motivation ========== @@ -33,9 +32,8 @@ Consider the following call: ) The case of a keyword argument name matching the variable name of its value is -prevalent among all major Python libraries. This verbosity and redundancy -discourages use of named arguments and reduces readability by increasing visual -noise. +prevalent among Python libraries. This verbosity and redundancy discourages +use of named arguments and reduces readability by increasing visual noise. Rationale ========= @@ -49,18 +47,17 @@ and visual noise. We contend that a simple syntactic sugar used to simplify this common pattern which would confer numerous benefits: -Encourages use of named variables +Encourages use of named arguments --------------------------------- -This syntax would encourage the use of named variables, thereby increasing -readability (*explicit is better than implicit*) and reducing bugs from argument -transposition. +This syntax would encourage the use of named arguments, thereby increasing +readability and reducing bugs from argument transposition. Reduces verbosity ----------------- By minimising visual noise and in some cases lines of code, we can increase -readability (*readability counts*). +readability. Encourages consistent variable names ------------------------------------ @@ -68,13 +65,42 @@ Encourages consistent variable names A common problem is that semantically identical variables have different names depending on their contexts. This syntax would encourage authors to use the same variable name when calling a function as the argument name, which would increase -consistency of variable names used and hence also *readability*. +consistency of variable names used and hence also readability. + +Highlights arguments not following this pattern +----------------------------------------------- + +With the current syntax, function calls where many arguments are forwarded from +the local context can make other argument values easy to miss due to the visual +noise. For example:: + + add_middleware( + excluded_urls=excluded_urls, + server_request=server_request, + client_request=client_request, + client_response=client_response, + span_details=_get_span_details(), + tracer=tracer, + meter=meter, + ) + +With this syntax, the exceptional arguments become easier to identify:: + + add_middleware( + excluded_urls=, + server_request=, + client_request=, + client_response=, + span_details=_get_span_details(), + tracer=, + meter=, + ) Applicability to dictionary construction ---------------------------------------- -This syntax can be applied to dictionary construction where a similar -pattern frequently occurs (where dictionary keys are identical the names of the +This syntax can be applied to dictionary construction where a similar pattern +frequently occurs (where dictionary keys are identical the names of the variables assigned as their values), ``{"x": x, "y": y}`` or ``dict(x=x, y=y)``. With this feature, this can now also be trivially written as ``dict(x=, y=)``. Whether to further support similar syntax in dictionary literals is an open @@ -104,13 +130,13 @@ Will be interpreted exactly equivalently to following in existing syntax: If no variable matches that name in the invocation scope, a ``NameError`` is raised in an identical manner as would be with the established expanded syntax. -This proposal only pertains to function invocations; function defintions are +This proposal only pertains to function invocations; function definitions are unaffected by the syntax change. All existing valid syntax is unchanged. Backwards Compatibility ======================= -Only new syntax is added which was previously syntactically erroreous. No +Only new syntax is added which was previously syntactically erroneous. No existing valid syntax is modified. As such, the changes proposed are fully backwards compatible. @@ -132,26 +158,35 @@ Prior Art ========= Python already possesses a very similar feature in f-string interpolation where -``f'{x=}'`` is effectively expanded to ``f'x={x}'`` [1]_. +``f'{x=}'`` is effectively expanded to ``f'x={x}'`` (see +`related GitHub issue `__). Several modern languages provide similar features during function invocation, sometimes referred to as 'punning'. For example: -* In Ruby, ``f(x:, y:)`` is syntactic sugar for ``f(x: x, y: y)`` [2]_. -* In ReasonML, ``f(~x, ~y)`` is syntactic sugar for ``f(~x=x, ~y=y)`` [3]_. +* In Ruby, ``f(x:, y:)`` is syntactic sugar for ``f(x: x, y: y)``. See the + `Ruby 3.1.0 release notes `__ (search for "keyword arguments"). +* In ReasonML, ``f(~x, ~y)`` is syntactic sugar for ``f(~x=x, ~y=y)``. See the + `ReasonML function documentation `__ (search for "punning"). * In SystemVerilog, ``(.mult, .mop1, .data);`` is syntactic sugar for - ``(.mult(mult), .mop1(mop1), .data(data));`` [4]_. + ``(.mult(mult), .mop1(mop1), .data(data));``. See + `SystemVerilog Implicit Port Connections `__. +* In Jakt, ``f(x, y)`` is syntactic sugar for ``f(x: x, y: y)``. See + `The Jakt programming language `__. Beyond function invocation specifically, more languages offer similar features: -* In OCaml, ``let+ x in …`` is syntactic sugar for ``let+ x = x in …`` [5]_. -* In JavaScript, ``{ x, y }`` is syntactic sugar for ``{x: x, y: y}`` [6]_. -* In Rust, ``User { x, y }`` is shorthand for ``User {x: x, y: y}`` [7]_. +* In OCaml, ``let+ x in …`` is syntactic sugar for ``let+ x = x in …``. See + `OCaml Short notation for variable bindings (let-punning) `__. +* In JavaScript, ``{ x, y }`` is syntactic sugar for ``{x: x, y: y}``. See + `JavaScript Object Initializer `__. +* In Rust, ``User { x, y }`` is shorthand for ``User {x: x, y: y}``. See + `Rust Using the Field Init Shorthand `__. Applicability ============= -We analysed popular Python libraries using +We analysed popular Python libraries from the last few years using `this script `__ to compute: @@ -161,18 +196,22 @@ to compute: * The number of lines of code which could be saved by using this syntactic sugar to reduce the need for line wraps. -===================================================================== ================ ============== =============== ===================== -Statistic `cpython `__ `numpy `__ `pandas `__ `scikit-learn `__ -===================================================================== ================ ============== =============== ===================== -Number of keyword arguments of the form ``f(x=x)`` at invocation 4,225 2,768 13,235 8,342 -Percentage of keyword arguments of the form ``f(x=x)`` at invocation 11.06% 13.17% 17.24% 18.64% -Lines saved 290 247 935 794 -===================================================================== ================ ============== =============== ===================== +The purpose of this exercise was to compute statistics about the prevalence of +this pattern and should not be interpreted as a recommendation that the proposed +syntactic sugar should be applied universally. -.. _a: https://github.com/python/cpython/pull/111423/ -.. _b: https://github.com/numpy/numpy/pull/25021/ -.. _c: https://github.com/pandas-dev/pandas/pull/55744/ -.. _d: https://github.com/scikit-learn/scikit-learn/pull/27680/ +===================================================================== =============== ================ ============== ============== +Statistic `polars `__ `fastapi `__ `rich `__ `httpx `__ +===================================================================== =============== ================ ============== ============== +Number of keyword arguments of the form ``f(x=x)`` at invocation 1,654 1,408 566 759 +Percentage of keyword arguments of the form ``f(x=x)`` at invocation 15.83% 28.11% 15.74% 45.13% +Lines saved 170 35 62 117 +===================================================================== =============== ================ ============== ============== + +.. _a: https://github.com/joshuabambrick/polars/pull/1 +.. _b: https://github.com/joshuabambrick/fastapi/pull/1 +.. _c: https://github.com/joshuabambrick/rich/pull/1 +.. _d: https://github.com/joshuabambrick/httpx/pull/1 Based on this, we note that the ``f(x=x)`` keyword argument pattern is widespread, accounting for 10-20% of all keyword argument uses. @@ -181,16 +220,17 @@ Proposed Syntax =============== While this feature has been proposed on numerous occasions with several -different forms [8]_ [9]_ [10]_ [11]_ [12]_, [13]_ we have opted to advocate +different forms [1]_ [2]_ [3]_ [4]_ [5]_, [6]_ we have opted to advocate for the ``f(x=)`` form for the following reasons: * This feature has been proposed frequently over a ten year period with the - ``f(x=)`` or ``f(=x)`` being by far the most common syntax [8]_ [9]_ [13]_. + ``f(x=)`` or ``f(=x)`` being by far the most common syntax [1]_ [2]_ [6]_. This is a strong indicator that it is the obvious notation. * The proposed syntax closely matches the f-string debug ``f'{var=}'`` syntax (established Pythonic style) and serves an almost identical purpose. * The proposed syntax is exactly analogous to the Ruby keyword argument - syntactic sugar [2]_. + syntactic sugar. See the + `Ruby 3.1.0 release notes `__ (search for "keyword arguments"). * The syntax is easy to implement as it is simple syntactic sugar. * When compared to the prefix form (see `Rejected Ideas`_), this syntax communicates "here is a parameter, go find its argument" which is more @@ -205,6 +245,33 @@ Many alternative syntaxes have been proposed however no syntax other than ``f(=x)`` or ``f(x=)`` has garnered significant support. We here enumerate some of the most popular proposed alternatives and why we ultimately reject them. +``f(a, b, *, x)`` +----------------- + +On a few occasions the idea has been floated to borrow the syntax from +keyword-only function definitions. + +In favour of this proposal: + +* This syntax is familiar from its use to require keyword-only arguments in + function definitions. +* `A poll of Python developers `__ + indicates that this is the second most popular syntax among those proposed. + +However, we object that: + +* For any given argument, it is less clear from local context whether it is + positional or named. The ``*`` could easily be missed in a long argument list + and named arguments may be read as positional or vice versa. +* It is unclear whether keyword arguments for which the value was not elided may + follow the ``*``. If so, then their relative position will be inconsistent but + if not, then an arbitrary grouping is enforced between different types of + keyword arguments and reordering would be necessary if only one name was + changed. +* The use of ``*`` in function calls is established and this proposal would + introduce a new effect which could cause confusion. For example, + ``f(a, *x, y)`` would mean something different than ``f(a, *, x, y)``. + ``f(=x)`` ---------- @@ -223,10 +290,10 @@ On the contrary: this feature to shout its presence any more than a typical named argument. By the time we read to the ``=`` it is clear that the value is filled in automatically just as the value is clear in the typical keyword argument case. -* Semantically, this form communicates 'here is a value, fill in the parameter'. -* which is not what we want to convey. -* Less similar to f-string syntax. -* Less obvious that arbitrary expressions are invalid, e.g. ``f(=a+b)``. +* Semantically, this form communicates 'here is a value, fill in the parameter' + which is not what we want to convey. +* It is less similar to f-string syntax. +* It is less obvious that arbitrary expressions are invalid, e.g. ``f(=a + b)``. ``f(%x)`` or ``f(:x)`` or ``f(.x)`` ----------------------------------- @@ -237,23 +304,6 @@ traction and the choice of symbol seems arbitrary compared to ``=``. Additionally, there is less precedent in terms of existing language features (such as f-string) or other languages (such as Ruby). -``f(a, b, *, x)`` ------------------ - -On a few occasions the idea has been floated to borrow the syntax from -keyword-only function definitions. This is less arbitrary than ``f(%x)`` or -variants, but no less so than ``f(x=)``. - -However, we object that: - -* For any given argument, it is less clear from local context whether it is - positional or named. The ``*`` could easily be missed in a long argument list - and named arguments may be read as positional or vice versa. -* It is unclear whether keyword arguments for which the value was not elided may - follow the ``*``. If so, then their relative position will be inconsistent but - if not, then an arbitrary grouping is enforced between different types of - keyword arguments. - Objections ========== @@ -267,7 +317,7 @@ The syntax is ugly This objection is by far the most common. On the contrary, we argue that: -* This objection is is subjective and many community members disagree. +* This objection is subjective and many community members disagree. * A nearly-identical syntax is already established for f-strings. * Programmers will, as ever, adjust over time. @@ -282,18 +332,27 @@ We argue that: * The expansion of ``x=`` to ``x=x`` is in fact a trivial feature and inherently significantly less complex than ``*arg`` and ``**kwarg`` expansion. * This particular syntactic form has been independently proposed on numerous - occasions, indicating that it is the most obvious [8]_ [9]_ [13]_. + occasions, indicating that it is the most obvious [1]_ [2]_ [6]_. The feature is not explicit --------------------------- -This is based on a misunderstanding of the Zen of Python. Keyword arguments are -fundamentally more explicit than positional ones where argument assignment is -only visible at the function definition. On the contrary, the proposed syntactic -sugar contains all the information as is conveyed by the established keyword -argument syntax but without the redundancy. Moreover, the introduction of this -syntactic sugar incentivises use of keyword arguments, making typical Python -codebases more explicit. +We recognise that, in an obvious sense, the argument value is 'implicit' in this +proposed syntax. However, we do not think that this is what the Zen of Python is +aiming to discourage. + +In the sense that we take the Zen to be referring to, keyword arguments (for +example) are more explicit than positional arguments where the argument name is +omitted and impossible to tell from the local context. Conversely, the syntactic +sugar for integers ``x += 1`` is not more implicit than ``x = x + 1`` in this +sense, even though the variable is omitted from the right hand side, because it +is immediately obvious from the local context what it is. + +The syntax proposed in this PEP is much more closely analogous to the ``x += 1`` +example (although simpler since we do not propose to introduce a new operation). +Moreover, the introduction of this syntactic sugar should encourage the use of +keyword arguments over positional ones, making typical Python codebases more +explicit in general. The feature adds another way of doing things -------------------------------------------- @@ -306,17 +365,115 @@ readable notation for the same way. Renaming the variable in the calling context will break the code ---------------------------------------------------------------- -A ``NameError`` would make the mistake abundantly clear. Moreover, text editors -could highlight this based on static analysis ‒ ``f(x=)`` is exactly equivalent -to writing ``f(x=x)``. If ``x`` does not exist, modern editors have no problem -highlighting the issue. +A ``NameError`` would make the mistake clear in most cases. There may be +confusion if a variable from a broader scope has the same name as the original +variable, so no ``NameError`` would be raised. However, this issue can also +occur with keyword arguments using the current syntax (arguably, this syntactic +sugar could make it harder to spot). Moreover, having variables with the same +name in different scopes is broadly considered bad practice and discouraged by +linters. -Recommendations -=============== +Code editors could highlight the issue based on static analysis - ``f(x=)`` is +exactly equivalent to writing ``f(x=x)``. If ``x`` does not exist, modern +editors have no problem highlighting the issue. + +This syntax increases coupling +------------------------------ + +We recognise that, as ever, all syntax has the potential for misuse and so +should be applied judiciously to improve codebases. In this case, if a parameter +and its value have the same semantics in both contexts, that may suggest that +using this new syntax is appropriate and will help ameliorate the risk of +unintentional desynchronisation which harms readability. + +However, if the two variables have different semantics, that may suggest that +this feature should not be used to encourage consistency or even that they +should be renamed. + +Recommendations for using this syntax +===================================== As with any other language feature, the programmer should exercise their own -judgement about whether to use it in any given context. We do not recommend -enforcing a rule to use the feature in all cases where it may be applicable. +judgement about whether it is prudent to use it in any given context. We do not +recommend enforcing a rule to use the feature in all cases where it may be +applicable. + +As described `above `__, we propose that a +reasonable rule of thumb would be to use this in cases where a parameter and its +argument have the same semantics in order to reduce unintentional +desynchronisation without causing inappropriate coupling. + +Impact on editing +================= + +Using a plain text editor +------------------------- + +Editing with a plain text editor should generally be unaffected. + +When renaming a variable using a 'Find-Replace' method, where this syntax is +used the developer will come across the function argument at invocation (as they +would if this syntax was not used). At that point, they can as usual decide +whether to update the argument as well or expand to the full ``f(x=x)`` syntax. + +As with the current syntax, a 'Find-Replace All' method would fail since the +keyword argument would not exist at function definition, in the vast majority +of cases. + +If the developer leaves the argument name unchanged and forgets to update its +value, a ``NameError`` will typically be raised as described +`above `__. + +Proposals for IDEs +------------------ + +In response to community feedback, we include some suggestions regarding how +IDEs could handle this syntax. However, we of course defer to the domain experts +developing IDEs to use their own discretion. + +Most considerations are made simple by recognising that ``f(x=)`` is just +syntactic sugar for ``f(x=x)`` and should be treated the same as at present. + +Highlighting NameErrors +''''''''''''''''''''''' + +IDEs typically offer a feature to highlight code that may cause a ``NameError``. +We recommend that this syntax be treated similarly to the expanded form +``f(x=x)`` to identify and highlight cases where the elided value variable may +not exist. What visual cue may be used to highlight these cases may be the same +or different from that which would be used with the current syntax, depending on +the IDE. + +Jump to definition +'''''''''''''''''' + +There are a few possible ways that a 'jump to definition' feature could be +implemented depending on the caret/cursor position. + +One option is to: + +* Jump to the argument in the function definition if the caret/cursor is on the + argument +* Jump to the definition of the elided variable if the caret/cursor is on the + character following the ``=`` in our proposed syntax. + +Another, potentially complementary, option would be to expand the syntax +visually on mouseover and enable a ``Ctrl+Click`` (or ``Cmd+Click``) to the +definition of the variable. + +Rename symbol +''''''''''''' + +There are a few ways that IDEs may wish to support a 'Rename symbol' feature for +this syntax. For example, if the argument is being renamed, the IDE may: + +* Also rename the variable used as its value in each calling context where this + syntax is used +* Expand to use the full syntax to pass the variable used as its value +* Prompt the developer to select between the two above options + +The last option here seems most preferable in order to reduce unintentional +desynchronisation of names while highlighting the user to the changes. Reference Implementation ======================== @@ -327,32 +484,18 @@ for cpython has been provided by @Hels15. References ========== -.. [1] Issue 36817: Add = to f-strings for easier debugging. - Python tracker - https://bugs.python.org/issue36817 -.. [2] Ruby keyword argument syntactic sugar - https://www.ruby-lang.org/en/news/2021/12/25/ruby-3-1-0-released/#:~:text=Other%20Notable%20New%20Features -.. [3] ReasonML named argument punning - https://reasonml.github.io/docs/en/function#:~:text=Named%20argument%20punning -.. [4] SystemVerilog Implicit Port Connections - http://www.sunburst-design.com/papers/CummingsDesignCon2005_SystemVerilog_ImplicitPorts.pdf -.. [5] OCaml Short notation for variable bindings (let-punning) - https://v2.ocaml.org/manual/bindingops.html#ss:letops-punning -.. [6] JavaScript Object Initializer - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer -.. [7] Rust Using the Field Init Shorthand - https://doc.rust-lang.org/book/ch05-01-defining-structs.html#using-the-field-init-shorthand-when-variables-and-fields-have-the-same-name -.. [8] Short form for keyword arguments and dicts (2013) +.. [1] Short form for keyword arguments and dicts (2013) https://mail.python.org/archives/list/python-ideas@python.org/thread/SQKZ273MYAY5WNIQRGEDLYTKVORVKNEZ/#LXMU22F63VPCF7CMQ4OQRH2CG6H7WCQ6 -.. [9] Keyword arguments self-assignment (2020) +.. [2] Keyword arguments self-assignment (2020) https://mail.python.org/archives/list/python-ideas@python.org/thread/SIMIOC7OW6QKLJOTHJJVNNBDSXDE2SGV/ -.. [10] Shorthand notation of dict literal and function call (2020) +.. [3] Shorthand notation of dict literal and function call (2020) https://discuss.python.org/t/shorthand-notation-of-dict-literal-and-function-call/5697/1 -.. [11] Allow identifiers as keyword arguments at function call site (extension +.. [4] Allow identifiers as keyword arguments at function call site (extension of PEP 3102?) (2023) https://discuss.python.org/t/allow-identifiers-as-keyword-arguments-at-function-call-site-extension-of-pep-3102/31677 -.. [12] Shorten Keyword Arguments with Implicit Notation: foo(a=a, b=b) to foo(.a, .b) (2023) +.. [5] Shorten Keyword Arguments with Implicit Notation: foo(a=a, b=b) to foo(.a, .b) (2023) https://discuss.python.org/t/shorten-keyword-arguments-with-implicit-notation-foo-a-a-b-b-to-foo-a-b/33080 -.. [13] Syntactic sugar to encourage use of named arguments (2023) +.. [6] Syntactic sugar to encourage use of named arguments (2023) https://discuss.python.org/t/syntactic-sugar-to-encourage-use-of-named-arguments/36217 Copyright