python-peps/pep-0671.rst

PEP: 671
Title: Syntax for late-bound function argument defaults
Author: Chris Angelico <rosuav@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 24-Oct-2021
Python-Version: 3.11
Post-History: 24-Oct-2021


Abstract
========

Function parameters can have default values which are calculated during
function definition and saved. This proposal introduces a new form of
argument default, defined by an expression to be evaluated at function
call time.


Motivation
==========

Optional function arguments, if omitted, often have some sort of logical
default value. When this value depends on other arguments, or needs to be
reevaluated each function call, there is currently no clean way to state
this in the function header.

Currently-legal idioms for this include::

    # Very common: Use None and replace it in the function
    def bisect_right(a, x, lo=0, hi=None, *, key=None):
        if hi is None:
            hi = len(a)

    # Also well known: Use a unique custom sentinel object
    _USE_GLOBAL_DEFAULT = object()
    def connect(timeout=_USE_GLOBAL_DEFAULT):
        if timeout is _USE_GLOBAL_DEFAULT:
            timeout = default_timeout

    # Unusual: Accept star-args and then validate
    def add_item(item, *optional_target):
        if not optional_target:
            target = []
        else:
            target = optional_target[0]

In each form, ``help(function)`` fails to show the true default value. Each
one has additional problems, too; using ``None`` is only valid if None is not
itself a plausible function parameter, the custom sentinel requires a global
constant; and use of star-args implies that more than one argument could be
given.

Specification
=============

Function default arguments can be defined using the new ``=>`` notation::

    def bisect_right(a, x, lo=0, hi=>len(a), *, key=None):
    def connect(timeout=>default_timeout):
    def add_item(item, target=>[]):

The expression is saved in its source code form for the purpose of inspection,
and bytecode to evaluate it is prepended to the function's body.

Notably, the expression is evaluated in the function's run-time scope, NOT the
scope in which the function was defined (as are early-bound defaults). This
allows the expression to refer to other arguments.

Multiple late-bound arguments are evaluated from left to right, and can refer
to previously-defined values. Order is defined by the function, regardless of
the order in which keyword arguments may be passed. Using names of other
arguments is a SyntaxError at function definition time::

    def spaminate(sausage=>eggs + 1, eggs=>sausage - 1): # SyntaxError
    def selfref(spam=>spam): # SyntaxError
    def frob(n=>len(items), items=[]): # SyntaxError


Choice of spelling
------------------

Our chief syntax proposal is ``name=>expression`` -- our two syntax proposals
... ahem. Amongst our potential syntaxes are::

    def bisect(a, hi=>len(a)):
    def bisect(a, hi=:len(a)):
    def bisect(a, hi?=len(a)):
    def bisect(a, hi!=len(a)):
    def bisect(a, hi=\len(a)):
    def bisect(a, hi=`len(a)`):
    def bisect(a, hi=@len(a)):

Since default arguments behave largely the same whether they're early or late
bound, the preferred syntax is very similar to the existing early-bind syntax.
The alternatives offer little advantage over the preferred one.

How to Teach This
=================

Early-bound default arguments should always be taught first, as they are the
simpler and more efficient way to evaluate arguments. Building on them, late
bound arguments are broadly equivalent to code at the top of the function::

    def add_item(item, target=>[]):

    # Equivalent pseudocode:
    def add_item(item, target=<OPTIONAL>):
        if target was omitted: target = []


Interaction with other open PEPs
================================

PEP 661 attempts to solve one of the same problems as this does. It seeks to
improve the documentation of sentinel values in default arguments, where this
proposal seeks to remove the need for sentinels in many common cases. PEP 661
is able to improve documentation in arbitrarily complicated functions (it
cites ``traceback.print_exception`` as its primary motivation, which has two
arguments which must both-or-neither be specified); on the other hand, many
of the common cases would no longer need sentinels if the true default could
be defined by the function. Additionally, dedicated sentinel objects can be
used as dictionary lookup keys, where PEP 671 does not apply.


Open Issues
===========

- yield/await? Will they cause problems? Might end up being a non-issue.

- annotations? They go before the default, so is there any way an anno could
  want to end with ``=>``?

- Rather than banning future refs, these could be permitted, at the price of
  harder-to-explain semantics. Arguments would be resolved first with those
  passed and those with early-bound defaults, and then late-bound ones would
  be evaluated, left-to-right; the consequences for getting it wrong would
  then be UnboundLocalError at call time, rather than SyntaxError from the
  function definition.


Implementation details
======================

The following relates to the reference implementation, and is not necessarily
part of the specification.

An **argument default**, rather than being an arbitrary value, is now a tuple
of 1-2 values. The first value is descriptive text and may be ``None``; the
compiler is free to put the source code for the default here, or None, on any
basis. (The reference implementation omits them from early-bound defaults, and
retains them for late-bound ones.) If there is a second value, it is an early
bound default value and will be used as the function parameter when none is
given.

When there is no second value in the tuple, and the parameter is omitted, the
function will begin with the parameter unbound. The function begins by testing
for each parameter with a late-bound default, and if unbound, evaluates the
original expression.

Inspecting the function (eg ``help()``) will use the provided description
where available, falling back on the repr of the value, or if there is none,
report "=> <calculated>".

Costs
-----

Every function default argument must now be wrapped in a tuple. This adds 56
bytes (CPython 3.11, 64-bit Linux), but where the same value is used in many
places (eg ``None``), this tuple can be shared.

Mapping arguments to parameters incurs the cost of tuple unpacking.

Constructing functions manually using FunctionType requires additional checks.

Backward incompatibility
------------------------

Inspecting ``spam.__defaults__`` shows a tuple of tuples rather than a tuple
of values. Similarly, ``spam.__kwdefaults__`` is a dict of tuples.

References
==========

https://github.com/rosuav/cpython/tree/pep-671

Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: