python-peps/pep-0457.txt

PEP: 457
Title: Syntax For Positional-Only Parameters
Version: $Revision$
Last-Modified: $Date$
Author: Larry Hastings <larry@hastings.org>
Discussions-To: Python-Dev <python-dev@python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 08-Oct-2013


========
Overview
========

This PEP proposes a syntax for positional-only parameters in Python.
Positional-only parameters are parameters without an externally-usable
name; when a function accepting positional-only parameters is called,
positional arguments are mapped to these parameters based solely on
their position.

=========
Rationale
=========

Python has always supported positional-only parameters.
Early versions of Python lacked the concept of specifying
parameters by name, so naturally all parameters were
positional-only.  This changed around Python 1.0, when
all parameters suddenly became positional-or-keyword.
But, even in current versions of Python, many CPython
"builtin" functions still only accept positional-only
arguments.

Functions implemented in modern Python can accept
an arbitrary number of positional-only arguments, via the
variadic ``*args`` parameter.  However, there is no Python
syntax to specify accepting a specific number of
positional-only parameters.  Put another way, there are
many builtin functions whose signatures are simply not
expressable with Python syntax.

This PEP proposes a backwards-compatible syntax that should
permit implementing any builtin in pure Python code.

-----------------------------------------------------
Positional-Only Parameter Semantics In Current Python
-----------------------------------------------------

There are many, many examples of builtins that only
accept positional-only parameters.  The resulting
semantics are easily experienced by the Python
programmer--just try calling one, specifying its
arguments by name::

    >>> pow(x=5, y=3)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: pow() takes no keyword arguments

In addition, there are some functions with particularly
interesting semantics:

  * ``range()``, which accepts an optional parameter
    to the *left* of its required parameter. [#RANGE]_

  * ``dict()``, whose mapping/iterator parameter is optional and
    semantically must be positional-only.  Any externally
    visible name for this parameter would occlude
    that name going into the ``**kwarg`` keyword variadic
    parameter dict! [#DICT]_

Obviously one can simulate any of these in pure Python code
by accepting ``(*args, **kwargs)`` and parsing the arguments
by hand.  But this results in a disconnect between the
Python function's signature and what it actually accepts,
not to mention the work of implementing said argument parsing.

==========
Motivation
==========

This PEP does not propose we implement positional-only
parameters in Python.  The goal of this PEP is simply
to define the syntax, so that:

    * Documentation can clearly, unambiguously, and
      consistently express exactly how the arguments
      for a function will be interpreted.

    * The syntax is reserved for future use, in case
      the community decides someday to add positional-only
      parameters to the language.

    * Argument Clinic can use a variant of the syntax
      as part of its input when defining
      the arguments for built-in functions.

=================================================================
The Current State Of Documentation For Positional-Only Parameters
=================================================================

The documentation for positional-only parameters is incomplete
and inconsistent:

* Some functions denote optional groups of positional-only arguments
  by enclosing them in nested square brackets. [#BORDER]_

* Some functions denote optional groups of positional-only arguments
  by presenting multiple prototypes with varying numbers of
  arguments. [#SENDFILE]_

* Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_

One more important idea to consider: currently in the documentation
there's no way to tell whether a function takes positional-only
parameters.  ``open()`` accepts keyword arguments, ``ord()`` does
not, but there is no way of telling just by reading the
documentation that this is true.

====================
Syntax And Semantics
====================

From the "ten-thousand foot view", and ignoring ``*args`` and ``**kwargs``
for now, the grammar for a function definition currently looks like this::

    def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that perspective, the new syntax for functions would look
like this::

    def name(positional_only_parameters, /, positional_or_keyword_parameters,
             *, keyword_only_parameters):

All parameters before the ``/`` are positional-only.  If ``/`` is
not specified in a function signature, that function does not
accept any positional-only parameters.

Positional-only parameters can be optional, but the mechanism is
significantly different from positional-or-keyword or keyword-only
parameters.  Positional-only parameters don't accept default
values.  Instead, positional-only parameters can be specified
in optional "groups".  Groups of parameters are surrounded by
square brackets, like so::

    def addch([y, x,] ch, [attr,] /):

Positional-only parameters that are not in an option group are
"required" positional-only parameters.  All "required" positional-only
parameters must be contiguous.

Parameters in an optional group accept arguments in a group; you
must provide arguments either for all of the them or for none of them.
Using the example of ``addch()`` above, you could not call ``addch()``
in such a way that ``x`` was specified but ``y`` was not (and vice versa).
The mapping of positional parameters to optional groups is done
based on fitting the number of parameters to groups.  Based on the
above definition, ``addch()`` would assign arguments to parameters
in the following way:

    +-------------------+------------------------------+
    |Number of arguments|Parameter assignment          |
    +-------------------+------------------------------+
    |0                  |*raises an exception*         |
    +-------------------+------------------------------+
    |1                  |``ch``                        |
    +-------------------+------------------------------+
    |2                  |``ch``, ``attr``              |
    +-------------------+------------------------------+
    |3                  |``y``, ``x``, ``ch``          |
    +-------------------+------------------------------+
    |4                  |``y``, ``x``, ``ch``, ``attr``|
    +-------------------+------------------------------+
    |5 or more          |*raises an exception*         |
    +-------------------+------------------------------+


More semantics of positional-only parameters:

  * Although positional-only parameter technically have names,
    these names are internal-only; positional-only parameters
    are *never* externally addressable by name.  (Similarly
    to ``*args`` and ``**kwargs``.)

  * It's possible to nest option groups.

  * If there are no required parameters, all option groups behave
    as if they're to the right of the required parameter group.

  * For clarity and consistency, the comma for a parameter always
    comes immediately after the parameter name.  It's a syntax error
    to specify a square bracket between the name of a parameter and
    the following comma.  (This is far more readable than putting
    the comma outside the square bracket, particularly for nested
    groups.)

  * If there are arguments after the ``/``, then you must specify
    a comma after the ``/``, just as there is a comma
    after the ``*`` denoting the shift to keyword-only parameters.

  * This syntax has no effect on ``*args`` or ``**kwargs``.

It's possible to specify a function prototype where the mapping
of arguments to parameters is ambiguous.  Consider::

    def range([start,] stop, [range,] /):

Python disambiguates these situations by preferring optional groups
to the *left* of the required group.

======================
Additional Limitations
======================

Argument Clinic uses a form of this syntax for specifying
builtins.  It imposes further limitations that are
theoretically unnecessary but make the implementation
easier.  Specifically:

* A function that has positional-only parameters currently
  cannot have any other kind of parameter.  (This will
  probably be relaxed slightly in the near future.)

* Multiple option groups on either side of the required
  positional-only parameters must be nested, with the
  nesting getting deeper the further away the group is
  from the required positional-parameter group.

  Put another way:
  all the left-brackets for option groups to the
  left of the required group must be specified contiguously,
  and
  all the right-brackets for option groups to the
  right of the required group must be specified contiguously.


==============================
Notes For A Future Implementor
==============================

If we decide to implement positional-only parameters in a future
version of Python, we'd have to do some additional work to preserve
their semantics.  The problem: how do we inform a parameter that
no value was passed in for it when the function was called?

The obvious solution: add a new singleton constant to Python
that is passed in when a parameter is not mapped to an argument.
I propose that the value be called called ``undefined``,
and be a singleton of a special class called ``Undefined``.
If a positional-only parameter did not receive an argument
when called, its value would be set to ``undefined``.

But this raises a further problem.  How do can we tell the
difference between "this positional-only parameter did not
receive an argument" and "the caller passed in ``undefined``
for this parameter"?

It'd be nice to make it illegal to pass ``undefined`` in
as an argument to a function--to, say, raise an exception.
But that would slow Python down, and the "consenting adults"
rule appears applicable here.  So making it illegal should
probably be strongly discouraged but not outright prevented.

However, it should be allowed (and encouraged) for user
functions to specify ``undefined`` as a default value for
parameters.

====================
Unresolved Questions
====================

There are three types of parameters in Python:

  1. positional-only parameters,
  2. positional-or-keyword parameters, and
  3. keyword-only parameters.

Python allows functions to have both 2 and 3.  And some
builtins (e.g. range) have both 1 and 3.  Does it make
sense to have functions that have both 1 and 2?  Or
all of the above?


======
Thanks
======

Credit for the use of '/' as the separator between positional-only and positional-or-keyword
parameters goes to Guido van Rossum, in a proposal from 2012. [#GUIDO]_

Credit for making left option groups higher precedence goes to
Nick Coghlan. (Conversation in person at PyCon US 2013.)

.. [#DICT]
    http://docs.python.org/3/library/stdtypes.html#dict
    
.. [#RANGE]
    http://docs.python.org/3/library/functions.html#func-range
    
.. [#BORDER]
    http://docs.python.org/3/library/curses.html#curses.window.border

.. [#SENDFILE]
    http://docs.python.org/3/library/os.html#os.sendfile

.. [#ADDCH]
    http://docs.python.org/3/library/curses.html#curses.window.addch

.. [#GUIDO]
   Guido van Rossum, posting to python-ideas, March 2012:
   http://mail.python.org/pipermail/python-ideas/2012-March/014364.html
   and
   http://mail.python.org/pipermail/python-ideas/2012-March/014378.html
   and
   http://mail.python.org/pipermail/python-ideas/2012-March/014417.html

=========
Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: