python-peps/pep-0570.rst

PEP: 570
Title: Python Positional-Only Parameters
Version: $Revision$
Last-Modified: $Date$
Author: Larry Hastings <larry@hastings.org>,
 Pablo Galindo <pablogsal@gmail.com>,
 Mario Corchero <mariocj89@gmail.com>
Discussions-To: Python-Dev <python-dev@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 20-Jan-2018


========
Overview
========

This PEP proposes a syntax for positional-only parameters in Python.
Positional-only parameters are parameters without an externally-usable
name; when a function accepting positional-only parameters is called,
positional arguments are mapped to these parameters based solely on
their position.

=========
Rationale
=========

Python has always supported positional-only parameters.
Early versions of Python lacked the concept of specifying
parameters by name, so naturally all parameters were
positional-only.  This changed around Python 1.0, when
all parameters suddenly became positional-or-keyword.
This allowed users to provide arguments to a function both
positionally or referencing the keyword used in the definition
of it. But, this is not always desired nor even available as
even in current versions of Python, many CPython
"builtin" functions still only accept positional-only arguments.

Even if positional arguments only in a function can be achieved
via using ``*args`` parameters and extracting them one by one,
the solution is far from ideal and not as expressive as the one
proposed in this PEP, which targets to provide syntax to specify
accepting a specific number of positional-only parameters.
Additionally, this will bridge the gap we currently find between
builtin functions that today allows to specify positional-only
parameters and pure Python implementations that lack the
syntax for it.

-----------------------------------------------------
Positional-Only Parameter Semantics In Current Python
-----------------------------------------------------

There are many, many examples of builtins that only
accept positional-only parameters.  The resulting
semantics are easily experienced by the Python
programmer--just try calling one, specifying its
arguments by name::


    >>> help(pow)
    ...
    pow(x, y, z=None, /)
    ...
    >>> pow(x=5, y=3)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: pow() takes no keyword arguments

Pow clearly expresses that its arguments are only positional
via the ``/`` marker, but this is at the moment only documentational,
Python developers cannot write such syntax.

In addition, there are some functions with particularly
interesting semantics:

* ``range()``, which accepts an optional parameter
  to the *left* of its required parameter. [#RANGE]_

* ``dict()``, whose mapping/iterator parameter is optional and
  semantically must be positional-only.  Any externally
  visible name for this parameter would occlude
  that name going into the ``**kwarg`` keyword variadic
  parameter dict! [#DICT]_

Obviously one can simulate any of these in pure Python code
by accepting ``(*args, **kwargs)`` and parsing the arguments
by hand.  But this results in a disconnect between the
Python function signature and what it actually accepts,
not to mention the work of implementing said argument parsing.

==========
Motivation
==========

The new syntax will allow developers to further control how their
API can be consumed. It will allow restricting the usage of keyword
Specify arguments by adding the new type of positional-only ones.

A similar PEP with a broader scope (PEP 457) was proposed
to define the syntax. This PEP builds on top of part of it
to define and provide an implementation for the ``/`` syntax on
function signatures.

=================================================================
The Current State Of Documentation For Positional-Only Parameters
=================================================================

The documentation for positional-only parameters is incomplete
and inconsistent:

* Some functions denote optional groups of positional-only arguments
  by enclosing them in nested square brackets. [#BORDER]_

* Some functions denote optional groups of positional-only arguments
  by presenting multiple prototypes with varying numbers of
  arguments. [#SENDFILE]_

* Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_

One more important idea to consider: currently in the documentation
there's no way to tell whether a function takes positional-only
parameters.  ``open()`` accepts keyword arguments, ``ord()`` does
not, but there is no way of telling just by reading the
documentation that this is true.

====================
Syntax And Semantics
====================

From the "ten-thousand foot view", and ignoring ``*args`` and ``**kwargs``
for now, the grammar for a function definition currently looks like this::

    def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that perspective, the new syntax for functions would look
like this::

    def name(positional_only_parameters, /, positional_or_keyword_parameters,
             *, keyword_only_parameters):

All parameters before the ``/`` are positional-only.  If ``/`` is
not specified in a function signature, that function does not
accept any positional-only parameters.
The logic around optional values for positional-only argument
Remains the same as the one for positional-or-keyword. Once
a positional-only argument is provided with a default,
the following positional-only and positional-or-keyword argument
need to have a default as well. Positional-only parameters that
don’t have a default value are "required" positional-only parameters.
Therefore the following are valid signatures::

    def name(p1, p2, /, p_or_kw, *, kw):
    def name(p1, p2=None, /, p_or_kw=None, *, kw):
    def name(p1, p2=None, /, *, kw):
    def name(p1, p2=None, /):
    def name(p1, p2, /, p_or_kw):
    def name(p1, p2, /):

Whilst the followings are not::

    def name(p1, p2=None, /, p_or_kw, *, kw):
    def name(p1=None, p2, /, p_or_kw=None, *, kw):
    def name(p1=None, p2, /):

==========================
Full grammar specification
==========================

A draft of the proposed grammar specification is::

    new_typedargslist:
      tfpdef (',' tfpdef)* ',' '/' [',' [typedargslist]] | typedargslist

    new_varargslist:
      vfpdef (',' vfpdef)* ',' '/' [',' [varargslist]] | varargslist

It will be added to the actual typedargslist and varargslist
but for easier discussion is presented as new_typedargslist and new_varargslist


===================
Implementation Plan
===================

The implementation will involve a full change of the Grammar. This will
involve following the steps outlined in PEP 306 [#PEP306]_. In addition, other
steps are needed including:

* Modifying the code object and the function object to be aware of positional
  only arguments.

* Modifiying ``ceval.c`` (``PyEval_EvalCodeEx``, ``PyEval_EvalFrameEx``...)
  to correctly handle positional-only arguments.

* Modifying ``marshal.c`` to account for the modifications of the code object.


This does not intend to be a guide or a comprehensive recipe on how to implement
this but a rough outline of the changes this will make to the codebase.

The advantages of this implementation involve speed, consistency with the
implementation of keyword-only parameters as in PEP 3102 and a simpler implementation
of all the tools and modules that will be impacted by this change.

==============
Rejected Ideas
==============

----------
Do Nothing
----------

Always an option, just not adding it. It was considered
though that the benefits of adding it is worth the complexity
it adds to the language.

---------------------
After marker proposal
---------------------

A complaint against the proposal is the fact that the modifier of
the signature impacts the "already passed" tokens.

This might make confusing to "human parsers" to read functions
with many arguments. Example::

  def really_bad_example_of_a_python_function(fist_long_argument, second_long_argument,
                                              third_long_argument, /):

It is not until you reach the end of the signature that the reader
realized the ``/`` and therefore the fact that the arguments are
position-only. This deviates from how the keyword-only marker works.

That said we could not find an implementation that would modify the
arguments after the marker, as that will force the one before the
marker to be position only as well. Example::

  def (x, y, /, z):

If we define that ``/`` makes only z position-only it won't be possible
to call x and y via keyword argument. Finding a way to work around it
will add confusion given that at the moment keyword arguments cannot be
followed by positional arguments. ``/`` will therefore make both the
preceding and following position-only.

-------------------
Per-argument marker
-------------------

Using a per argument marker might be an option as well. The approach
basically adds a token to each of the arguments that are position only
and requires those to be placed together. Example::

  def (.arg1, .arg2, arg3):

Note the dot on arg1 and arg2. Even if this approach might look easier
to read it has been discarded as ``/`` goes further inline with the
keyword-only approach and is less error prone.


----------------
Using decorators
----------------

It has been suggested on python-ideas [#python-ideas-decorator-based]_ to provide
a decorator written in Python as an implementation for this feature. This approach
has the advantage that keeps parameter declaration more easy to read but also
introduces an asymmetry on how parameter behaviour is declared. Also, as the ``/``
syntax is already introduced for C functions, this inconsistency will make more
difficult to implement all tools and modules that deal with this syntax including
but not limited to, the argument clinic, the inspect module and the ast module.
Another disadvantage of this approach is that calling the decorated functions
will be slower than the functions generated if the feature was implemented directly
in C.

======
Thanks
======

Credit for most of the content of this PEP is contained in Larry Hastings’s PEP 457.

Credit for the use of '/' as the separator between positional-only and positional-or-keyword
parameters go to Guido van Rossum, in a proposal from 2012. [#GUIDO]_

Credit for discussion about the simplification of the grammar goes to
Braulio Valdivieso.

.. [#DICT]
    http://docs.python.org/3/library/stdtypes.html#dict

.. [#RANGE]
    http://docs.python.org/3/library/functions.html#func-range

.. [#BORDER]
    http://docs.python.org/3/library/curses.html#curses.window.border

.. [#SENDFILE]
    http://docs.python.org/3/library/os.html#os.sendfile

.. [#ADDCH]
    http://docs.python.org/3/library/curses.html#curses.window.addch

.. [#GUIDO]
   Guido van Rossum, posting to python-ideas, March 2012:
   https://mail.python.org/pipermail/python-ideas/2012-March/014364.html
   and
   https://mail.python.org/pipermail/python-ideas/2012-March/014378.html
   and
   https://mail.python.org/pipermail/python-ideas/2012-March/014417.html

.. [#PEP306]
   https://www.python.org/dev/peps/pep-0306/

.. [#python-ideas-decorator-based]
   https://mail.python.org/pipermail/python-ideas/2017-February/044888.html

=========
Copyright
=========

This document has been placed in the public domain.