PEP 570: Restructure sections as recommended by PEP 1 (GH-972)

Organize PEP 570 according to the recommended ordering of sections
defined in PEP 1.  This presents the proposal in a more organized and
compelling way.

Notable changes are:

- The content more easily reads in from top to bottom by moving core
  content into Abstract, Motivation, Rationale, Specification.
- Making a clear distinction of the impact to library authors and
  callers of APIs.
- Fixed-up usage of "parameters" and "arguments" on the context of the
  what is being discussed.
- Grammatical edits and simplified wording, while maintaining the core
  content and intent of the text.
This commit is contained in:
Eric N. Vander Weele 2019-04-03 11:31:33 -04:00 committed by Pablo Galindo
parent 7a4c4969c1
commit d94eb8cd7d
1 changed files with 430 additions and 389 deletions

View File

@ -4,7 +4,8 @@ Version: $Revision$
Last-Modified: $Date$
Author: Larry Hastings <larry@hastings.org>,
Pablo Galindo <pablogsal@gmail.com>,
Mario Corchero <mariocj89@gmail.com>
Mario Corchero <mariocj89@gmail.com>,
Eric N. Vander Weele <ericvw@gmail.com>
BDFL-Delegate: Guido van Rossum <guido@python.org>
Discussions-To: https://discuss.python.org/t/pep-570-python-positional-only-parameters/1078
Status: Draft
@ -14,141 +15,50 @@ Created: 20-Jan-2018
========
Overview
Abstract
========
This PEP proposes a syntax for positional-only parameters in Python.
Positional-only parameters are parameters without an externally-usable
name; when a function accepting positional-only parameters is called,
positional arguments are mapped to these parameters based solely on
their position.
This PEP proposes to introduce a new syntax, ``/``, for specifying
positional-only parameters in Python.
=========
Rationale
=========
Positional-only parameters have no externally-usable name. When a function
accepting positional-only parameters is called, positional arguments are mapped
to these parameters based solely on their position.
Python has always supported positional-only parameters.
Early versions of Python lacked the concept of specifying
parameters by name, so naturally, all parameters were
positional-only. This changed around Python 1.0 when
all parameters suddenly became positional-or-keyword.
This change allowed users to provide arguments to a function
either positionally or referencing the keyword used in the
function's definition. However, this is not always desirable,
and in fact even in current versions of Python many CPython
"builtin" functions still only accept positional-only
arguments.
Design of APIs (application programmable interfaces) is important for library
authors to ensure correct and intended usage of an API. The inability to
specify which parameters are positional-only requires careful consideration
when choosing appropriate parameter names, even if the parameters are required
or have no external semantic meaning for callers of the API.
Users might want to restrict their API to not allow for parameters
to be referenced via keywords, as that exposes the name of the
parameter as part of the API. If a user of said API starts using the
argument by keyword when calling it and then the parameter
gets renamed, it will be a breaking change. By using positional-only
parameters the developer can later change the name of any arguments or
transform them to ``*args`` without breaking the API.
In this PEP, we discuss Python's history and current semantics for
positional-only parameters, the problems encountered by not having them, how
these problems are handled without language-intrinsic support, and the benefits
of having positional-only parameters. With context of the motivation, we then
discuss the rationale for why this change should be a feature intrinsic to the
language. Next, we propose the syntax for demarcating positional-only
parameters. Following, we present how to teach this new feature. Finally, we
conclude with noting rejected ideas in further detail, complementing the
rationale.
Even if making arguments positional-only in a function can be achieved
by using ``*args`` parameters and extracting them one by one,
the solution is far from ideal and not as expressive as the one
proposed in this PEP, which targets providing syntax to specify
accepting a specific number of positional-only parameters. Also,
it makes the signature of the function ambiguous as users won't
know how many parameters the function takes by looking at ``help()``
or auto-generated documentation.
==========
Motivation
==========
Additionally, this will bridge the gap we currently find between
builtin functions that can specify positional-only
parameters and pure Python implementations that lack the
syntax for it. The '/' syntax is already exposed in the
documentation of some builtins and interfaces generated by
the argument clinic.
--------------------------------------------------------
History of Positional-Only Parameter Semantics in Python
--------------------------------------------------------
Making positional-only arguments a possibility in Python will make the
language more consistent and since it would be a normal feature of Python
rather than a feature exclusive to extension modules, it should reduce
surprise and confusion by users encountering functions with positional-only
arguments. Notably, major third-party packages are already using the "/"
notation in their interfaces [#numpy-ufuncs]_ [#scipy-gammaln]_.
Positional-only arguments may be useful in several situations. One of the more
extreme situations is in a function that can take any keyword parameter but
also can take a positional one. Well-known examples for this situation are
``Formatter.format`` and ``dict.update``. For instance, ``dict.update``
accepts a dictionary (positionally) and/or any set of keyword parameters to use
as key/value pairs. In this case, if the dictionary parameter were not
positional-only, the user could not use the name that the interface uses for
said parameter or, conversely, the function could not distinguish easily if
the parameter received is the dictionary or one key/value pair.
Another important scenario is when argument names do not have semantic meaning.
For example, let's say we want to create a function that converts from one type
to another::
def as_my_type(x):
...
The name of the parameter provides no value whatsoever, and forces
the developer to maintain its name forever, as users might pass ``x`` as a
keyword.
Another good example is an API that wants make it clear that one of its
parameters is the "main" argument through positional-only arguments.
For example, see::
def add_to_queue(item: QueueItem):
...
Again we get no value from using keyword arguments here, and it can limit
future evolution of the API. Say at a later time we want this function
to be able to take multiple items while preserving backwards compatibility::
def add_to_queue(items: Union[QueueItem, List[QueueItem]]):
...
or to take them by using argument lists::
def add_to_queue(*items: QueueItem):
...
we will be forced to always keep the original argument or we would
potentially break users. By being able to define positional-only arguments,
we can change the name of the parameters at will or even change them to
``*args`` as in the previous example. There are multiple interfaces in the
standard library that fall into this category, for example the "main"
argument of ``collections.defaultdict`` (called *default_factory* in its
documentation) can only be passed positionally. One special case of this
situation is the *self* parameter for class methods: it is undersired that
a user can bind by keyword to the name "self" when calling the method from
the class::
io.FileIO.write(self=f, b=b"data")
Indeed, interfaces from the standard library implemented in C usually take
"self" as a positional-only argument::
>>> help(io.FileIO.write)
Help on method_descriptor:
write(self, b, /)
Write buffer b to file, return number of bytes written.
Another essential aspect to consider is PEP 399 [#PEP399]_, that mandates
that pure Python versions of modules in the standard library *must* have the
same interface and semantics that the accelerator modules implemented in C
(). For example, if ``collections.defaultdict`` were to have a pure Python
implementation it would need to make use of positional-only parameters to
match the interface of its C counterpart. A more detailed discussion about
this topic can be found in the Motivation_ section.
---------------------------------------------------
Positional-Only Parameter Semantics In Python Today
---------------------------------------------------
There are many, many examples of functions that only accept positional-only
parameters in the standard library. The resulting semantics are easily
experienced by the Python programmer -- just try calling one, specifying its
arguments by name::
Python originally supported positional-only parameters. Early versions of the
language lacked the ability to call functions binding arguments to a parameter
by name. Around Python 1.0, parameter semantics changed to be
positional-or-keyword. Since then, callers have been able to provide arguments
to a function either positionally or by the keyword name specified in the
function's definition.
In current versions of Python, many CPython "builtin" and standard library
functions only accept positional-only arguments. The resulting semantics can be
easily observed by calling one of these functions using keyword arguments::
>>> help(pow)
...
@ -159,70 +69,221 @@ arguments by name::
File "<stdin>", line 1, in <module>
TypeError: pow() takes no keyword arguments
``pow()`` clearly expresses that its arguments are only positional
via the ``/`` marker, but this at the moment is only a documentation convention,
Python developers cannot write such syntax.
``pow()`` clearly expresses that its arguments are positional-only via the
``/`` marker. However, this is only a documentation convention; Python
developers cannot use this syntax in code.
Besides, there are some functions with particularly
interesting semantics:
There are other functions with other interesting semantics:
* ``range()``, which accepts an optional parameter
to the *left* of its required parameter. [#RANGE]_
* ``range()``, an overloaded function, accepts an optional parameter to the
*left* of its required parameter. [#RANGE]_
* ``dict()``, whose mapping/iterator parameter is optional and semantically
must be positional-only. Any externally visible name for this parameter
would occlude that name going into the ``**kwarg`` keyword variadic
parameter dict! [#DICT]_
must be positional-only. Any externally visible name for this parameter
would occlude that name going into the ``**kwarg`` keyword variadic parameter
dict. [#DICT]_
One can simulate any of these in pure Python code
by accepting ``(*args, **kwargs)`` and parsing the arguments
by hand. However, this results in a disconnect between the
Python function signature and what the function accepts,
not to mention the work of implementing said argument parsing
and the lack of clarity in the resulting signature.
One can emulate the aforementioned semantics in Python code by accepting
``(*args, **kwargs)`` and parsing the arguments manually. However, this results
in a disconnect between the function definition and what the function
contractually accepts — the function definition does not match the logic of the
argument handling.
As mentioned before, this syntax is already being used outside the
CPython code base for similar use cases [#numpy-ufuncs]_ [#scipy-gammaln]_,
remarking that these scenarios are not exclusive to CPython and the
standard library.
Additionally, the ``/`` syntax is used beyond CPython for specifying similar
semantics (i.e., [#numpy-ufuncs]_ [#scipy-gammaln]_); thus, indicating that
these scenarios are not exclusive to CPython and the standard library.
Currently users are surprised when first encountering this notation, but this
is to be expected given that it has only recently been adequately documented
[#document-positional-only]_, and it is not possible to use it in Python code.
For these reasons, this notation is currently an oddity that appears only in
-------------------------------------------
Problems Without Positional-Only Parameters
-------------------------------------------
Without positional-only parameters, there are challenges for library authors
and callers of APIs. The following subsections outline the problems
encountered by each entity.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Challenges for Library Authors
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With positional-or-keyword parameters, the mix of calling conventions is not
always desirable. Authors may want to restrict usage of an API by disallowing
calling them with keyword arguments, which exposes the name of the parameter as
part of the public API. This is especially useful for required parameters that
already have semantic meaning with respect to function (e.g,
``namedtuple(typenames, field_names, …)`` or when the parameter name has no
true external meaning (e.g., ``arg1``, ``arg2``, …, etc for ``min()``). If a
caller of an API starts using a keyword argument, the parameter cannot be
renamed because it would be a breaking change.
Positional-only parameters can be emulated by extracting arguments from
``*args`` one by one. However, this approach is error-prone and is not
synonymous with the function definition, as previously mentioned. The usage of
the function is ambiguous and forces callers to look at ``help()`` or the
associated auto-generated documentation to understand what parameters the
function contractually accepts.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Challenges for Callers of an API
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Callers may be surprised when first encountering this notation. This is
expected given that it has only recently been documented
[#document-positional-only]_ and it is not possible to use in Python code. For
these reasons, this notation is currently an outlier that appears only in
CPython's APIs developed in C. Documenting the notation and making it possible
to be used in Python code will certainly eliminate this problem.
to be used in Python code would eliminate this disconnect.
==========
Motivation
==========
Furthermore, the documentation for positional-only parameters is inconsistent:
.. _Motivation:
* Some functions denote optional groups of positional-only arguments by
enclosing them in nested square brackets. [#BORDER]_
The new syntax will allow developers to further control how their
API can be consumed. It will allow restricting certain arguments
to be positional-only, so they cannot be passed with a keyword.
* Some functions denote optional groups of positional-only arguments by
presenting multiple prototypes with varying numbers of arguments.
[#SENDFILE]_
A similar PEP with a broader scope (PEP 457) was proposed earlier
to define the syntax. This PEP builds partially on top of that,
to define and provide an implementation for the ``/`` syntax in
function signatures.
* Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_
In addition to the API benefits outlined earlier in this document,
positional-only arguments are also faster, as demonstrated in this thread
about converting keyword arguments to positional:
[#thread-keyword-to-positional]_. In fact, because of these benefits there has
even been a recent trend towards moving builtins away from keyword arguments:
recently, backwards-incompatible changes were made to disallow keyword
arguments to ``bool``, ``float``, ``list``, ``int``, ``tuple``.
Another point of consideration the current documentation does not distinguish
whether a function takes positional-only parameters. ``open()`` accepts keyword
arguments; however, ``ord()`` does not — there is no way of telling just by
reading the documentation.
Providing a way to specify positional-only arguments in Python will make it
easier to maintain pure Python implementations of C modules and will allow
users to take advantage of these benefits even in code written only in Python.
It will also encourage users to start with positional-only arguments when they
believe that passing a keyword argument provides no clarity; unlike making a
keyword argument positional-only, allowing a positional argument to be passed
positionally is not a breaking change.
--------------------------------------
Benefits of Positional-Only Parameters
--------------------------------------
Positional-only parameters gives more control to library authors to better
express the intended usage of an API and allows the API to evolve in a safe,
backward-compatible way. Additionally, it makes the Python language more
consistent with respect to existing documentation and the behavior of various
"builtin" and standard library functions.
^^^^^^^^^^^^^^^^^^^^^^^^^^
Empowering Library Authors
^^^^^^^^^^^^^^^^^^^^^^^^^^
Library authors would have have the flexibility to change the name of
positional-only parameters without breaking callers. It reduces the
cognitive burden for choosing an appropriate public-facing name for required
parameters or parameters that have no true external semantic meaning.
Positional-only parameters are useful in several situations. An extreme
scenario is when a function accepts any keyword argument but can also accepts a
positional one. Prominent examples are ``Formatter.format`` and
``dict.update``. For instance, ``dict.update`` accepts a dictionary
(positionally), an iterable of key/value pairs (positionally), or multiple
keyword arguments. In this scenario, if the dictionary parameter were not
positional-only, the user could not use the name that the function definition
uses for said parameter or, conversely, the function could not distinguish
easily if the argument received is the dictionary/iterable or a keyword
argument for updating the key/value pair.
Another scenario where positional-only parameters are useful is when the
parameter name has no true external semantic meaning. For example, let's say
we want to create a function that converts from one type to another::
def as_my_type(x):
...
The name of the parameter provides no intrinsic value and forces the API author
to maintain its name forever since callers might pass ``x`` as a keyword
argument.
Additionally, positional-only arguments are useful when an API's parameters
are required and is unambiguous with respect to function. For example::
def add_to_queue(item: QueueItem):
...
It is clear by the name of the function the argument expected. A keyword
argument provides minimal benefit and also limits the future evolution of the
API. Say at a later time we want this function to be able to take multiple
items, while preserving backwards compatibility::
def add_to_queue(items: Union[QueueItem, List[QueueItem]]):
...
or to take them by using argument lists::
def add_to_queue(*items: QueueItem):
...
the author would be forced to always keep the original parameter name to avoid
potentially break callers.
By being able to specify positional-only arguments, an author can change the
name of the parameters freely or even change them to ``*args``, as seen in the
previous example. There are multiple function definitions in the standard
library which fall into this category. For example, the required parameter to
``collections.defaultdict`` (called *default_factory* in its documentation) can
only be passed positionally. One special case of this situation is the *self*
parameter for class methods: it is undesirable that a caller can bind by
keyword to the name ``self`` when calling the method from the class::
io.FileIO.write(self=f, b=b"data")
Indeed, function definitions from the standard library implemented in C usually
take ``self`` as a positional-only argument::
>>> help(io.FileIO.write)
Help on method_descriptor:
write(self, b, /)
Write buffer b to file, return number of bytes written.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Improving Language Consistency
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The Python language, itself, would be more consistent with positional-only
parameters. If the concept is a normal feature of Python rather than a feature
exclusive to extension modules, it would reduce confusion for users
encountering functions with positional-only arguments. Again, major
third-party packages are already using the ``/`` notation in their function
definitions [#numpy-ufuncs]_ [#scipy-gammaln]_.
Additionally, this would bridge the gap found between "builtin" functions which
specify positional-only parameters and pure Python implementations that lack
the syntax for it. The ``/`` syntax is already exposed in the documentation of
some builtins and interfaces generated by the argument clinic.
Another essential aspect to consider is PEP 399 [#PEP399]_, which mandates that
pure Python versions of modules in the standard library *must* have the same
interface and semantics that the accelerator modules implemented in C. For
example, if ``collections.defaultdict`` were to have a pure Python
implementation it would need to make use of positional-only parameters to match
the interface of its C counterpart.
=========
Rationale
=========
We propose to introduce positional-only parameters as a new syntax to the
Python language.
The new syntax would enable library authors to further control how their API
can be called. It would restrict arguments to be called as positional-only,
while not allowing them to be called as keyword arguments.
Previously, PEP 457 proposed to define the syntax, but with a much broader
scope. This PEP takes the original proposal a step further that by justifying
the syntax and providing an implementation for the ``/`` syntax in function
definitions.
In addition to the aforementioned benefits, the parsing and handling of
positional-only arguments is faster. This performance benefit can be
demonstrated in this thread about converting keyword arguments to positional:
[#thread-keyword-to-positional]_. Due to this speedup, there has been a recent
trend towards moving builtins away from keyword arguments: recently,
backwards-incompatible changes were made to disallow keyword arguments to
``bool``, ``float``, ``list``, ``int``, ``tuple``.
Providing a way to specify positional-only parameters in Python would make it
easier to maintain pure Python implementations of C modules. Additionally,
library authors defining functions would have the choice for choosing
positional-only parameters if they determine that passing a keyword argument
provides no additional clarity.
This is a well discussed, recurring topic on the Python mailing lists:
@ -241,10 +302,10 @@ This is a well discussed, recurring topic on the Python mailing lists:
* May 2006: `Benji York: [Python-Dev] Positional-only Arguments
<https://mail.python.org/pipermail/python-dev/2006-May/064790.html>`_
Positional-only parameters have also the (minor) advantage of enforcing
some logical order when calling interfaces that make use of them. For
example, the ``range`` function takes all its parameters positionally and
this disallows forms like::
Positional-only parameters also have the (minor) benefit of enforcing some
logical order when calling interfaces that make use of them. For example, the
``range`` function takes all its parameters positionally and disallows forms
like::
range(stop=5, start=0, step=2)
range(stop=5, step=2, start=0)
@ -256,20 +317,20 @@ intended order::
range(start=0, stop=5, step=2)
Another critical aspect that motivates positional-only arguments is
PEP 399 [#PEP399]_: Pure Python/C Accelerator Module Compatibility
Requirements. This PEP states that :
Another critical aspect which motivates positional-only parameters is PEP 399
[#PEP399]_: Pure Python/C Accelerator Module Compatibility Requirements. This
PEP states that :
This PEP requires that in these instances that the C code must pass
the test suite used for the pure Python code to act as much as
a drop-in replacement as reasonably possible
This PEP requires that in these instances that the C code must pass the
test suite used for the pure Python code to act as much as a drop-in
replacement as reasonably possible
It is clear that if the C code is implemented using the existing capabilities
to implement positional-only parameters using the argument clinic and
related machinery, it is not possible for the pure Python counterpart
to match the provided interface and requirements. This also creates a disparity
between the interfaces of some functions and classes in the CPython standard
library and other Python implementations. For example::
to implement positional-only parameters using the argument clinic, and related
machinery, it is not possible for the pure Python counterpart to match the
provided interface and requirements. This also creates a disparity between the
interfaces of some functions and classes in the CPython standard library and
other Python implementations. For example::
$ python3 # CPython 3.7.2
>>> import binascii; binascii.crc32(data=b'data')
@ -279,15 +340,14 @@ library and other Python implementations. For example::
>>>> import binascii; binascii.crc32(data=b'data')
2918445923
Other Python implementations can, of course, reproduce the CPython APIs
manually, but this goes against the spirit of PEP 399 [#PEP399]_ that
intends to avoid duplication of effort by mandating that all modules added
to Python's standard library **must** have a pure Python implementation
with the same interface and semantics.
Other Python implementations can reproduce the CPython APIs manually, but this
goes against the spirit of PEP 399 [#PEP399]_ to avoid duplication of effort by
mandating that all modules added to Python's standard library **must** have a
pure Python implementation with the same interface and semantics.
Another interesting scenario where positional-only arguments are important
appears when a sublclass overrides a method of the parent class changing
the name of one the arguments that is intended as a positional parameter::
Another scenario where positional-only parameters provide benefit occurs when a
subclass overrides a method of the base class and changes the name of arguments
that are intended to be positional::
class Base:
def meth(self, arg: int) -> str:
@ -302,76 +362,56 @@ the name of one the arguments that is intended as a positional parameter::
func(Sub()) # Runtime error
This situation can be considered a Liskov violation, as the subclass cannot be
used in a context when an instance of the parent class is expected. Renaming
arguments when overloading methods can happen when the subclass has good
reasons to use a different choice for the name that is more adequate to the
specific domain of the subclass (for example, when subclassing ``Mapping`` to
This situation could be considered a Liskov violation — the subclass cannot be
used in a context when an instance of the base class is expected. Renaming
arguments when overloading methods can happen when the subclass has reasons to
use a different choice for the parameter name that is more appropriate for the
specific domain of the subclass (e.g., when subclassing ``Mapping`` to
implement a DNS lookup cache, the derived class may not want to use the generic
argument names key and value but rather host and address). Declaring
this signatures with positional-only arguments can prevent this situation from
being a problem as users will not be able to call the interface using keyword
arguments. In general, designing for subclassing usually involves anticipating
code that hasn't been written yet and over which the author has no control.
Having measures that can facilitate the evolution of interfaces in a
backwards-compatible can be very useful for library authors and API designers.
argument names key and value but rather host and address). Having this
function definition with positional-only parameters can avoid this problem
because users will not be able to call the interface using keyword arguments.
In general, designing for subclassing usually involves anticipating code that
hasn't been written yet and over which the author has no control. Having
measures that can facilitate the evolution of interfaces in a
backwards-compatible would be useful for library authors.
A final argument in favor of positional-only arguments is that they allow
some new optimizations like the ones already present in the argument clinic
since said parameters must be passed in strict order. For instance, CPython's
internal *METH_FASTCALL* calling convention has been recently speciallized
for functions with positional-only parameters to eliminate the cost for
handling empty keywords. Similar performance improvements can be
applied when creating the evaluation frame of Python functions thanks to
positional-only parameters.
A final argument in favor of positional-only parameters is that they allow some
new optimizations like the ones already present in the argument clinic due to
the fact that parameters must be passed in strict order. For example, CPython's
internal *METH_FASTCALL* calling convention has been recently specialized for
functions with positional-only parameters to eliminate the cost for handling
empty keywords. Similar performance improvements can be applied when creating
the evaluation frame of Python functions thanks to positional-only parameters.
=================================================================
The Current State Of Documentation For Positional-Only Parameters
=================================================================
=============
Specification
=============
The documentation for positional-only parameters is incomplete
and inconsistent:
--------------------
Syntax and Semantics
--------------------
* Some functions denote optional groups of positional-only arguments
by enclosing them in nested square brackets. [#BORDER]_
* Some functions denote optional groups of positional-only arguments
by presenting multiple prototypes with varying numbers of
arguments. [#SENDFILE]_
* Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_
One more important idea to consider: currently in the documentation
there is no way to tell whether a function takes positional-only
parameters. ``open()`` accepts keyword arguments, ``ord()`` does
not, but there is no way of telling just by reading the
documentation.
====================
Syntax And Semantics
====================
From the "ten-thousand foot view", and ignoring ``*args`` and ``**kwargs``
for now, the grammar for a function definition currently looks like this::
From the "ten-thousand foot view", eliding ``*args`` and ``**kwargs`` for
illustration, the grammar for a function definition would look like::
def name(positional_or_keyword_parameters, *, keyword_only_parameters):
Building on that perspective, the new syntax for functions would look
like this::
Building on that example, the new syntax for function definitions would look
like::
def name(positional_only_parameters, /, positional_or_keyword_parameters,
*, keyword_only_parameters):
All parameters before the ``/`` are positional-only. If ``/`` is
not specified in a function signature, that function does not
accept any positional-only parameters.
The logic around optional values for positional-only arguments
remains the same as for positional-or-keyword arguments. Once
a positional-only argument is provided with a default,
the following positional-only and positional-or-keyword arguments
need to have defaults as well. Positional-only parameters that
do not have a default values are *required* positional-only parameters.
Therefore the following are valid signatures::
All parameters left of the ``/`` are demarcated as positional-only. If ``/``
is not specified in the function definition, that function does not accept any
positional-only arguments. The logic around optional values for
positional-only arguments remains the same as for positional-or-keyword
arguments. Once a positional-only parameter is specified with a default, the
following positional-only and positional-or-keyword arguments need to have
defaults as well. Positional-only parameters which do not have a default
values are *required* positional-only parameters. Therefore the following are
valid signatures::
def name(p1, p2, /, p_or_kw, *, kw):
def name(p1, p2=None, /, p_or_kw=None, *, kw):
@ -386,6 +426,26 @@ While the followings are not::
def name(p1=None, p2, /, p_or_kw=None, *, kw):
def name(p1=None, p2, /):
--------------------------
Full grammar specification
--------------------------
A draft of the proposed grammar specification is::
new_typedargslist:
tfpdef ['=' test] (',' tfpdef ['=' test])* ',' '/' [',' [typedargslist]] | typedargslist
new_varargslist:
vfpdef ['=' test] (',' vfpdef ['=' test])* ',' '/' [',' [varargslist]] | varargslist
It would be added to the actual ``typedargslist`` and ``varargslist``, but for
more relaxed discussion it is presented as ``new_typedargslist`` and
``new_varargslist``. Note that using a construction with two new rules
(``new_varargslist`` and ``new_varargslist``) is not possible with the current
parser as a rule is not LL(1). This is the reason the rule needs to be
included in the existing ``typedargslist`` and ``varargslist`` (in the same way
keyword-only arguments were introduced).
--------------------------------
Origin of the "/" as a separator
--------------------------------
@ -396,57 +456,36 @@ in 2012 [#GUIDO]_ :
Alternative proposal: how about using '/' ? It's kind of the opposite
of '*' which means "keyword argument", and '/' is not a new character.
==========================
Full grammar specification
==========================
A draft of the proposed grammar specification is::
new_typedargslist:
tfpdef ['=' test] (',' tfpdef ['=' test])* ',' '/' [',' [typedargslist]] | typedargslist
new_varargslist:
vfpdef ['=' test] (',' vfpdef ['=' test])* ',' '/' [',' [varargslist]] | varargslist
It will be added to the actual ``typedargslist`` and ``varargslist``, but for
more relaxed discussion it is presented as ``new_typedargslist`` and
``new_varargslist``. Also, notice that using a construction with two new rules
(new_varargslist and new_varargslist) is not possible with the current parser
as a rule is not LL(1). This is the reason the rule needs to be included in
the existing typedargslist and varargslist (in the same way keyword-only
arguments were introduced).
=================
How to teach this
=================
Since this concept is closely analogous to keyword-only arguments, introducing
a dedicated syntax to mark positional-only arguments may in fact make it
*easier* to teach the possible function signatures a user may encounter or
*easier* to teach the possible function definitions a user may encounter or
design, by teaching the two concepts together.
This PEP recommends adding a new subsection to the Python documentation,
in the section `"More on Defining Functions"`_, where the rest of the
argument types are discussed. The following paragraphs serve as a draft
for these additions that will serve to introduce the notation for both
positional-only and keyword-only parameters. It does not intend to be
exhaustive, nor should it be considered the final version to be incorporated
into the documentation.
This PEP recommends adding a new subsection to the Python documentation, in the
section `"More on Defining Functions"`_, where the rest of the argument types
are discussed. The following paragraphs serve as a draft for these additions
that will serve to introduce the notation for both positional-only and
keyword-only parameters. It does not intend to be exhaustive, nor should it be
considered the final version to be incorporated into the documentation.
.. _"More on Defining Functions": https://docs.python.org/3.7/tutorial/controlflow.html#more-on-defining-functions
-------------------------------------------------------------------------------
By default, all parameters in a Python function signature can be passed either
by position or explicitly by keyword. Occasionally, it makes sense to restrict
the way arguments can be passed. To this end, it is possible to mark certain
parameters as *positional-only*, meaning that they cannot be passed by keyword,
by placing a ``/`` (forward-slash) in the arguments list after the last
positional-only parameter. In order to mark parameters as *keyword-only*, meaning
that they can *only* be passed by position, place a ``*`` in the arguments list
before the first keyword-only parameter.
By default, all arguments passed to a Python function can be either by position
or explicitly by keyword. Occasionally, it makes sense to restrict the way
arguments can be passed. To this end, it is possible to mark certain parameters
as *positional-only*, meaning that they cannot be passed by keyword. This can
be achieved by placing a ``/`` (forward-slash) in the arguments list after the
last positional-only parameter. In order to mark parameters as *keyword-only*,
meaning that they can *only* be passed by keyword argument, place a ``*`` in
the arguments list before the first keyword-only parameter.
A function signature that makes use of both of these features is::
A function definition which makes use of both of these features may look like::
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
----------- ---------- ----------
@ -495,7 +534,8 @@ The third only allows keyword arguments::
>>> kwd_only_arg(arg=3)
3
And the last uses all three calling conventions in the same function signature::
And the last uses all three calling conventions in the same function
definition::
>>> combined_example(1, 2, 3)
Traceback (most recent call last):
@ -511,17 +551,17 @@ And the last uses all three calling conventions in the same function signature::
TypeError: combined_example() got an unexpected keyword argument 'pos_only'
==============
Implementation
==============
========================
Reference Implementation
========================
An initial implementation that passes the CPython test suite is available
for evaluation [#posonly-impl]_.
An initial implementation that passes the CPython test suite is available for
evaluation [#posonly-impl]_.
The advantages of this implementation involve speed, consistency with the
implementation of keyword-only parameters as in PEP 3102 and a simpler
implementation of all the tools and modules that will be impacted by
this change.
The benefits of this implementations are speed of handling positional-only
parameters, consistency with the implementation of keyword-only parameters (PEP
3102), and a simpler implementation of all the tools and modules that would be
impacted by this change.
==============
Rejected Ideas
@ -531,63 +571,95 @@ Rejected Ideas
Do Nothing
----------
Always an option, just not adding it. It was considered
though that the benefits of adding it is worth the complexity
it adds to the language.
Always an option — the status quo. While this was considered, the
aforementioned benefits is worth the additional complexity to the language.
---------------------
After marker proposal
---------------------
----------
Decorators
----------
A complaint against the proposal is the fact that the modifier of
the signature impacts the tokens already passed.
It has been suggested on python-ideas [#python-ideas-decorator-based]_ to
provide a decorator written in Python for this feature.
This might make it confusing to users to read functions
with many arguments. Example::
This approach has the benefit of not polluting function definition with
additional syntax. However, we have decided to reject this idea because:
def really_bad_example_of_a_python_function(fist_long_argument, second_long_argument,
third_long_argument, /):
* It introduces an asymmetry with how parameter behavior is declared.
It is not until reaching the end of the signature that the reader
realises the ``/``, and therefore the fact that the arguments are
position-only. This deviates from how the keyword-only marker works.
* It makes it difficult to safely for static analyzers and type checkers to
safely identify positional-only parameters. They would need to query the AST
for the list of decorators and identify the correct one by name or with extra
heuristics, whereas opposed to how keyword-only parameters are exposed
directly in the AST. In order for tools to correctly identify
positional-only parameters, they would need to execute the module to access
any metadata the decorator is setting.
That said we could not find an implementation that would modify the
arguments after the marker, as that will force the one before the
marker to be position-only as well. Example::
* Any error with the declaration will be reported only at runtime.
def (x, y, /, z):
* It may be more difficult to identify positional-only parameters in long
function definitions, as it forces the user to count them to know which is
the last one that is impacted by the decorator.
* The ``/`` syntax has already been introduced for C functions. This
inconsistency will make it more challenging to implement any tools and
modules that deal with this syntax — including but not limited to, the
argument clinic, the inspect module and the ``ast`` module.
* The decorator implementation would likely impose a runtime performance cost,
particularly when compared to adding support directly to the interpreter.
If we define that ``/`` makes only z position-only, it will not be possible
to call x and y via keyword argument. Finding a way to work around it
will add confusion given that at the moment keyword arguments cannot be
followed by positional arguments. ``/`` will, therefore, make both the
preceding and following parameters position-only.
-------------------
Per-argument marker
-------------------
Using a per-argument marker might be an option as well. The approach adds a
token to each of the arguments that are position only and requires those to be
placed together. Example::
A per-argument marker is another language-intrinsic option. The approach adds
a token to each of the parameters to indicate they are positional-only and
requires those parameters to be placed together. Example::
def (.arg1, .arg2, arg3):
Note the dot on arg1 and arg2. Even if this approach might look easier
to read, it has been discarded as ``/`` goes further in line with the
keyword-only approach and is less error-prone.
Note the dot (i.e., ``.``) on ``.arg1`` and ``.arg2``. While this approach
may be easier to read, it has been rejected because ``/`` as an explicit marker
is congruent with ``*`` for keyword-only arguments and is less error-prone.
It should be noted that some libraries already use leading underscore
[#leading-underscore]_ to conventionally indicate parameters as positional-only.
-----------------------------------
Using "__" as a per-argument marker
-----------------------------------
Some libraries and applications (like ``mypy`` or ``jinja``) use names
prepended with a double underscore (i.e., ``__``) as a convention to indicate
positional-only parameters. We have rejected the idea of introducing ``__`` as
a new syntax because:
* It is a backwards-incompatible change.
* It is not symmetric with how the keyword-only parameters are currently
declared.
* Querying the AST for positional-only parameters would require checking the
normal arguments and inspecting their names, whereas keyword-only parameters
have a property associated with them (``FunctionDef.args.kwonlyargs``).
* Every parameter would need to be inspected to know when positional-only
arguments end.
* The marker is more verbose, forcing marking every positional-only parameter.
* It clashes with other uses of the double underscore prefix like invoking name
mangling in classes.
Some libraries use leading underscore [#leading-underscore]_
to mark those arguments as positional-only.
-------------------------------------------------
Group positional-only parameters with parenthesis
-------------------------------------------------
Tuple parameter unpacking is Python 2 feature which allows the use of a tuple
as a parameter in a function definition. It allows a sequence argument
to be unpacked automatically. An example is::
Tuple parameter unpacking is a Python 2 feature which allows the use of a tuple
as a parameter in a function definition. It allows a sequence argument to be
unpacked automatically. An example is::
def fxn(a, (b, c), d):
pass
@ -597,62 +669,31 @@ proposition to reuse this syntax to implement positional-only parameters. We
have rejected this syntax for indicating positional only parameters for several
reasons:
* The syntax is asymmetric with respect to how keyword-only parameters are declared.
* The syntax is asymmetric with respect to how keyword-only parameters are
declared.
* Python 2 uses this syntax which could raise confusion regarding the behavior of this syntax.
This would be surprising to users porting Python 2 codebases that were using
this feature.
* Python 2 uses this syntax which could raise confusion regarding the behavior
of this syntax. This would be surprising to users porting Python 2 codebases
that were using this feature.
* This syntax is very similar to tuple literals. This can raise additional
confusion because it can be confused with a tuple declaration.
----------------------------------
Using "__" prepended as convention
----------------------------------
------------------------
After separator proposal
------------------------
Demarcating positional-parameters after the ``/`` was another consideration.
However, we were unable to find an approach which would modify the arguments
after the marker. Otherwise, would force the parameters before the marker to
be positional-only as well. For example::
Some libraries and applications (like mypy or jinja) use names prepended with a double
underscore ("__") as a convention to indicate positional-only parameters. We have rejected
this idea of introducing ``__`` as a new syntax because:
def (x, y, /, z):
* Is strictly backwards incompatible.
* Is not symmetric on how the keyword-only parameters are currently declared.
* Querying the ast for positional-only parameters requires now checking the
normal arguments and inspecting their names as opposed to how keyword-only
parameters have a property associated with it (``FunctionDef.args.kwonlyargs``).
* Every parameter needs to be inspected to know when positional-only arguments end.
* This proposal forces users to mark every parameter, making it more verbose.
* It clashes with other uses of the double underscore prefix like invoking name
mangling in classes.
----------------
Using decorators
----------------
It has been suggested on python-ideas [#python-ideas-decorator-based]_ to
provide a decorator written in Python as an implementation for this feature.
We have decided reject this idea because:
* It introduces an asymmetry on how parameter behavior is declared.
* It makes very difficult to safely identify positional-only parameters for
static analyzers and type checkers. They would need to query the AST for the
list of decorators and identify the correct one by name or via extra heuristics,
as opposed to how keyword-only parameters are exposed in the AST. If tools
would want to correctly identify positional-only parameters, they would need to execute the module
to access any metadata the decorator is setting.
* Any error with the declaration will be reported at runtime.
* It may be more difficult to identify positional-only parameters in long function
definitions as it forces the user to count them to know which is the last one
that is impacted by the decorator.
* The ``/`` syntax is already introduced for C functions, this inconsistency
will make it more difficult to implement all tools and modules that deal
with this syntax including but not limited to, the argument clinic, the inspect
module and the ast module.
* Calling the decorated functions could be slower than the functions generated if the feature was implemented directly in C.
If we define that ``/`` demarcates ``z`` as positional-only, it would not be
possible to specify ``x`` and ``y`` as keyword arguments. Finding a way to
work around this limitation would add confusion given that at the moment
keyword arguments cannot be followed by positional arguments. Therefore, ``/``
will make both the preceding and following parameters positional-only.
======
Thanks