python-peps/pep-0448.txt

PEP: 448
Title: Additional Unpacking Generalizations
Version: $Revision$
Last-Modified: $Date$
Author: Joshua Landau <joshua@landau.ws>
Discussions-To: python-ideas@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 29-Jun-2013
Python-Version: 3.4
Post-History:


Abstract
========

This PEP proposes extended usages of the ``*`` iterable unpacking
operator and ``**`` dictionary unpacking operators
to allow unpacking in more positions, an arbitrary number of
times, and in additional circumstances.  Specifically,
in function calls, in comprehensions and generator expressions, and
in displays.

Function calls are proposed to support an arbitrary number of
unpackings rather than just one::

    >>> print(*[1], *[2], 3)
    1 2 3
    >>> dict(**{'x': 1}, y=2, **{'z': 3})
    {'x': 1, 'y': 2, 'z': 3}

Unpacking is proposed to be allowed inside tuple, list, set,
and dictionary displays::

    >>> *range(4), 4
    (0, 1, 2, 3, 4)
    >>> [*range(4), 4]
    [0, 1, 2, 3, 4]
    >>> {*range(4), 4}
    {0, 1, 2, 3, 4}
    >>> {'x': 1, **{'y': 2}}
    {'x': 1, 'y': 2}

In dictionaries, later values will always override earlier ones::

    >>> {'x': 1, **{'x': 2}}
    {'x': 2}

    >>> {**{'x': 2}, 'x': 1}
    {'x': 1}

Unpacking is proposed to be allowed inside list, set,
and dictionary comprehensions::

    >>> ranges = [range(i) for i in range(5)]
    >>> [*item for item in ranges]
    [0, 0, 1, 0, 1, 2, 0, 1, 2, 3]

    >>> {*item for item in ranges}
    {0, 1, 2, 3}


Rationale
=========

Current usage of the ``*`` iterable unpacking operator features
unnecessary restrictions that can harm readability.

Unpacking multiple times has an obvious rationale.  When you want to
unpack several iterables into a function definition or follow an unpack
with more positional arguments, the most natural way would be to write::

    function(**kw_arguments, **more_arguments)

    function(*arguments, argument)

Simple examples where this is useful are ``print`` and ``str.format``.
Instead, you could be forced to write::

    kwargs = dict(kw_arguments)
    kwargs.update(more_arguments)
    function(**kwargs)

    args = list(arguments)
    args.append(arg)
    function(*args)

or, if you know to do so::

    from collections import ChainMap
    function(**ChainMap(more_arguments, arguments))

    from itertools import chain
    function(*chain(args, [arg]))

which add unnecessary line-noise and, with the first methods, causes
duplication of work.


There are two primary rationales for unpacking inside of containers.
Firstly there is a symmetry of assignment, where ``fst, *other, lst =
elems`` and ``elems = fst, *other, lst`` are approximate inverses,
ignoring the specifics of types.  This, in effect, simplifies the
language by removing special cases.

Secondly, it vastly simplifies types of "addition" such as combining
dictionaries, and does so in an unambiguous and well-defined way::

    combination = {**first_dictionary, "x": 1, "y": 2}

instead of::

    combination = first_dictionary.copy()
    combination.update({"x": 1, "y": 2})

which is especially important in contexts where expressions are
preferred.  This is also useful as a more readable way of summing
iterables into a list, such as ``my_list + list(my_tuple) +
list(my_range)`` which is now equivalent to just ``[*my_list,
*my_tuple, *my_range]``.


The addition of unpacking to comprehensions is a logical extension.
It's usage will primarily be a neat replacement for ``[i for j in
list_of_lists for i in j]``, as the more readable
``[*l for l in list_of_lists]``. The iterable version,
``(*l for l in list_of_lists)``, replaces ``itertools.chain.from_iterable``.
Other uses are possible, but expected to occur rarely.


Specification
=============

Function calls may accept an unbounded number of ``*`` and ``**``
unpackings.  There will be no restriction of the order of positional
arguments with relation to ``*`` unpackings nor any restriction of the
order of keyword arguments with relation to ``**`` unpackings.

Function calls continue to have the restriction that keyword arguments
must follow positional arguments and ``**`` unpackings must additionally
follow ``*`` unpackings.

Currently, if an argument is given multiple times — such as a
positional argument given both positionally and by keyword — a
``TypeError`` is raised.  This remains true for duplicate arguments
provided through multiple ``**`` unpackings,
e.g. ``f(**{'x': 2}, **{'x': 3})``.

A function looks like this::

    function(
        argument or *args, argument or *args, ...,
        kwargument or *args, kwargument or *args, ...,
        kwargument or **kwargs, kwargument or **kwargs, ...
    )

Tuples, lists, sets and dictionaries will allow unpacking.  This will
act as if the elements from unpacked items were inserted in order at
the site of unpacking, much as happens in unpacking in a function-call.
Dictionaries require ``**`` unpacking; all the others require ``*`` unpacking.
A dictionary's key remain in a right-to-left priority order, so
``{**{'a': 1}, 'a': 2, **{'a': 3}}`` evaluates to ``{'a': 3}``.  There
is no restriction on the number or position of unpackings.

Comprehensions, by simple extension, will support unpacking.  As before,
dictionaries require ``**`` unpacking, all the others require ``*``
unpacking and key priorities are unchanged.

For example::

    {*[1, 2, 3], 4, 5, *{6, 7, 8}}

    (*e for e in [[1], [3, 4, 5], [2]])

    {**dictionary for dictionary in (globals(), locals())}

    {**locals(), "override": None}

Unbracketed comprehensions in function calls, such as ``f(x for x in it)``,
are already valid. These could be extended to::

    f(*x for x in it) == f((*x for x in it))
    f(**x for x in it) == f({**x for x in it})

However, this is likely to be confusing and is not included in this
PEP. These will throw ``SyntaxError`` and comprehensions with explicit
brackets should be used instead.


Disadvantages
=============

The allowable orders for arguments in a function call are more
complicated than before. The simplest explanation for the rules
may be "positional arguments precede keyword arguments and ``**``
unpacking; ``*`` unpacking precedes ``**`` unpacking".

Whilst ``*elements, = iterable`` causes ``elements`` to be a list,
``elements = *iterable,`` causes ``elements`` to be a tuple.  The
reason for this may confuse people unfamiliar with the construct.


Implementation
==============

An implementation for Python 3.5 is found at Issue 2292 on bug tracker [1]_.


References
==========

.. [1] Issue 2292, "Missing `*`-unpacking generalizations", Thomas Wouters
   (http://bugs.python.org/issue2292)

.. [2] Discussion on Python-ideas list,
   "list / array comprehensions extension", Alexander Heger
   (http://mail.python.org/pipermail/python-ideas/2011-December/013097.html)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: