Add PEP 448: Additional Unpacking Generalizations

by Joshua Landau.

pep-0448.txt

@@ -0,0 +1,247 @@
+PEP: 448
+Title: Additional Unpacking Generalizations
+Version: $Revision$
+Last-Modified: $Date$
+Author: Joshua Landau <joshua@landau.ws>
+Discussions-To: python-ideas@python.org
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 29-Jun-2013
+Python-Version: 3.4
+Post-History:
+
+
+Abstract
+========
+
+This PEP proposes extended usages of the ``*`` iterable unpacking
+operator to allow unpacking in more positions, an arbitrary number of
+times, and in several additional circumstances.
+
+Specifically:
+
+Arbitrarily positioned unpacking operators::
+
+    >>> print(*[1], *[2], 3)
+    1 2 3
+    >>> dict(**{'x': 1}, y=3, **{'z': 2})
+    {'x': 1, 'y': 2, 'z': 3}
+
+Function calls currently have the restriction that keyword arguments
+must follow positional arguments and ``**`` unpackings must additionally
+follow ``*`` unpackings.  Because of the new levity for ``*`` and ``**``
+unpackings, it may be advisable to lift some or all of these
+restrictions.
+
+As currently, if an argument is given multiple times - such as a
+positional argument given both positionally and by keyword - a
+TypeError is raised.
+
+Unpacking is proposed to be allowed inside tuples, lists, sets,
+dictionaries and comprehensions::
+
+    >>> *range(4), 4
+    (0, 1, 2, 3, 4)
+    >>> [*range(4), 4]
+    [0, 1, 2, 3, 4]
+    >>> {*range(4), 4}
+    {0, 1, 2, 3, 4}
+    >>> {'x': 1, **{'y': 2}}
+    {'x': 1, 'y': 2}
+
+    >>> ranges = [range(i) for i in range(5)]
+    >>> [*item for item in ranges]
+    [0, 0, 1, 0, 1, 2, 0, 1, 2, 3]
+
+
+Rationale
+=========
+
+Current usage of the ``*`` iterable unpacking operator features
+unnecessary restrictions that can harm readability.
+
+Unpacking multiple times has an obvious rationale.  When you want to
+unpack several iterables into a function definition or follow an unpack
+with more positional arguments, the most natural way would be to write::
+
+    function(**kw_arguments, **more_arguments)
+
+    function(*arguments, argument)
+
+Simple examples where this is useful are ``print`` and ``str.format``.
+Instead, you could be forced to write::
+
+    kwargs = dict(kw_arguments)
+    kwargs.update(more_arguments)
+    function(**kwargs)
+
+    args = list(arguments)
+    args.append(arg)
+    function(*args)
+
+or, if you know to do so::
+
+    from collections import ChainMap
+    function(**ChainMap(more_arguments, arguments))
+
+    from itertools import chain
+    function(*chain(args, [arg]))
+
+which add unnecessary line-noise and, with the first methods, causes
+duplication of work.
+
+
+There are two primary rationales for unpacking inside of containers.
+Firstly there is a symmetry of assignment, where ``fst, *other, lst =
+elems`` and ``elems = fst, *other, lst`` are approximate inverses,
+ignoring the specifics of types.  This, in effect, simplifies the
+language by removing special cases.
+
+Secondly, it vastly simplifies types of "addition" such as combining
+dictionaries, and does so in an unambiguous and well-defined way::
+
+    combination = {**first_dictionary, "x": 1, "y": 2}
+
+instead of::
+
+    combination = first_dictionary.copy()
+    combination.update({"x": 1, "y": 2})
+
+which is especially important in contexts where expressions are
+preferred.  This is also useful as a more readable way of summing
+iterables into a list, such as ``my_list + list(my_tuple) +
+list(my_range)`` which is now equivalent to just ``[*my_list,
+*my_tuple, *my_range]``.
+
+
+The addition of unpacking to comprehensions is a logical extension.
+It's usage will primarily be a neat replacement for ``[i for j in
+2D_list for i in j]``, as the more readable ``[*l for l in 2D_list]``.
+Other uses are possible, but expected to occur rarely.
+
+
+Specification
+=============
+
+Function calls may accept an unbound number of ``*`` and ``**``
+unpackings.  There will be no restriction of the order of positional
+arguments with relation to ``*`` unpackings nor any restriction of the
+order of keyword arguments with relation to ``**`` unpackings.
+
+Function calls currently have the restriction that keyword arguments
+must follow positional arguments and ``**`` unpackings must additionally
+follow ``*`` unpackings.  Because of the new levity for ``*`` and ``**``
+unpackings, it may be advisable to list some or all of these
+restrictions.
+
+As currently, if an argument is given multiple times - such as a
+positional argument given both positionally and by keyword - a
+TypeError is raised.
+
+If the restrictions are kept, a function call will look like this::
+
+	function(
+	    argument or *args, argument or *args, ...,
+	    kwargument or *args, kwargument or *args, ...,
+	    kwargument or **kwargs, kwargument or **kwargs, ...
+	)
+
+If they are removed completely, a function call will look like this::
+    
+    function(
+        argument or keyword_argument or `*`args or **kwargs,
+        argument or keyword_argument or `*`args or **kwargs,
+        ...
+    )
+
+
+Tuples, lists, sets and dictionaries will allow unpacking.  This will
+act as if the elements from unpacked items were inserted in order at
+the site of unpacking, much as happens in unpacking in a function-call.
+Dictionaries require ``**`` unpacking; all the others require ``*`` unpacking.
+A dictionary's key remain in a right-to-left priority order, so
+``{**{'a': 1}, 'a': 2, **{'a': 3}}`` evaluates to ``{'a': 3}``.  There
+is no restriction on the number or position of unpackings.
+
+Comprehensions, by simple extension, will support unpacking.  As before,
+dictionaries require ``**`` unpacking, all the others require ``*``
+unpacking and key priorities are unchanged.
+
+Examples include::
+
+    {*[1, 2, 3], 4, 5, *{6, 7, 8}}
+
+    (*e for e in [[1], [3, 4, 5], [2]])
+
+    {**dictionary for dictionary in (globals(), locals())}
+
+    {**locals(), "override": None}
+
+
+Disadvantages
+=============
+
+If the current restrictions for function call arguments (keyword
+arguments must follow positional arguments and ``**`` unpackings must
+additionally follow ``*`` unpackings) are kept, the allowable orders
+for arguments in a function call is more complicated than before.
+The simplest explanation for the rules may be "positional arguments
+come first and keyword arguments follow, but ``*`` unpackings are
+allowed after keyword arguments".
+
+If the current restrictions are lifted, there are no obvious gains to
+code as the only new orders that are allowed look silly: ``f(a, e=e,
+d=d, b, c)`` being a simpler example.
+
+
+Whilst ``*elements, = iterable`` causes ``elements`` to be a list,
+``elements = *iterable,`` causes ``elements`` to be a tuple.  The
+reason for this may not be obvious at first glance and may confuse
+people unfamiliar with the construct.
+
+
+Implementation
+==============
+
+An implementation for an old version of Python 3 is found at Issue
+2292 on bug tracker [1]_, although several changes should be made:
+
+- It has yet to be updated to the most recent Python version
+
+- It features a now redundant replacement for "yield from" which
+  should be removed
+
+- It also loses support for calling function with keyword arguments before
+  positional arguments, which is an unnecessary backwards-incompatible change
+
+- If the restrictions on the order of arguments in a function call are
+  partially or fully lifted, they would need to be included
+
+
+References
+==========
+
+.. [1] Issue 2292, "Missing `*`-unpacking generalizations", Thomas Wouters
+   (http://bugs.python.org/issue2292)
+
+.. [2] Discussion on Python-ideas list,
+   "list / array comprehensions extension", Alexander Heger
+   (http://mail.python.org/pipermail/python-ideas/2011-December/013097.html)
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End: