From c5270848fe4481947ee951c2a415824b4dcd8a4f Mon Sep 17 00:00:00 2001 From: Chris Angelico Date: Wed, 21 Oct 2015 12:44:00 +1100 Subject: [PATCH] Apply more PEP 505 changes from Mark --- pep-0505.txt | 184 +++++++++++++++++++++++++++-------------------- pep-0505/test.py | 2 +- 2 files changed, 108 insertions(+), 78 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt index 221bd095c..fa3a25dd3 100644 --- a/pep-0505.txt +++ b/pep-0505.txt @@ -22,7 +22,7 @@ patterns involving null references. * The "``null``-aware member access" operator accesses an instance member only if that instance is non-``null``. Otherwise it returns ``null``. (This is also called a "safe navigation" operator.) -* The "``null``-aware index access" operator accesses a member of a collection +* The "``null``-aware index access" operator accesses an element of a collection only if that collection is non-``null``. Otherwise it returns ``null``. (This is another type of "safe navigation" operator.) @@ -36,9 +36,10 @@ Python should not add this behavior, so that it can be pointed to in the future when the question inevitably arises again. (This is the null alternative, so to speak!) -This proposal includes several new operators and should not be considered an -all-or-nothing proposal. For example, the safe navigation operators might be -rejected even if the ``null``-coalescing operator is approved, or vice-versa. +This proposal advances multiple alternatives, and it should be considered +severable. It may be accepted in whole or in part. For example, the safe +navigation operators might be rejected even if the ``null``-coalescing operator +is approved, or vice-versa. Of course, Python does not have ``null``; it has ``None``, which is conceptually distinct. Although this PEP is inspired by "``null``-aware" operators in other @@ -110,8 +111,8 @@ has `a get_notBefore() method `_ that returns either a timestamp or ``None``. This function is a thin wrapper around an OpenSSL function with the return type -``ASN1_TIME *``. Since this C pointer is allowed to be null, the Python wrapper -must be able to express a missing "not before" date, e.g. ``None``. +``ASN1_TIME *``. Because this C pointer may be ``null``, the Python wrapper must +be able to represent ``null``, and ``None`` is the chosen representation. The representation of ``null`` is particularly noticeable when Python code is marshalling data between two systems. For example, consider a Python server that @@ -123,7 +124,7 @@ converting between these representations adds unnecessary complexity to the Python glue code. Therefore, the preference for avoiding ``None`` is nothing more than a -preference; ``None`` has legitimate uses, particularly in specific types of +preference. ``None`` has legitimate uses, particularly in specific types of software. Any hypothetical ``None``-aware operators should be construed as syntactic sugar for simplifying common patterns involving ``None``, and *should not be construed* as error handling behavior. @@ -219,8 +220,8 @@ products a customer has in his/her shopping cart:: An experienced Python developer should know how ``or`` works and be capable of avoiding bugs like this. However, getting in the habit of using ``or`` for this -purpose still might cause even an experienced developer to occasionally make -this mistake, especially refactoring existing code and not carefully paying +purpose still might cause an experienced developer to occasionally make this +mistake, especially when refactoring existing code and not carefully paying attention to the possible values of the left-hand operand. For inexperienced developers, the problem is worse. The top Google hit for @@ -266,9 +267,9 @@ The author of this package could have written it like this instead:: This ordering of the operands is more intuitive, but it requires 4 extra characters (for "not "). It also highlights the repetition of identifiers: -``data if data``, ``files if files``, etc. This example also benefits from short -identifiers. What if the tested expression is longer and/or has side effects? -This is addressed in the next section. +``data if data``, ``files if files``, etc. This example benefits from short +identifiers, but what if the tested expression is longer and/or has side +effects? This is addressed in the next section. Motivating Examples @@ -498,9 +499,10 @@ Usage Of ``None`` In The Standard Library ----------------------------------------- The previous sections show some code patterns that are claimed to be "common", -but how common are they? The attached script ``find-pep505.py`` is meant to -answer this question. It uses the ``ast`` module to search for these patterns in -any ``*.py`` file. It checks for variations of the following patterns. +but how common are they? The attached script `find-pep505.py +`_ is meant to +answer this question. It uses the ``ast`` module to search for variations of the +following patterns in any ``*.py`` file. >>> # None-coalescing if block ... @@ -566,14 +568,16 @@ The script prints out any matches it finds. Sample:: .. note:: Coalescing with ``or`` is marked as a "possible" match, because it's not - trivial to infer whether ``or`` it is meant to coalesce False-y values + trivial to infer whether ``or`` is meant to coalesce False-y values (correct) or if it meant to coalesce ``None`` (incorrect). On the other - hand, we assume that `and` is always incorrect for safe navigation. + hand, we assume that ``and`` is always incorrect for safe navigation. -The script is tested against ``test.py`` (also attached to this document) and -the Python 3.4 standard library, but it should work on any arbitrary Python -source code. The complete output from running it against the standard library is -attached to this proposal as ``find-pep505.out``. +The script has been tested against `test.py +`_ and the Python 3.4 +standard library, but it should work on any arbitrary Python 3 source code. The +complete output from running it against the standard library is attached to this +proposal as `find-pep505.out `_. The script counts how many matches it finds and prints the totals at the end:: @@ -602,13 +606,13 @@ those ideas are recorded here. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``None``-aware syntax applies to attribute and index access, so it seems -natural to ask if it should also apply to function invocation syntax. Borrowing -a spelling similar to C#, it might be spelled ``foo?()``, where ``foo`` is only -called if it is not None. This idea was quickly rejected, for several reasons. +natural to ask if it should also apply to function invocation syntax. It might +be written as ``foo?()``, where ``foo`` is only called if it is not None. This +idea was quickly rejected, for several reasons. -No other mainstream language has such syntax. Moreover, it would be difficult to -discern if a function expression returned ``None`` because it was short- -circuited or because the function itself returned ``None``. Finally, Python +First, no other mainstream language has such syntax. Moreover, it would be +difficult to discern if a function call returned ``None`` because the function +itself returned ``None`` or because it was short-circuited. Finally, Python evaluates arguments to a function before it looks up the function itself, so ``foo?(bar())`` would still call ``bar()`` even if ``foo`` is ``None``. This behaviour is unexpected for a so-called "short-circuiting" operator. @@ -622,8 +626,10 @@ truly short-circuiting. To generalize the ``None``-aware behavior and limit the number of new operators introduced, a unary, postfix operator spelled ``?`` was suggested. The idea is -that ``?`` would return a special object that could would override dunder -methods to return itself or to return None. For example:: +that ``?`` might return a special object that could would override dunder +methods that return ``self``. For example, ``foo?`` would evaluate to ``foo`` if +it is not ``None``, otherwise it would evaluate to an instance of +``NoneQuestion``:: class NoneQuestion(): def __call__(self, *args, **kwargs): @@ -633,14 +639,13 @@ methods to return itself or to return None. For example:: return self def __getitem__(self, key): - reutrn self + return self -An expression like ``foo?`` would return a ``NoneQuestion`` instance if ``foo`` -is ``None``; otherwise, it returns ``foo``. With this operator, an expression -like ``foo?.bar[baz]`` evaluates to ``NoneQuestion`` if ``foo`` is None. This is -a nice generalization, but it's difficult to use in practice since most existing -code won't know what ``NoneQuestion`` is. +With this new operator and new type, an expression like ``foo?.bar[baz]`` +evaluates to ``NoneQuestion`` if ``foo`` is None. This is a nifty +generalization, but it's difficult to use in practice since most existing code +won't know what ``NoneQuestion`` is. Going back to one of the motivating examples above, consider the following:: @@ -652,8 +657,9 @@ The JSON serializer does not know how to serialize ``NoneQuestion``, nor will any other API. This proposal actually requires *lots of specialized logic* throughout the standard library and any third party library. -The ``?`` operator may also be **too general**, in the sense that it can be -combined with any other operator. What should the following expressions mean? +At the same time, the ``?`` operator may also be **too general**, in the sense +that it can be combined with any other operator. What should the following +expressions mean? >>> x? + 1 >>> x? -= 1 @@ -707,9 +713,12 @@ This PEP suggests 4 new operators be added to Python: 3. ``None``-aware attribute access 4. ``None``-aware index access/slicing -We will continue to assume the same spellings as in the previous sections in -order to focus on behavior before diving into the much more contentious issue of -how to spell these operators. +We will continue to assume the same spellings as in +the previous sections in order to focus on behavior before diving into the much +more contentious issue of how to spell these operators. + +A generalization of these operators is also proposed below under the heading +"Generalized Coalescing". ``None``-Coalescing Operator @@ -831,13 +840,11 @@ preserving the short circuit semantics of the code that it replaces. -------------------------- The idea of a ``None``-aware function invocation syntax was discussed on python- -ideas, but the idea was rejected by BDFL. The syntax comes dangerously close to -allowing a caller to change the return type of a function. If a function is -defined to always return a value, then it seems strange that the call site could -change the function to return "value or ``None``". +ideas, but the idea was rejected by BDFL. The reasons for this rejection are +detailed above. -Still, conditional function execution is a common idiom in Python, particularly -for callback functions. Consider this hypothetical example:: +Still, calling a function when it is not ``None`` is a common idiom in Python, +particularly for callback functions. Consider this hypothetical example:: import time @@ -847,8 +854,8 @@ for callback functions. Consider this hypothetical example:: if callback is not None: callback() -With a ``None``-aware function invocation, this example might be written more -concisely as:: +With the rejected ``None``-aware function call syntax, this example might be +written more concisely as:: import time @@ -856,11 +863,12 @@ concisely as:: time.sleep(seconds) callback?() -Consider a "``None``-severing" operator, however, which is a short-circuiting, -boolean operator similar to the ``None``-coalesing operator, except it returns -its left operand if that operand is None. If the left operand is None, then the -right operand is not evaluated. Let's temporarily spell this operator ``✂`` and -rewrite the example accordingly:: +Instead, consider a "``None``-severing" operator, however, which is a short- +circuiting, boolean operator similar to the ``None``-coalesing operator, except +it returns its left operand if that operand is None and otherwise returns the +right operand. If the left operand is None, then the right operand is not +evaluated. Let's temporarily spell this operator ``✂`` and rewrite the example +accordingly:: import time @@ -925,8 +933,9 @@ section, we continue to use the temporary spelling ``💩``:: The operator has the same precedence and associativity as the plain attribute access operator ``.``, but this operator is also short-circuiting in a unique -way: if the left operand is ``None``, then any adjacent attribute access, index -access, slicing, or function call operators *are not evaluated*. +way: if the left operand is ``None``, then any series of attribute access, index +access, slicing, or function call operators immediately to the right of it *are +not evaluated*. >>> name = ' The Black Knight ' >>> name.strip()[4:].upper() @@ -953,10 +962,10 @@ To put it another way, the following expressions are semantically equivalent:: original goal of writing common cases more concisely. The Dart semantics are nearly useless. -This operator short circuits one or more immediately adjacent attribute access, -index access, slicing, or function call operators, but it does not short circuit -any other operators (logical, bitwise, arithmetic, etc.), nor does it escape -parentheses:: +This operator short circuits one or more attribute access, index access, +slicing, or function call operators that are immediately to its right, but it +does not short circuit any other operators (logical, bitwise, arithmetic, etc.), +nor does it escape parentheses:: >>> d = date.today() >>> d💩year.numerator + 1 @@ -982,22 +991,23 @@ The third example fails because the operator does not escape parentheses. In that example, the attribute access ``numerator`` is evaluated and fails because ``None`` does not have that attribute. -Finally, observe that short circuiting adjacent operators is not at all the same thing as propagating ``None`` throughout an expression. +Finally, observe that short circuiting adjacent operators is not at all the same thing as propagating ``None`` throughout an expression:: >>> user💩first_name.upper() If ``user`` is not ``None``, then ``user.first_name`` is evaluated. If ``user.first_name`` evaluates to ``None``, then ``user.first_name.upper()`` is -an error! In English, this expression says ``user`` is optional but if it has a -value, then it must have a ``first_name``, too. +an error! In English, this expression says, "``user`` is optional but if it has +a value, then it must have a ``first_name``, too."" If ``first_name`` is supposed to be optional attribute, then the expression must -make that explicit: +make that explicit:: >>> user💩first_name💩upper() The operator is not intended as an error silencing mechanism, and it would be -wrong if its presence infected nearby operators. +undesirable if its presence infected nearby operators. + ``None``-Aware Index Access/Slicing Operator -------------------------------------------- @@ -1036,11 +1046,12 @@ The ``None``-aware slicing operator behaves similarly:: None These operators have the same precedence as the plain index access and slicing -operators. +operators. They also have the same short-circuiting behavior as the +``None``-aware attribute access. -Generalization --------------- +Generalized Coalescing +---------------------- Making ``None`` a special case may seem too specialized and magical. It is possible to generalize the behavior by making the ``None``-aware operators @@ -1058,13 +1069,32 @@ to this:: def __coalesce__(self): return True +If this generalization is accepted, then the operators will need to be renamed +such that the term ``None`` is not used, e.g. "Coalescing Operator", "Coalesced +Member Access Operator", etc. + +The coalescing operator would invoke this dunder method. The following two expressions are semantically equivalent:: + + >>> foo ✊🍆 bar + >>> bar if foo.__coalesce__() else foo + +The coalesced attribute and index access operators would invoke the same dunder +method:: + + >>> user💩first_name.upper() + >>> None if user.__coalesce__() else user.first_name.upper() + This generalization allows for domain-specific ``null`` objects to be coalesced just like ``None``. For example the ``pyasn1`` package has a type called ``Null`` that represents an ASN.1 ``null``. -If this generalization is accepted, then the operators will need to be renamed -such that the term ``None`` is not used, e.g. "Coalescing Operator", "Coalesced -Member Access Operator", etc. + >>> from pyasn1.type import univ + >>> univ.Null() ✊🍆 univ.Integer(123) + Integer(123) + +In addition to making the proposed operators less specialized, this +generalization also makes it easier to work with the Null Object Pattern, [3]_ +for those developers who prefer to avoid using ``None``. Operator Spelling @@ -1081,7 +1111,7 @@ major version, e.g. Python 4. (Even then, `there would be resistance `_.) Furthermore, nearly every single punctuation character on a standard keyboard -already has special meaning in Python. The only exceptions are ``$``, ``~``, +already has special meaning in Python. The only exceptions are ``$``, ``!``, ``?``, and backtick (as of Python 3). This leaves few options for a new, single- character operator. A two character spelling is more likely, such as the ``??`` and ``?.`` spellings in other programming languages, but this decreases the @@ -1124,24 +1154,24 @@ operator. - Pros: similar to existing ``or`` operator - Cons: the difference between this and ``or`` is not intuitive; punctuation is ugly -3. ``foo ? bar`` +3. ``foo ? bar ? baz`` - Pros: similar to ``??`` used in other languages - Cons: punctuation is ugly; possible conflict with IPython; not used by any other language -4. ``foo $$ bar`` +4. ``foo $$ bar $$ baz`` - Pros: pronounced "value operator" because it returns the first operand that has a "value" - Cons: punctuation is ugly; not used by any other language -5. ``foo else bar`` +5. ``foo else bar else baz`` - Pros: prettier than punctuation; uses an existing keyword - Cons: difficult or impossible to implement with Python's LL(1) parser -6. ``foo or else bar`` +6. ``foo or else bar or else baz`` - Pros: prettier than punctuation; use existing keywords - Cons: difficult or impossible to implement with Python's LL(1) parser -7. ``foo def bar`` +7. ``foo def bar def baz`` - Pros: pronounced 'default'; prettier than punctuation - Cons: difficult or impossible to implement with Python's LL(1) parser -8. ``foo then bar`` +8. ``foo then bar then baz`` - Pros: prettier than punctuation - Cons: requires a new keyword, probably can't be implemented until Python 4 (and maybe not even then) diff --git a/pep-0505/test.py b/pep-0505/test.py index 1f8a488ab..06ae0c3aa 100644 --- a/pep-0505/test.py +++ b/pep-0505/test.py @@ -1,5 +1,5 @@ ''' -This file is used for testing find-pep505.out. +This file is used for testing find-pep505.py. nc_* and Nc* are examples of null coalescing. sn_* and Sn* are examples of save navigation.