python-peps/pep-0614.rst

PEP: 614
Title: Relaxing Grammar Restrictions On Decorators
Version: $Revision$
Last-Modified: $Date$
Author: Brandt Bucher <brandtbucher@gmail.com>
Sponsor: Guido van Rossum <guido@python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 10-Feb-2020
Python-Version: 3.9
Post-History: 10-Feb-2020
Resolution:


Abstract
========

This PEP proposes allowing functions and classes to be decorated with
any valid expression.


Motivation
==========

When decorators were first being introduced, `Guido described
<https://mail.python.org/archives/list/python-dev@python.org/message/P3JD24UFFPZUUDANOAI6GZAPIGY4CVK7>`_
the motivation to limit their syntax as a preference, not a technical
requirement: 

    I have a gut feeling about this one.  I'm not sure where it comes
    from, but I have it...  So while it would be quite easy to change
    the syntax to ``@test`` in the future, I'd like to stick to with
    the more restricted form unless a real use case is presented where
    allowing ``@test`` would increase readability.

While these limitations were rarely encountered in practice, `BPO
issues <https://bugs.python.org/issue19660>`_ and `mailing list posts
<https://mail.python.org/archives/list/python-ideas@python.org/thread/UQOCJH3KOPBP7P3AVNS3OYBGZPR3V2WO/#CAOXYF4GV76AFJNCYSYMQTBM7CIPPH5M>`_
have consistently surfaced over the years requesting that they be
removed. The `most recent one
<https://mail.python.org/archives/list/python-ideas@python.org/thread/WOWD4P323DYDIGUQVWMESDWUG6QOW4MP>`_
(which `prompted this proposal
<https://mail.python.org/archives/list/python-ideas@python.org/message/FKE7ZFGUDCU5WVOE2QTD5XGMCNCOMETV>`_)
contained a good example of code using the ``PyQt5`` library that
would become more readable, idiomatic, and maintainable if the
existing restrictions were relaxed.  Slightly modified::

    buttons = [QPushButton(f'Button {i}') for i in range(10)]
    
    # Do stuff with the list of buttons...

    @buttons[0].clicked.connect
    def spam():
        ...

    @buttons[1].clicked.connect
    def eggs():
        ...
    
    # Do stuff with the list of buttons...

Currently, these decorations must be rewritten as something like::

    button_0 = buttons[0]

    @button_0.clicked.connect
    def spam():
        ...

    button_1 = buttons[1]

    @button_1.clicked.connect
    def eggs():
        ...

Further, the current grammar is already loose enough that it's trivial
to hack more complicated decorator expressions together. So rather
than disallow arbitrarily complex expressions, as intended, the
current restrictions only make them uglier and less efficient::

    # Identity function hack:

    def _(x):
        return x

    @_(buttons[0].clicked.connect)
    def spam():
        ...

    # eval hack:

    @eval("buttons[1].clicked.connect")
    def eggs():
        ...


Rationale
=========

Allowing Any Expression
-----------------------

The decision to allow *any* valid expression (and not just relaxing
the current restrictions to allow, for example, subscripting) has
been considered as the next logical step in the evolution of decorator
grammar for quite some time.  As `Guido noted
<https://mail.python.org/archives/list/python-ideas@python.org/message/CAOXYF4GV76AFJNCYSYMQTBM7CIPPH5M>`_,
during yet another mailing list thread:

    I don't think it's reasonable to constrain it less than it
    currently is but more than a general expression.

Special-casing the grammar to allow *some* useful cases would only
complicate the current situation, and all but guarantee that the
process would repeat itself sometime in the future.  Further, one
purpose of this grammatical change is to discourage the temptation to
use hacks like the ``eval`` and identity-function anti-patterns shown
above.

In short: if we're removing somewhat arbitrary restrictions, we should
remove *all* of them.


What Counts As An "Expression"
------------------------------

Throughout this document, the word "expression" is used as defined in
the `Python Language Reference
<https://docs.python.org/3.9/reference/expressions.html#grammar-token-expression>`_.
This can be summarized as "anything that's valid as a test in ``if``,
``elif``, and ``while`` blocks".  This differs subtly from a perhaps
more popular `definition
<https://docs.python.org/3/glossary.html#term-expression>`_, which can
be summarized as "anything that's valid as string input to ``eval``".

This definition of "expression" is convenient in that it fits our
needs well, and reuses the allowed grammar of existing language
constructs.  It has two subtle differences from the other definition:


Tuple Displays *Must* Be Parenthesized
''''''''''''''''''''''''''''''''''''''

This is based on an observation Guido made in the same email.
Continued immediately from above:

    Though I wouldn't allow commas-- there's no way that

    .. code::

        @f, g
        def pooh(): ...

    can make sense.

Indeed, it may even lead inexperienced readers to conclude that
several decorators are being applied, as if they were stacked.
Requiring parentheses here makes the (admittedly nonsensical) intent
clear without imposing further restrictions and grammar complications.


Named Expressions *Need Not* Be Parenthesized
'''''''''''''''''''''''''''''''''''''''''''''

Here, the the choice of syntax is unambiguous.  :pep:`572` explains
why it requires parentheses around top-level expression statements:

    This rule is included to simplify the choice for the user between 
    an assignment statement and an assignment expression -- there is
    no syntactic position where both are valid.

Since an assignment statement is not valid here, assignment
expressions should not be unnecessarily burdened with parentheses.


Specification
=============

The grammar for decorators is currently::

    decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE

This PEP proposes that it be simplified to::

    decorator: '@' namedexpr_test NEWLINE


Backwards Compatibility
=======================

This new grammar is fully backward-compatible with the existing
grammar.


Binary Matrix Multiplication
----------------------------

One interesting case is the binary ``@`` operator, which was added to
the language after decorators were first introduced. With the proposed
change to the grammar, the following decorator expression will be
unambiguous legal syntax::

    @a @ b
    def f():
        ...

However, this pathological case is unlikely to occur in practice, as
there are no known examples of binary matrix multiplication results
which may used as decorators. Likely the only people who would do this
are those who currently write similarly obfuscated toy code::

    @_(a
    @b)
    def f():
        ...


How To Teach This
=================

Decorators can continue to be taught as they always have; the average
Python programmer is likely unaware that the current restriction even
exists.


Reference Implementation
========================

The author has drafted a CPython implementation, which can be found on
`GitHub <https://github.com/brandtbucher/cpython/tree/decorators>`_.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: