PEP: 318 Title: Decorators for Functions and Methods Version: $Revision$ Last-Modified: $Date$ Author: Kevin D. Smith , Jim Jewett , Skip Montanaro Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 05-Jun-2003 Python-Version: 2.4 Post-History: 09-Jun-2003, 10-Jun-2003, 27-Feb-2004, 23-Mar-2004 Abstract ======== The current method for declaring class and static methods is awkward and can lead to code that is difficult to understand. Ideally, these transformations should be made at the same point in the code where the declaration itself is made. This PEP introduces new syntax for transformations of a declaration. Motivation ========== The current method of applying a transformation to a function or method places the actual translation after the function body. For large functions this separates a key component of the function's behavior from the definition of the rest of the function's external interface. For example:: def foo(self): perform method operation foo = classmethod(foo) This becomes less readable with longer methods. It also seems less than pythonic to name the function three times for what is conceptually a single declaration. A solution to this problem is to move the transformation of the method closer to the method's own declaration. While the new syntax is not yet final, the intent is to replace:: def foo(cls): pass foo = synchronized(lock)(foo) foo = classmethod(foo) with an alternative that places the decoration in the function's declaration:: @classmethod @synchronized(lock) def foo(cls): pass Modifying classes in this fashion is also possible, though the benefits are not as immediately apparent. Almost certainly, anything which could be done with class decorators could be done using metaclasses, but using metaclasses is sufficiently obscure that there is some attraction to having an easier way to make simple modifications to classes. For Python 2.4, only function decorators are being added. Background ========== There is general agreement that syntactic support is desirable to the current state of affairs. Guido mentioned `syntactic support for decorators`_ in his DevDay keynote presentation at the `10th Python Conference`_, though `he later said`_ it was only one of several extensions he proposed there "semi-jokingly". `Michael Hudson raised the topic`_ on ``python-dev`` shortly after the conference, attributing the bracketed syntax to an earlier proposal on ``comp.lang.python`` by `Gareth McCaughan`_. .. _syntactic support for decorators: http://www.python.org/doc/essays/ppt/python10/py10keynote.pdf .. _10th python conference: http://www.python.org/workshops/2002-02/ .. _michael hudson raised the topic: http://mail.python.org/pipermail/python-dev/2002-February/020005.html .. _he later said: http://mail.python.org/pipermail/python-dev/2002-February/020017.html .. _gareth mccaughan: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrna40k88.2h9o.Gareth.McCaughan%40g.local Class decorations seem like an obvious next step because class definition and function definition are syntactically similar. The discussion continued on and off on python-dev from February 2002 through July 2004. Many hundreds of posts were made, with people proposing many possible syntax variations. Guido took a list of proposals to `EuroPython 2004`_, where a discussion took place. Subsequent to this, he decided that for 2.4a2 we'd have the Java-style @decorator syntax. Barry Warsaw named this the 'pie-decorator' syntax, in honor of the Pie-thon Parrot shootout which was announced about the same time as the decorator syntax, and because the @ looks a little like a pie. Guido `outlined his case`_ on Python-dev, including `this piece`_ on the various rejected forms. .. _EuroPython 2004: http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf .. _outlined his case: http://mail.python.org/pipermail/python-dev/2004-August/authors.html .. _this piece: http://mail.python.org/pipermail/python-dev/2004-August/046672.html Design Goals ============ The new syntax should * work for arbitrary wrappers, including user-defined callables and the existing builtins ``classmethod()`` and ``staticmethod()`` * work with multiple wrappers per definition * make it obvious what is happening; at the very least it should be obvious that new users can safely ignore it when writing their own code * not make future extensions more difficult * be easy to type; programs that use it are expected to use it very frequently * not make it more difficult to scan through code quickly. It should still be easy to search for all definitions, a particular definition, or the arguments that a function accepts * not needlessly complicate secondary support tools such as language-sensitive editors and other "`toy parser tools out there`_" .. _toy parser tools out there: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=mailman.1010809396.32158.python-list%40python.org Andrew Kuchling has links to a bunch of the discussions about motivations `in his blog`_. .. _in his blog: http://www.amk.ca/diary/archives/cat_python.html#003255 Proposed Syntax =============== The current syntax for function decorators as implemented in Python 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass This is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) without the intermediate assignment to the variable ``func``. The decorators are near the function declaration. The @ sign makes it clear that something new is going on here. The decorator statement is limited in what it can accept - arbitrary expressions will not work. Guido preferred this because of a `gut feeling`_ .. _gut feeling: http://mail.python.org/pipermail/python-dev/2004-August/046711.html Alternate Proposals =================== Several other syntaxes have been proposed:: def func(arg1, arg2, ...) as dec1, dec2, ...: pass The absence of brackets makes it cumbersome to break long lists of decorators across multiple lines, and the keyword "as" doesn't have the same meaning as its use in the ``import`` statement. Plenty of `alternatives to "as"`_ have been proposed. :-) .. _alternatives to "as": http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=mailman.236.1079968472.742.python-list%40python.org&rnum=2&prev=/groups%3Fq%3Dpython%2Bpep%2B318%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26selm%3Dmailman.236.1079968472.742.python-list%2540python.org%26rnum%3D2 :: def [dec1, dec2, ...] func(arg1, arg2, ...): pass This form has the disadvantage that the decorators visually assume higher priority than the function name and argument list. :: def func [dec1, dec2, ...] (arg1, arg2, ...): pass Quixote's `Python Template Language`_ uses this form, but only supports a single decorator chosen from a restricted set. For short lists it works okay, but for long list it separates the argument list from the function name. .. _Python Template Language: http://www.mems-exchange.org/software/quixote/doc/PTL.html :: using: dec1 dec2 ... def foo(arg1, arg2, ...): pass The function definition is not nested within the using: block making it impossible to tell which objects following the block will be decorated. Nesting the function definition within the using: block suggests nesting of namespaces that doesn't exist. The name ``foo`` would actually exist at the same scope as the using: block. Finally, it would require the introduction of a new keyword. The obvious alternative that nests the function within the block :: using: dec1 dec2 ... def foo(arg1, arg2, ...): pass has its own set of drawbacks. Having the minimal indent level be three deep for methods is painful for those using limited-width windows. The inconsistent indentation between methods of the same class with and without decorators would be a readability problem. Finally, adding or removing decorators would require reindenting the entire function/method body. Guido proposed and implementated a patch to support interpretation of a `list of decorators`_ as a prefix to function definitions :: [dec1, dec2, ...] def foo(arg1, arg2, ...): pass For a while this was Guido's preferred solution, but negative sentiment ran high, mostly because that syntax, though useless except for side effects of the list, is already legal and thus creates a special case. .. _list of decorators: http://python.org/sf/926860 Another variant on the list syntax that was initially favored was:: def func(arg1, arg2, ...) [dec1, dec2]: pass Guido decided `he preferred`_ having the decorators on the line before the 'def', because it was felt that a long argument list would mean that the decorators would be 'hidden' .. _he preferred: http://mail.python.org/pipermail/python-dev/2004-March/043756.html Phillip Eby and Jp Calderone both proposed variants that required no new syntax, but instead used some fairly advanced introspection to provide decorator-like behavoiur, but Guido was unimpressed by these, stating:: Using functions with "action-at-a-distance" through sys.settraceback may be okay for an obscure feature that can't be had any other way yet doesn't merit changes to the language, but that's not the situation for decorators. The widely held view here is that decorators need to be added as a syntactic feature to avoid the problems with the postfix notation used in 2.2 and 2.3. Decorators are slated to be an important new language feature and their design needs to be forward-looking, not constrained by what can be implemented in 2.3. A `page on the Python Wiki`_ was created to summarize a number of the proposals. Once it stabilizes perhaps someone would care to incorporate its content into this PEP (hint, hint). .. _page on the Python Wiki: http://www.python.org/moin/PythonDecorators Why @? ------ There is some history in Java using @ initially as a marker in `Javadoc comments`_ and later in ... mumble mumble ... The fact that @ was previously unused as a token in Python also means it's clear there is no possibility of such code being parsed by an earlier version of Python, leading to possibly subtle semantic bugs. That said, @ is still a fairly arbitrary choice. Some have suggested using | instead. For syntax options which use a list-like syntax (no matter where it appears) to specify the decorators a few alternatives were proposed: ``[|...|]``, ``*[...]*``, and ``<...>``. None gained much traction. The alternatives which involve square brackets only serve to make it obvious that the decorator construct is not a list. They do nothing to make parsing any easier. The '<...>' alternative presents parsing problems because '<' and '>' already parse as un-paired. They present a further parsing ambiguity because a right angle bracket might be a greater than symbol instead of a closer for the decorators. .. _Javadoc comments: http://java.sun.com/j2se/javadoc/writingdoccomments/ Current Implementation ====================== Guido asked for a voluteer to implement his preferred syntax, and Mark Russell stepped up and posted a `patch`_ to SF. The syntax accepted for 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) though without the intermediate creation of a variable named ``func``. .. _patch: http://www.python.org/sf/979728 A `previous patch`_ from Michael Hudson which implements the list-after-def syntax is also still kicking around. .. _previous patch: http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff Examples ======== Much of the discussion on ``comp.lang.python`` and the ``python-dev`` mailing list focuses on the use of decorators as a cleaner way to use the ``staticmethod()`` and ``classmethod()`` builtins. This capability is much more powerful than that. This section presents some examples of use. 1. Define a function to be executed at exit. Note that the function isn't actually "wrapped" in the usual sense. :: def onexit(f): import atexit atexit.register(f) return f @onexit def func(): ... 2. Define a class with a singleton instance. Note that once the class disappears enterprising programmers would have to be more creative to create more instances. (From Shane Hathaway on ``python-dev``.) :: def singleton(cls): instances = {} def getinstance(): if cls not in instances: instances[cls] = cls() return instances[cls] return getinstance @singleton class MyClass: ... 3. Add attributes to a function. (Based on an example posted by Anders Munch on ``python-dev``.) :: def attrs(**kwds): def decorate(f): for k in kwds: setattr(f, k, kwds[k]) return f return decorate @attrs(versionadded="2.2", author="Guido van Rossum") def mymethod(f): ... 4. Enforce function argument and return types. (Note that this is not exactly correct, as the returned new_f doesn't have "func" as its func_name attribute.) :: def accepts(*types): def check_accepts(f): assert len(types) == f.func_code.co_argcount def new_f(*args, **kwds): for (a, t) in zip(args, types): assert isinstance(a, t), \ "arg %r does not match %s" % (a,t) return f(*args, **kwds) return new_f return check_accepts def returns(rtype): def check_returns(f): def new_f(*args, **kwds): result = f(*args, **kwds) assert isinstance(result, rtype), \ "return value %r does not match %s" % (result,rtype) return result return new_f return check_returns @accepts(int, (int,float)) @returns((int,float)) def func(arg1, arg2): return arg1 * arg2 5. Declare that a class implements a particular (set of) interface(s). This is from a posting by Bob Ippolito on ``python-dev`` based on experience with `PyProtocols`_. .. _PyProtocols: http://peak.telecommunity.com/PyProtocols.html :: def provides(*interfaces): """ An actual, working, implementation of provides for the current implementation of PyProtocols. Not particularly important for the PEP text. """ def provides(typ): declareImplementation(typ, instancesProvide=interfaces) return typ return provides class IBar(Interface): """Declare something about IBar here""" @provides(IBar) class Foo(object): """Implement something here...""" Of course, all these examples are possible today, though without syntactic support. Open Issues =========== 1. It's not yet certain that class decorators will be incorporated into the language at this point. Guido expressed skepticism about the concept, but various people have made some `strong arguments`_ (search for ``PEP 318 - posting draft``) on their behalf in ``python-dev``. 2. Decorators which wrap a function and return a different function should be able to easily change the func_name attribute without constructing it with new.function(). Perhaps the func_name attribute should be writable. .. _strong arguments: http://mail.python.org/pipermail/python-dev/2004-March/thread.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: