Kent's Korner
by Kent S Johnson

2006-12-19 18:22:39


First-class functions

Functions in Python are first-class objects. In particular, functions can be passed as values to other functions and returned as values from functions. For example, here is a function that takes a function as an argument and prints the name of the given function:

>>> def show_func(f):
...   print f.__name__

and here is a sample usage:

>>> def foo():
...   print 'foo here'

>>> show_func(foo)

Here is a function that creates a new function and returns it as its result. In this case, make_adder() creates a function that adds a constant to its argument:

>>> def make_adder(x):
...   def adder(y):
...     return x+y
...   return adder

>>> a=make_adder(5)
>>> a(10)

Wrapping a function

Combining both of these ideas, we can make a function that takes a function as a parameter and returns a new function that is somehow based on the parameter. For example, we might wrap the passed-in function with a new function that logs calls to it:

>>> def show_call(f):
...   def shown(*args, **kwds):
...     print 'Calling', f.__name__
...     return f(*args, **kwds)
...   return shown

>>> bar = show_call(foo)
>>> bar()
Calling foo
foo here

If we rebind the result of show_foo() to the same name, we have effectively replaced the original function with the wrapped version:

>>> foo = show_call(foo)
>>> foo()
Calling foo
foo here

Decorators are syntactic sugar

If you have followed this discussion so far, then you understand the basics of decorators, because a decorator is syntactic sugar for what we just did. Specifically, the code

def foo():

is equivalent to this code:

def foo():

foo = show_call(foo)

Any function that accepts a function as its single argument and returns a function or other callable (or a descriptor, but I'm not going to talk about them) can be used as a decorator.

So what's the big deal?

In a way, there is no big deal; decorators don't add any new functionality to Python. They are a new syntax for an old idea. But decorators also show that syntax matters - they are very popular and widely used in modern Python code. Decorators are very useful and widely used to factor out common code that must be applied to several functions. Putting the decoration before the function with a distinctive syntax makes the intent and function of the code much clearer.

Well-behaved decorators

With a simple decorator as shown above, there are some notable differences between the original function and the decorated function:

>>> def bar():
...   ''' Function bar() '''
...   pass

>>> bar.__name__, bar.__doc__, bar.__module__
('bar', ' Function bar() ', '__main__')

>>> import inspect
>>> inspect.getargspec(bar)
([], None, None, None)

>>> bar2=show_call(bar)
>>> bar2.__name__, bar2.__doc__, bar2.__module__
('shown', None, '__main__')

>>> inspect.getargspec(bar2)
([], 'args', 'kwds', None)

Function attributes are not copied to the wrapper function, and the signature of the wrapped function is different than that of the original.

The attributes of the original function can be preserved by copying them from the original function. Here is a better show_call():

def show_call(f):
  def shown(*args, **kwds):
    print 'Calling', f.__name__
    return f(*args, **kwds)
  shown.__name__ = f.__name__
  shown.__doc__ = f.__doc__
  shown.__module__ = f.__module__
  return shown

With this version, the attributes are correct, though the signature is still wrong:

>>> bar2=show_call(bar)
>>> bar2.__name__, bar2.__doc__, bar2.__module__
('bar', ' Function bar() ', '__main__')

The functools module (new in Python 2.5) provides a decorator for decorators, called wrap(), that does just this. The above example could be written as:

>>> def show_call(f):
...   @wraps(f)
...   def shown(*args, **kwds):
...     print 'Calling', f.__name__
...     return f(*args, **kwds)
...   return shown

>>> bar2=show_call(bar)
>>> bar2.__name__, bar2.__doc__, bar2.__module__
('bar', ' Function bar() ', '__main__')

Michele Simionato's decorator module includes a decorator that uses eval() to create decorators that preserve the signature of the decorated function.

Decorators with arguments

You might have noticed something new in the example above: the decorator itself takes an argument. How does that work?

Suppose we have this code:

def do_nothing(*args, **kwds):
  return f(*args, **kwds)

By the definition of decorator syntax, this is equivalent to:

def do_nothing(*args, **kwds):
  return f(*args, **kwds)
do_nothing = wraps(f)(do_nothing)

For this to make sense, wraps() must be a decorator factory - a function whose return value is itself a decorator. In other words, wraps() is a function whose return value is a function that takes a function as its argument and returns a function!


How about a simple example? Can we make a decorator that repeatedly calls the function it is decorating? For a fixed number of repeats, this is straightforward:

>>> def repeat3(f):
...   def inner(*args, **kwds):
...     f(*args, **kwds)
...     f(*args, **kwds)
...     return f(*args, **kwds)
...   return inner

>>> f3=repeat3(foo)
>>> f3()
foo here
foo here
foo here

But suppose we want to provide the number of repeats as an argument? Then we need a function whose return value is a decorator. The returned decorator will be similar to repeat3() above. This requires another level of nested functions:

>>> def repeat(n):
...   def repeatn(f):
...     def inner(*args, **kwds):
...       for i in range(n):
...         ret = f(*args, **kwds)
...       return ret
...     return inner
...   return repeatn

Here repeat() is the decorator factory, repeatn() is the actual decorator, and inner() is the wrapper function that will be called instead of the original. Here it is in use with decorator syntax:

>>> @repeat(4)
... def bar():
...   print 'bar here'

>>> bar()
bar here
bar here
bar here
bar here

Real-world examples

Good beginner examples of real-world decorators are hard to come by. Decorators tend to be either very simple, in which case they don't add much to the examples above, or significantly more complex and difficult to understand. Also real-world decorators are often composed from other decorators and don't stand alone.

The Python Decorator Library is a good source of fairly simple examples. The Synchronize decorator is one of the simplest. Easy Dump of Function Arguments is a more complete version of my show_call() example.

TurboGears uses its @expose decorator extensively to configure the output of methods that are exposed to the web. The source is here.

Further reading

What's new in Python 2.4 gives a brief, informal introduction to decorators.

PEP 318 is the formal specification for decorators.

The Python Decorator Library has many examples of useful decorators.

The documentation for Michele Simionato's decorator module has a description of the problem of creating well-behaved decorators and many examples of decorators. His decorator module itself provides a way to create decorators that preserve function signatures.

David Mertz has written a very readable Charming Python column called Decorators make magic easy.

This essay has been translated to Portuguese by Ruivaldo Neto.

© Kent S Johnson Creative Commons License

Short, introductory essays about Python.

Kent's Korner home


Other Essays