Kent's Korner

by Kent S Johnson

Filtering a list

One common use of list comprehensions is to filter a list. Suppose you have a list of integers and want to filter it to just include the elements that are even. This is easy to do with a for loop:

>>> l = range(10)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> even = []
>>> for i in l:
...   if i%2==0:
...     even.append(i)
...
>>> even
[0, 2, 4, 6, 8]

This is pretty simple, but the code is at a low level of abstraction. The idea - "make a list containing all the elements of l that are even" - has turned into a four-line recipe:

• make an empty list
• iterate over l
• if the current element is even,
  • append to the new list

List comprehensions are syntactic sugar for this recipe that brings it to the level of abstraction of the original concept. Here is the list comprehension equivalent to the above loop:

>>> even = [ i for i in l if i%2==0 ]

Short and sweet! The form of this list comprehension is

[ expression for variable in sequence if condition ]

This expression has five elements:

• The brackets indicate that this is a list comprehension. (The spaces around the brackets are not needed, they are my preference.)
• expression is any valid expression using any variables in scope, including variable. In this example, expression is just variable, i.e. i.
• variable is an identifier that will receive the values of the sequence. It is analogous to the loop variable in the for loop.
• sequence is any iterable, for example a list, iterator or generator.
• condition is any expression; it is evaluated to a True or False value.

Notice how the order of elements in the list comprehension parallel the problem statement:

• "make a list" - [
• "containing all the elements of l" - i for i in l
• "that are even" - if i%2==0

Functional programming aficionados may have noticed that this list comprehension could also be written using filter() and lambda.

Transforming a list

Another common use of a list comprehension is to apply a function or expression to each element of a list, creating a new list. For example, to create a new list containing the squares of the elements of a list, use this list comprehension:

squares = [ x*x for x in l ]

I used a different name for the loop variable in this example; any valid identifier can be used. The conditional clause is optional and omitted in this example.

Again, the order of elements in the list comprehension parallel the problem statement:

• "create a new list" - [
• "containing the squares'' - x*x
• "of the elements of a list" - for x in l

The functional programming equivalent would use map().

Nested loops

List comprehensions can have multiple for clauses. This is the equivalent of nested for loops. The order of the for clauses can be confusing. With a single loop, the syntax of a list comprehension feels "inside out" compared to a regular for loop. But with nested loops, the order of for clauses in a list comprehension is the same as they would be for the equivalent nested for loops. For example, here are nested loops to flatten a list which contains lists:

>>> l = [ [1, 2], [5, 6] ]
>>> flat = []
>>> for sub in l:
...   for i in sub:
...     flat.append(i)
...
>>> flat
[1, 2, 5, 6]

Here is the equivalent list comprehension:

>>> [ i for sub in l for i in sub ]
[1, 2, 5, 6]

Notice that the order of for clauses is the same in both places.

Speed

List comprehensions are usually faster than the equivalent for loop. map() and filter() are generally faster than list comprehensions. If the function being mapped is not a built-in and can be in-lined as an expression in a list comprehension then the speed of the list comprehension may be comparable to an equivalent map() or filter() with a user-defined function or lambda.

Side effects

It is possible to use a list comprehension purely for some side-effect of the expression in the list comprehension. For example, you could print every element of a list like this:

[ sys.stdout.write(i + '\n') for i in l ]

I personally think this is bad style and should be avoided except perhaps as an optimization.

One caveat

One thing to note is that a list comprehension does create a new list. The new list is often rebound to the same name:

l = [ foo(i) for i in l ]

but since a new list is created, any other names that refer to the original list will not be changed. For example, if you have a function that filters a list using a list comprehension, the list passed to the function will not be changed.

This is rarely a problem. If you do want to change the list in place, just use slice assignment instead of normal assignment:

l[:] = [ foo(i) for i in l ]

References

There is scant mention of list comprehensions in the official Python documentation. They are mentioned in the Tutorial and in the Language Reference in Section 5.2.4 List displays.

List comprehensions were introduced in Python 2.0. Here is Andrew Kuchling's writeup - part of his "What's New in Python 2.0" document.