Comprehensions

lecture
python
comprehensions
for-loops

Lecture 5B. See csse1001 for course logistics and python-comprehensions for the full reference on comprehension syntax.

Outline

  • List comprehensions
  • Dictionary comprehensions
  • Nested comprehensions
  • Two worked exercises: disassemble and more_capitals
  • all and any
  • Filtering, and the general pattern

List comprehension

Mathematics builds sets and sums this way:

\[ \{k^2 : k \in \{0, 1, 2, 3\}\} = \{0, 1, 4, 9\} \qquad \sum_{k \in \{0,1,2,3\}} k^2 = 0^2+1^2+2^2+3^2 \]

Python mirrors this directly:

>>> [k**2 for k in range(4)]
[0, 1, 4, 9]
>>> sum(k**2 for k in range(4))
14

Dictionary comprehension

>>> {x: f"value is {x}" for x in range(3)}
{0: 'value is 0', 1: 'value is 1', 2: 'value is 2'}

>>> {x: 0 for x in "ABC"}   # useful for assigning default values
{'A': 0, 'B': 0, 'C': 0}

>>> cs = "hello world"
>>> {x: cs.count(x) for x in cs}
{'h': 1, 'e': 1, 'l': 3, 'o': 2, ' ': 1, 'w': 1, 'r': 1, 'd': 1}

Nested comprehensions

Multiple for clauses can appear in one comprehension — order matters:

>>> [(a, x) for a in "ABC" for x in range(2)]
[('A', 0), ('A', 1), ('B', 0), ('B', 1), ('C', 0), ('C', 1)]

>>> [(a, x) for x in range(2) for a in "ABC"]
[('A', 0), ('B', 0), ('C', 0), ('A', 1), ('B', 1), ('C', 1)]

Implementing [(a, x) for a in "ABC" for x in range(2)] with a for-loop:

>>> xs = []
>>> for a in "ABC":
...     for x in range(2):
...         xs.append((a, x))

Reading the two side by side is what makes the order rule obvious — and it explains why swapping the two for clauses swaps which loop is on the outside, and therefore changes the order of the result:

Figure 1: The same computation written both ways, piece by piece. Colour keys the boxed code: magenta the output expression, blue the outer generator clause, purple the inner one. The two italic sentences are annotations, not entries in that colour key. The for clauses keep their order, but the output expression jumps from the innermost loop body to the front.

Exercise: disassemble a string

def disassemble(cs: str) -> list[str]:
    """ Returns a list of characters comprising cs, maintaining order.
    >>> disassemble("")
    []
    >>> disassemble("abcdef")
    ['a', 'b', 'c', 'd', 'e', 'f']
    """

With a for-loop:

>>> def disassemble(xs: str) -> list[str]:
...     ans = []
...     for x in xs:
...         ans.append(x)
...     return ans

With a list comprehension the whole body collapses to one line:

>>> def disassemble(xs: str) -> list[str]:
...     return [x for x in xs]
>>> disassemble("abcdef")
['a', 'b', 'c', 'd', 'e', 'f']

The accumulator (ans = []), the loop, the append and the return of the four-line version all fold into the single comprehension.

return [c for c in cs]

Exercise: more capitals than lower-case

Check if a string has (strictly) more capitals than lower-case letters, ignoring all other characters:

def more_capitals(cs: str) -> bool:
    """
    >>> more_capitals("aAbBcCdD")
    False
    >>> more_capitals("CSSE 1001")
    True
    >>> more_capitals("")
    False
    """

A useful “hack”: the integer value of True and False is 1 and 0 respectively, so sum(...) over a sequence of booleans counts how many are True:

>>> True + True + False
2
return (sum('A' <= c <= 'Z' for c in cs)
        > sum('a' <= c <= 'z' for c in cs))

Exercise: all

Check if every element of a list is truthy:

def all(xs: list) -> bool:
    """
    >>> all([1, 2, 3])
    True
    >>> all(["a", "", "c"])
    False
    >>> all([])
    True
    """
return len(xs) == sum(bool(x) for x in xs)

Exercise: any

Check if any element of a list is truthy:

def any(xs: list) -> bool:
    """
    >>> any([0, 0, 0])
    False
    >>> any(["a", "", "c"])
    True
    >>> any([])
    False
    """
return sum(bool(x) for x in xs) > 0

Both are already included in base Python — no library call needed — and they are useful:

>>> xs = "HELLO WORLD"

>>> all(['A' <= x <= 'Z' for x in xs])   # is every letter a capital?
False                                    # (space is not a capital)

>>> any(['A' <= x <= 'Z' for x in xs])   # is any letter a capital?
True

Defining your own all/any as the two exercises above do shadows the built-ins for the rest of that scope — fine as an exercise in reimplementing them, a bug anywhere else.

Filtering

Comprehension elements may be filtered with a trailing if. E.g. the numbers less than 100 divisible by both 3 and 7:

>>> [x for x in range(101) if not (x % 3) and not (x % 7)]
[0, 21, 42, 63, 84]

range(101) includes 100. It makes no difference here — 100 is divisible by neither 3 nor 7 — but range(100) is what “less than 100” asks for.

The general pattern

[x for x in xs if P(x)]
[x for x in xs for y in ys if P(x, y)]
[x for x in xs for y in ys for z in zs if P(x, y, z)]
...

where P is some predicate.

Summary

Comprehensions are a rapid way of forming lists and dictionaries that would otherwise require an accumulator for-loop.

Next: week-five-exercises (Lecture 5C). See python-comprehensions for the syntax reference and python-for-loops for the accumulator forms comprehensions shortcut.