Comprehensions
Lecture 5B. See csse1001 for course logistics and python-comprehensions for the full reference on comprehension syntax.
Outline
- List comprehensions
- Dictionary comprehensions
- Nested comprehensions
- Two worked exercises:
disassembleandmore_capitals allandany- Filtering, and the general pattern
List comprehension
Mathematics builds sets and sums this way:
\[ \{k^2 : k \in \{0, 1, 2, 3\}\} = \{0, 1, 4, 9\} \qquad \sum_{k \in \{0,1,2,3\}} k^2 = 0^2+1^2+2^2+3^2 \]
Python mirrors this directly:
>>> [k**2 for k in range(4)]
[0, 1, 4, 9]
>>> sum(k**2 for k in range(4))
14
Dictionary comprehension
>>> {x: f"value is {x}" for x in range(3)}
{0: 'value is 0', 1: 'value is 1', 2: 'value is 2'}
>>> {x: 0 for x in "ABC"} # useful for assigning default values
{'A': 0, 'B': 0, 'C': 0}
>>> cs = "hello world"
>>> {x: cs.count(x) for x in cs}
{'h': 1, 'e': 1, 'l': 3, 'o': 2, ' ': 1, 'w': 1, 'r': 1, 'd': 1}
Nested comprehensions
Multiple for clauses can appear in one comprehension — order matters:
>>> [(a, x) for a in "ABC" for x in range(2)]
[('A', 0), ('A', 1), ('B', 0), ('B', 1), ('C', 0), ('C', 1)]
>>> [(a, x) for x in range(2) for a in "ABC"]
[('A', 0), ('B', 0), ('C', 0), ('A', 1), ('B', 1), ('C', 1)]
Implementing [(a, x) for a in "ABC" for x in range(2)] with a for-loop:
>>> xs = []
>>> for a in "ABC":
... for x in range(2):
... xs.append((a, x))
Reading the two side by side is what makes the order rule obvious — and it explains why swapping the two for clauses swaps which loop is on the outside, and therefore changes the order of the result:
Figure 1: The same computation written both ways, piece by piece. Colour keys the boxed code: magenta the output expression, blue the outer generator clause, purple the inner one. The two italic sentences are annotations, not entries in that colour key. The for clauses keep their order, but the output expression jumps from the innermost loop body to the front.
Exercise: disassemble a string
def disassemble(cs: str) -> list[str]:
""" Returns a list of characters comprising cs, maintaining order.
>>> disassemble("")
[]
>>> disassemble("abcdef")
['a', 'b', 'c', 'd', 'e', 'f']
"""
With a for-loop:
>>> def disassemble(xs: str) -> list[str]:
... ans = []
... for x in xs:
... ans.append(x)
... return ans
With a list comprehension the whole body collapses to one line:
>>> def disassemble(xs: str) -> list[str]:
... return [x for x in xs]
>>> disassemble("abcdef")
['a', 'b', 'c', 'd', 'e', 'f']
The accumulator (ans = []), the loop, the append and the return of the four-line version all fold into the single comprehension.
return [c for c in cs]
Exercise: more capitals than lower-case
Check if a string has (strictly) more capitals than lower-case letters, ignoring all other characters:
def more_capitals(cs: str) -> bool:
"""
>>> more_capitals("aAbBcCdD")
False
>>> more_capitals("CSSE 1001")
True
>>> more_capitals("")
False
"""
A useful “hack”: the integer value of True and False is 1 and 0 respectively, so sum(...) over a sequence of booleans counts how many are True:
>>> True + True + False
2
return (sum('A' <= c <= 'Z' for c in cs)
> sum('a' <= c <= 'z' for c in cs))
Exercise: all
Check if every element of a list is truthy:
def all(xs: list) -> bool:
"""
>>> all([1, 2, 3])
True
>>> all(["a", "", "c"])
False
>>> all([])
True
"""
return len(xs) == sum(bool(x) for x in xs)
Exercise: any
Check if any element of a list is truthy:
def any(xs: list) -> bool:
"""
>>> any([0, 0, 0])
False
>>> any(["a", "", "c"])
True
>>> any([])
False
"""
return sum(bool(x) for x in xs) > 0
Both are already included in base Python — no library call needed — and they are useful:
>>> xs = "HELLO WORLD"
>>> all(['A' <= x <= 'Z' for x in xs]) # is every letter a capital?
False # (space is not a capital)
>>> any(['A' <= x <= 'Z' for x in xs]) # is any letter a capital?
True
Defining your own all/any as the two exercises above do shadows the built-ins for the rest of that scope — fine as an exercise in reimplementing them, a bug anywhere else.
Filtering
Comprehension elements may be filtered with a trailing if. E.g. the numbers less than 100 divisible by both 3 and 7:
>>> [x for x in range(101) if not (x % 3) and not (x % 7)]
[0, 21, 42, 63, 84]
range(101) includes 100. It makes no difference here — 100 is divisible by neither 3 nor 7 — but range(100) is what “less than 100” asks for.
The general pattern
[x for x in xs if P(x)]
[x for x in xs for y in ys if P(x, y)]
[x for x in xs for y in ys for z in zs if P(x, y, z)]
...
where P is some predicate.
Summary
Comprehensions are a rapid way of forming lists and dictionaries that would otherwise require an accumulator for-loop.
Next: week-five-exercises (Lecture 5C). See python-comprehensions for the syntax reference and python-for-loops for the accumulator forms comprehensions shortcut.