SyntaxPilot
Courses/Python Coding Questions/Lesson 19
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Python Coding Questions · Lesson 19 of 30

Closure and generator questions

counter factory and nonlocallate-binding lambdas in a loopwrite your own range() as a generator

These are the notes for Python Coding Questions, episode 19: closure and generator questions. Last episode we wrote decorators: functions that build other functions. This time we look at what those inner functions remember. Three interview questions: a counter factory (and the nonlocal keyword), lambdas made in a loop that all give the same answer, and writing your own range() as a generator with yield.

Every program was run with Python 3.13 (python3 main.py), the SyntaxPilot compiler's Python command. Every output below is real (times and memory addresses change a little from run to run).

Two words first. A scope is the region of code where a name can be used. Python looks a name up in four scopes, in this order, called LEGB: Local (the current function), Enclosing (the functions around it), Global (the file), Built-in (print, len ...).

x = "global"
def outer():
    x = "enclosing"
    def inner():
        print(x)          # not local: found in the enclosing function
    inner()
outer()
print(x)

Output:

enclosing
global

Q1: A counter factory

Real-life example: a clinic has a token machine: each patient gets the next number, 1, 2, 3. A second clinic opens next door and needs its own machine, also starting at 1; the two counts must never mix. We want a factory: a function that makes token machines, where each machine keeps its own count.

Attempt 1: one global counter

count = 0
def next_token():
    global count
    count += 1
    return count

print(next_token(), next_token())   # clinic A
print(next_token())                 # clinic B starts at 3?!

Output:

1 2
3

Why it fails: there's only one count in the whole program, so every caller shares it. We need a separate count for each machine.

Attempt 2: the count inside a factory

make_counter sets up a count, defines next_token inside, and returns it:

def make_counter():
    count = 0
    def next_token():
        count += 1
        return count
    return next_token

clinic_a = make_counter()
print(clinic_a())

Output:

Traceback (most recent call last):
  File "main.py", line 9, in <module>
    print(clinic_a())
          ~~~~~~~~^^
  File "main.py", line 4, in next_token
    count += 1
    ^^^^^
UnboundLocalError: cannot access local variable 'count' where it is not associated with a value

Why it fails: count += 1 means count = count + 1, an assignment. When a function assigns to a name anywhere in its body, Python treats that name as the function's own local variable for the whole function. The local count has no value yet when count + 1 reads it, so: UnboundLocalError. (Just reading an outer variable is fine; assigning is what makes it local.)

The fix: nonlocal

nonlocal count says: "count is the variable of the function around me, not a new local one".

def make_counter(start=0):
    count = start
    def next_token():
        nonlocal count          # the count from make_counter
        count += 1
        return count
    return next_token

clinic_a = make_counter()
clinic_b = make_counter(100)
print(clinic_a(), clinic_a(), clinic_a())
print(clinic_b(), clinic_b())
print(clinic_a())
print(clinic_a.__closure__[0].cell_contents)

Output:

1 2 3
101 102
4
4
  • make_counter has already returned, yet its count stays alive, because next_token still needs it. An inner function together with the outer variables it remembers is a closure. __closure__ holds those remembered variables (Python calls each one a cell).
  • Every call to make_counter runs its body again and creates a new count, so each machine has its own state.
  • global is for module-level (file-level) variables; nonlocal is for the enclosing function's variables. nonlocal can't reach a global:
count = 0
def f():
    nonlocal count
    count += 1

Output:

  File "main.py", line 3
    nonlocal count
    ^^^^^^^^^^^^^^
SyntaxError: no binding for nonlocal 'count' found

Complexity: each token is O(1) time; each machine stores one number, O(1) space.

A mutable object doesn't need nonlocal because you never re-assign the name, you only change the object:

def make_log():
    entries = []
    def log(msg):
        entries.append(msg)      # changes the list, no assignment to entries
        return len(entries)
    return log

log = make_log()
print(log("opened"), log("sold 3 chai"))

Output:

1 2

Q2: Lambdas made in a loop

A lambda is a small, one-line function with no name: lambda x: x * 2 is the same as def f(x): return x * 2.

Real-life example: a teacher hands out three cards. Each card says "multiply by the number on the board". She writes 1 on the board and hands out a card, then 2, then 3. Later the students use their cards, but the board now says 3, so every card multiplies by 3. The card never held the number, only where to look.

Attempt 1

tables = []
for i in range(1, 4):
    tables.append(lambda x: x * i)   # x times 1, 2, 3?

print([t(10) for t in tables])
print(i)

Output:

[30, 30, 30]
3

We expected [10, 20, 30].

Why it fails

A closure remembers the variable i, not its value at the moment the lambda is made. It looks up i when the lambda is called. By then the loop has finished and i is 3. This is called late binding. All three lambdas share the same i:

tables = [lambda x: x * i for i in range(1, 4)]
print([t(10) for t in tables])
print(tables[0].__closure__[0] is tables[2].__closure__[0])

Output:

[30, 30, 30]
True

The fix: save the value when the function is made

Episode 18's rule: a default value is evaluated once, when the function is defined. Last time that was a bug; here it's exactly what we want. i=i makes a parameter i whose default is the loop's current i, saved right now. Or use functools.partial, which stores argument values inside a new function:

from functools import partial

tables = []
for i in range(1, 4):
    tables.append(lambda x, i=i: x * i)   # i saved NOW
print([t(10) for t in tables])

def times(x, n):
    return x * n
tables = [partial(times, n=i) for i in range(1, 4)]
print([t(10) for t in tables])

Output:

[10, 20, 30]
[10, 20, 30]

A third fix is a factory function (a fresh enclosing scope for every value):

def make_times(n):
    return lambda x: x * n      # each call has its own n

tables = [make_times(i) for i in range(1, 4)]
print([t(10) for t in tables])

Output:

[10, 20, 30]

You'll meet this bug wherever functions are created in a loop: GUI buttons, callbacks, threads. Calling each lambda is O(1).

Q3: Write your own range()

Real-life example: at a dinner, one cook makes 1,000 rotis before anyone eats: the guests wait, and the pile fills the kitchen. Another cook makes one roti when a plate is empty: always hot, no pile. When a guest asks for the next one, she carries on from where she stopped. In Python, that second cook is a generator; handing over one roti is yield; asking for the next is next().

Attempt 1: build a list

import sys, time

def my_range(stop):
    out = []
    i = 0
    while i < stop:
        out.append(i)
        i += 1
    return out

start = time.perf_counter()
nums = my_range(5_000_000)
print("first value:", nums[0])
print(f"waited {time.perf_counter() - start:.2f} s for it")
print(sys.getsizeof(nums), "bytes")

Output:

first value: 0
waited 0.31 s for it
43947864 bytes

We wanted one number and still waited for all five million to be made, and the list alone takes about 44 MB (sys.getsizeof gives an object's own size in bytes; the int objects inside take even more). A trillion numbers would need terabytes. But Python's own range is instant at any size:

import sys
r = range(10**12)
print(sys.getsizeof(r), "bytes")
print(r[0], r[1], len(r))

Output:

48 bytes
0 1 1000000000000

yield: hand out one value, then pause

A function with yield in its body is a generator function. Calling it doesn't run the body: it returns a generator object, a paused function. Each next(g) runs it up to the next yield, hands out that value, and freezes there with all its local variables. The next next(g) continues from that line. When the function ends, next raises StopIteration.

def my_range(stop):
    i = 0
    while i < stop:
        yield i                 # hand out one value, pause here
        i += 1

g = my_range(3)
print(g)
print(next(g), next(g), next(g))
print(next(g))

Output:

<generator object my_range at 0x7f67ddeed9c0>
0 1 2
Traceback (most recent call last):
  File "main.py", line 10, in <module>
    print(next(g))
          ~~~~^^^
StopIteration

A for loop calls next() for you and stops quietly at StopIteration. You can watch the pausing:

def cook(n):
    for k in range(1, n + 1):
        print(f"  (cooking roti {k})")
        yield f"roti {k}"
    print("  (kitchen closed)")

for roti in cook(2):
    print("eating", roti)

Output:

  (cooking roti 1)
eating roti 1
  (cooking roti 2)
eating roti 2
  (kitchen closed)

Attempt 2: add start and step

def my_range(start, stop, step=1):
    i = start
    while i < stop:
        yield i
        i += step

print(list(my_range(2, 10, 3)))
print(list(my_range(10, 0, -3)))
print(list(range(10, 0, -3)))

Output:

[2, 5, 8]
[]
[10, 7, 4, 1]

Why it fails: with a negative step we count down, so the loop should run while i > stop. 10 < 0 is false, so the loop never starts and we get [].

The fix: the check depends on the step's sign

import sys

def my_range(start, stop=None, step=1):
    if stop is None:                 # my_range(5) means 0 to 5
        start, stop = 0, start
    if step == 0:
        raise ValueError("step must not be zero")
    i = start
    while (step > 0 and i < stop) or (step < 0 and i > stop):
        yield i
        i += step

print(list(my_range(5)))
print(list(my_range(2, 10, 3)))
print(list(my_range(10, 0, -3)))
print(sys.getsizeof(my_range(10**12)), "bytes")
big = my_range(10**12)
print(next(big), next(big))
g = my_range(4)
print(sum(g), sum(g))

Output:

[0, 1, 2, 3, 4]
[2, 5, 8]
[10, 7, 4, 1]
216 bytes
0 1
6 0
  • One argument means stop, starting from 0, like the real range.
  • A step of 0 would never reach stop, so raise ValueError (the real range does too).
  • A trillion-number generator is 216 bytes: only i, stop, step and the paused position are kept. O(1) memory, O(1) time per value.
  • A generator can be walked once. The first sum(g) used it up, so the second gives 0. Make a new generator, or keep a list if you need the values twice.

Our version still differs from the real range in two ways worth knowing: range is a sequence (you can index it, take len(), and loop over it many times), while a generator only goes forward, once.

print(range(10, 0, -3)[1], len(range(10, 0, -3)))
def gen():
    yield 1
print(len(gen()))

Output:

7 4
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    print(len(gen()))
          ~~~^^^^^^^
TypeError: object of type 'generator' has no len()

Common mistakes (and what you really see)

Mistake What you see Why Fix
one global counter clinic B's first token is 3 one count for everyone a factory that makes a closure
count += 1 in the inner function UnboundLocalError: cannot access local variable 'count' where it is not associated with a value assigning makes the name local nonlocal count
nonlocal for a module variable SyntaxError: no binding for nonlocal 'count' found nonlocal only sees enclosing functions global count (or better, a closure)
lambda x: x * i in a loop [30, 30, 30] i is read at call time lambda x, i=i: x * i, partial, or a factory
returning a list of n values 44 MB and a wait for 5,000,000 the whole pile first yield one at a time
print(g) <generator object my_range at 0x...> a generator is not a list list(g) or loop over it
next(g) after the end StopIteration no values left use a for loop, or next(g, default)
while i < stop with a negative step [] for my_range(10, 0, -3) counting down i > stop when step < 0
sum(g) twice 6, then 0 a generator is used up make a new one, or a list
len(g) TypeError: object of type 'generator' has no len() a generator doesn't know its length len(list(g)) or count while looping

next(g, default) returns the default instead of raising at the end:

g = iter([7])
print(next(g, "done"), next(g, "done"))

Output:

7 done

Your turn: a running average (solution)

Question: write make_averager(). Each call adds a new mark and returns the average of all marks so far: 10.0, 15.0, 30.0.

Idea: a closure that keeps a running total and a count, changed with nonlocal. Keeping two numbers (not the whole list) makes each call O(1) time and O(1) space.

def make_averager():
    total = 0
    count = 0
    def add(mark):
        nonlocal total, count
        total += mark
        count += 1
        return total / count
    return add

avg = make_averager()
print(avg(10), avg(20), avg(60))
other = make_averager()
print(other(99))

Output:

10.0 15.0 30.0
99.0

Bonus: chunks(items, size), a generator that hands out a list in pieces of size items (the last piece may be shorter):

def chunks(items, size):
    if size <= 0:
        raise ValueError("size must be positive")
    for start in range(0, len(items), size):
        yield items[start:start + size]

print(list(chunks([1, 2, 3, 4, 5, 6, 7], 3)))
print(list(chunks([], 3)))

Output:

[[1, 2, 3], [4, 5, 6], [7]]
[]

Mini project: a token display board

A closure counter per counter desk, and a generator that merges their calls in turn, the way a bank's display board shows "Token A1, B101, A2, ...".

def make_desk(prefix, start=1):
    n = start - 1
    def call():
        nonlocal n
        n += 1
        return f"{prefix}{n}"
    return call

def board(desks, rounds):
    for _ in range(rounds):
        for desk in desks:
            yield desk()

cash = make_desk("A")
loans = make_desk("B", 101)
for token in board([cash, loans], 3):
    print("Now serving:", token)
print("next cash token:", cash())

Output:

Now serving: A1
Now serving: B101
Now serving: A2
Now serving: B102
Now serving: A3
Now serving: B103
next cash token: A4

Practice questions (with solutions)

1. Write an endless fibonacci() generator and take the first 10 values.

Solution

An endless generator never returns; the caller decides when to stop. itertools.islice takes the first n values from any iterator.

from itertools import islice

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

print(list(islice(fibonacci(), 10)))
g = fibonacci()
print([next(g) for _ in range(5)])

Output:

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
[0, 1, 1, 2, 3]

2. Write countdown(n) as a generator: n, n − 1, ..., 1, then "Go!".

Solution
def countdown(n):
    while n > 0:
        yield n
        n -= 1
    yield "Go!"

print(list(countdown(3)))
print(list(countdown(0)))

Output:

[3, 2, 1, 'Go!']
['Go!']

3. What does this print, and how do you fix it?

buttons = {}
for name in ["chai", "coffee"]:
    buttons[name] = lambda: f"You ordered {name}"
print(buttons["chai"]())
Solution

Late binding: both lambdas read name when called, after the loop left it at "coffee". Save it with a default.

buttons = {}
for name in ["chai", "coffee"]:
    buttons[name] = lambda: f"You ordered {name}"
print(buttons["chai"]())

for name in ["chai", "coffee"]:
    buttons[name] = lambda name=name: f"You ordered {name}"
print(buttons["chai"]())

Output:

You ordered coffee
You ordered chai

4. A generator expression: the sum of the squares of 1 to 10 million, in O(1) memory.

Solution

(x * x for x in ...) with round brackets is a generator expression: like a list comprehension, but lazy (values made one at a time).

import sys
gen = (x * x for x in range(1, 10_000_001))
print(sys.getsizeof(gen), "bytes")
print(sum(gen))

Output:

200 bytes
333333383333335000000

5. Write make_limited(func, n): a closure that lets func run at most n times, then returns "limit reached".

Solution
def make_limited(func, n):
    left = n
    def wrapper(*args, **kwargs):
        nonlocal left
        if left == 0:
            return "limit reached"
        left -= 1
        return func(*args, **kwargs)
    return wrapper

send_otp = make_limited(lambda phone: f"OTP sent to {phone}", 2)
print(send_otp("98xxxx01"))
print(send_otp("98xxxx01"))
print(send_otp("98xxxx01"))

Output:

OTP sent to 98xxxx01
OTP sent to 98xxxx01
limit reached

Quick quiz

What does this print?

def squares(n):
    for i in range(n):
        yield i * i

g = squares(4)
print(next(g), list(g), list(g))

(A) 0 [0, 1, 4, 9] [0, 1, 4, 9] (B) 0 [1, 4, 9] [] (C) 0 [1, 4, 9] [1, 4, 9] (D) [0, 1, 4, 9] [] []

Answer

(B). next(g) takes the first value, 0. The first list(g) takes everything left: [1, 4, 9]. The generator is now used up, so the second list(g) is [].

def squares(n):
    for i in range(n):
        yield i * i

g = squares(4)
print(next(g), list(g), list(g))

Output:

0 [1, 4, 9] []

Interview questions

  1. What is a closure? An inner function together with the variables from its enclosing function that it uses. They stay alive after the outer function returns (f.__closure__).
  2. Why does count += 1 inside an inner function raise UnboundLocalError? Assignment makes count local to the inner function for its whole body; it's read before it has a value. Declare nonlocal count.
  3. global vs nonlocal? global refers to a module-level name; nonlocal to a name in the nearest enclosing function.
  4. What is LEGB? Python's name lookup order: Local, Enclosing, Global, Built-in.
  5. Why do lambdas created in a loop all return the same value? Late binding: closures look up variables at call time, and the loop variable has its final value by then. Fix with a default argument (i=i), functools.partial, or a factory function.
  6. What is a generator? A function with yield; calling it returns an iterator that produces values lazily, pausing at each yield and resuming on next().
  7. Generator vs list? A generator uses O(1) memory and produces values on demand, but can be walked only once and has no len() or indexing. A list holds everything in memory.
  8. What happens when a generator finishes? next() raises StopIteration; for loops handle it for you.
  9. What's a generator expression? (expr for x in iterable): a lazy version of a list comprehension.
  10. How would you implement range yourself? A generator taking start, stop=None, step=1; handle one argument, reject step == 0, and loop while i < stop for positive steps or i > stop for negative steps.

Glossary

Term Meaning Example
Scope Where in the code a name can be used a function's local names
LEGB Lookup order: Local, Enclosing, Global, Built-in print is built-in
Local variable A name assigned inside a function count = 0 in make_counter
Enclosing function The function around an inner function make_counter around next_token
Global variable A name at module (file) level count = 0 at the top
global Assign to a module-level name from a function global count
nonlocal Assign to the enclosing function's name nonlocal count
UnboundLocalError A local name read before it has a value count += 1 without nonlocal
Factory function A function that builds and returns new objects or functions make_counter()
Closure An inner function plus the outer variables it remembers clinic_a
Cell / __closure__ Where a closure keeps a remembered variable f.__closure__[0].cell_contents
Lambda A one-line function with no name lambda x: x * 2
Late binding A closure reads a variable's value when called, not when made [30, 30, 30]
functools.partial Makes a new function with some arguments fixed partial(times, n=2)
Generator function A function containing yield def my_range(stop): ... yield i
Generator The paused-function object it returns g = my_range(3)
yield Hand out one value and pause yield i
next() Run a generator to its next yield next(g)
StopIteration Raised by next() when no values are left after the last value
Iterator Anything next() works on, used up as you go a generator, iter([1, 2])
Lazy Values made only when asked for generators, range
Generator expression A lazy comprehension in round brackets (x * x for x in nums)
sys.getsizeof An object's own size in bytes sys.getsizeof([1, 2])
itertools.islice Take a slice of any iterator islice(fibonacci(), 10)
Sequence Indexable, sized, reusable (list, tuple, range) range(5)[2]

Summary

  • A factory function gives each object its own state; a global counter is shared by everyone.
  • Assigning to a name makes it local. To change an enclosing function's variable, declare nonlocal.
  • A closure is an inner function plus the variables it remembers; they live on after the outer function returns.
  • Closures remember variables, not values (late binding). Lambdas made in a loop all see the final loop value; save it with i=i, partial, or a factory.
  • yield makes a generator: one value at a time, pausing in between. O(1) memory, any size. next() ends with StopIteration.
  • Write my_range with the loop check chosen by the step's sign, reject step == 0, and remember a generator is used up after one pass.
  • Next episode: OOP questions: __eq__ with __hash__, a Stack class with __len__ and __repr__, and the class attribute trap.