These are the notes for Python episode 32. Raju's delivery app must ask 10,000 delivery partners "are you online?" every minute. Each answer takes about a second of pure waiting. Last episode's tool, one thread per job, works but gets heavy. In this episode you invent a lighter way: one thread that juggles thousands of waiting jobs, using async, await and the asyncio library. Every program below ran for real (Python 3.13.18). Timings depend on your computer; the patterns are what matter.
The problem: 10,000 threads
import threading, time
def check(partner):
time.sleep(1) # waiting for the partner's phone
start = time.perf_counter()
workers = [threading.Thread(target=check, args=(p,)) for p in range(10_000)]
for t in workers: t.start()
for t in workers: t.join()
print(f"10,000 threads: {time.perf_counter() - start:.1f} s")
Output on the laptop in the video (peak memory measured with resource.getrusage):
memory: 129 MB
10,000 threads: 5.5 s
It works, but every thread is a full worker with its own stack memory, and the operating system has to create and switch between all of them. Ten thousand workers, just to stand and wait.
The event loop: one waiter, many tables
| In a restaurant | In Python |
|---|---|
| one smart waiter who never stands at the kitchen door | the event loop: one thread that runs many tasks |
| an order in progress that can be picked up later | a coroutine: a function that can pause, made by async def |
| "I'll come back when it's ready" | await: pause here and let the loop run something else |
- Asynchronous means "not waiting in line": start something, and do other work until it finishes.
asynciois Python's built-in library for asynchronous code.
async def and await
import asyncio
async def check(partner):
await asyncio.sleep(0.1)
return f"partner {partner}: online"
result = check(7)
print(result)
Output:
<coroutine object check at 0x7fe437425fc0>
<sys>:0: RuntimeWarning: coroutine 'check' was never awaited
- Calling an
async deffunction does not run it. It creates a coroutine object, a paused job. asyncio.sleep()is the waiting version ofsleep: it pauses only this coroutine and lets others run.
Awaiting it at the top level fails:
print(await check(7))
File "main.py", line 7
print(await check(7))
^^^^^^^^^^^^^^
SyntaxError: 'await' outside function
await is only allowed inside an async def. (The interactive python -m asyncio REPL and Jupyter allow top-level await; normal scripts don't.)
asyncio.run(): start the loop
import asyncio, time
async def check(partner):
await asyncio.sleep(1)
return f"partner {partner}: online"
async def main():
start = time.perf_counter()
print(await check(1))
print(await check(2))
print(await check(3))
print(f"took {time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
partner 1: online
partner 2: online
partner 3: online
took 3.0 s
asyncio.run(main())creates the event loop, runsmain()until it finishes, and closes the loop. Call it once, at the bottom of the program.- Awaiting one coroutine after another is still sequential: each
awaitwaits for its result before the next line starts.
asyncio.gather: many waits at once
Real-life picture: the waiter places the chai (3 min), samosa (5 min) and toast (1 min) orders together; the table is served in 5 minutes, not 9.
import asyncio
async def make(item, minutes):
print("start", item)
await asyncio.sleep(minutes / 10)
print("ready", item)
return item
async def main():
orders = await asyncio.gather(make("chai", 3), make("samosa", 5), make("toast", 1))
print("serve:", orders)
asyncio.run(main())
Output:
start chai
start samosa
start toast
ready toast
ready chai
ready samosa
serve: ['chai', 'samosa', 'toast']
asyncio.gather(c1, c2, ...)turns every coroutine into a task (a coroutine scheduled on the loop), runs them together, and returns their results in the order you passed them, not the order they finished.- At every
await, the loop switches to another task that is ready.
Now the real job:
import asyncio, time
async def check(partner):
await asyncio.sleep(1) # waiting, but letting others run
return partner
async def main():
start = time.perf_counter()
results = await asyncio.gather(*(check(p) for p in range(10_000)))
print(len(results), "partners checked")
print(f"one thread: {time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
10000 partners checked
one thread: 1.1 s
(check(p) for p in range(10_000))is a generator expression (episode 24) making 10,000 coroutines; the*unpacks them into separate arguments.- About 1 second, on one thread, using about 34 MB on the same laptop (vs 129 MB for 10,000 threads).
create_task: start now, await later
Real-life picture: put the milk on the stove first (it boils by itself), cut the ginger meanwhile, and come back for the milk when you need it.
import asyncio, time
async def boil_milk():
await asyncio.sleep(1)
return "milk boiled"
async def main():
start = time.perf_counter()
milk = asyncio.create_task(boil_milk()) # starts now
print("milk on the stove, cutting ginger...")
await asyncio.sleep(0.6) # other work meanwhile
print("ginger ready at", round(time.perf_counter() - start, 1), "s")
print(await milk, "at", round(time.perf_counter() - start, 1), "s")
asyncio.run(main())
Output:
milk on the stove, cutting ginger...
ginger ready at 0.6 s
milk boiled at 1.0 s
asyncio.create_task(coro)schedules the coroutine on the event loop immediately and returns a Task; it runs whenever the current coroutine awaits something.await taskcollects its result (waiting only if it isn't finished yet). Total 1.0 s, not 1.6 s.- Keep a reference (a variable) to every task you create; a task nobody refers to can be garbage-collected before it finishes.
Other ways to start tasks:
task = asyncio.create_task(check(1)) # starts running at the next await
...
result = await task
async with asyncio.TaskGroup() as tg: # Python 3.11+: waits for all, cancels the rest if one fails
t1 = tg.create_task(check(1))
t2 = tg.create_task(check(2))
print(t1.result(), t2.result())
The blocking trap
import asyncio, time
async def check(partner):
time.sleep(0.1) # the wrong sleep!
return partner
async def main():
start = time.perf_counter()
await asyncio.gather(*(check(p) for p in range(20)))
print(f"20 checks: {time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
20 checks: 2.0 s
No error, no speed-up. time.sleep is a blocking call: it holds the only thread, so the loop can't switch. There's no await, so there's no pause point.
The rule: inside async def, only await things built for asyncio (asyncio.sleep, aiohttp for web requests, asyncpg for databases, ...). For a blocking function you can't replace, run it in a thread:
data = await asyncio.to_thread(requests.get, url) # runs in a worker thread, the loop keeps going
Semaphores: at most N at once
Real-life picture: 10 tokens on a hook at a busy counter. Take a token to be served; no token, wait.
import asyncio, time
limit = asyncio.Semaphore(10) # the server allows 10 at once
busy, peak = 0, 0
async def check(partner):
global busy, peak
async with limit:
busy += 1
peak = max(peak, busy)
await asyncio.sleep(0.1)
busy -= 1
async def main():
start = time.perf_counter()
await asyncio.gather(*(check(p) for p in range(100)))
print("most at once:", peak)
print(f"100 checks: {time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
most at once: 10
100 checks: 1.0 s
- A semaphore is a counter of permits.
async with limit:takes one on entry and returns it on exit. It's a lock that lets N in at once. - 100 checks at 10 at a time = 10 rounds × 0.1 s.
busy += 1is safe here without a lock: on one thread, a coroutine can only be switched at anawait, and there's none between the read and the write.
Timeouts: asyncio.wait_for
import asyncio
async def check(partner, seconds):
await asyncio.sleep(seconds)
return f"partner {partner}: online"
async def main():
print(await asyncio.wait_for(check(1, 0.2), timeout=1))
try:
print(await asyncio.wait_for(check(2, 5), timeout=1))
except TimeoutError:
print("partner 2: no reply in 1 s, skipped")
asyncio.run(main())
Output:
partner 1: online
partner 2: no reply in 1 s, skipped
wait_for(coro, timeout=seconds) cancels the coroutine and raises TimeoutError when time runs out (since Python 3.11, asyncio.TimeoutError is the same class). Python 3.11+ also has async with asyncio.timeout(1):.
Common mistakes
| Mistake | What you see | Why | Fix |
|---|---|---|---|
result = check(7) |
RuntimeWarning: coroutine 'check' was never awaited |
calling only creates a coroutine | result = await check(7) |
await at the top of a script |
SyntaxError: 'await' outside function |
await only inside async def |
async def main() + asyncio.run(main()) |
await a(); await b(); await c() |
took 3.0 s |
each await waits for the previous | await asyncio.gather(a(), b(), c()) |
time.sleep() / requests.get() in async def |
20 checks: 2.0 s |
blocks the loop's only thread | await asyncio.sleep(), aiohttp, asyncio.to_thread |
calling asyncio.run() inside async code |
RuntimeError: asyncio.run() cannot be called from a running event loop |
a loop is already running | just await the coroutine |
| no timeout on a reply that may never come | the program hangs | nothing stops the wait | asyncio.wait_for(..., timeout=...) |
async, threads or processes?
| Situation | Tool |
|---|---|
| thousands of network calls, with async libraries | async / await |
some waiting, with libraries that block (requests) |
threads (ThreadPoolExecutor) |
| heavy calculation | processes (ProcessPoolExecutor) |
Async only helps if everything inside it awaits. Web frameworks such as FastAPI, and libraries like aiohttp, are built on asyncio.
Mini project: a delivery partner health checker
1,000 partners; each replies in a random 0.1-2 s, partner 13 never replies. At most 50 checks at once, 1 s timeout each.
import asyncio, random, time
limit = asyncio.Semaphore(50)
async def ping(partner):
if partner == 13:
await asyncio.sleep(60) # phone switched off
else:
await asyncio.sleep(random.uniform(0.1, 2))
return partner
async def check(partner):
async with limit:
try:
await asyncio.wait_for(ping(partner), timeout=1)
return "online"
except TimeoutError:
return "offline"
async def main():
start = time.perf_counter()
results = await asyncio.gather(*(check(p) for p in range(1, 1001)))
print("online:", results.count("online"), "| offline:", results.count("offline"))
print("partner 13:", results[12])
print(f"took {time.perf_counter() - start:.1f} s")
random.seed(7)
asyncio.run(main())
Output:
online: 507 | offline: 493
partner 13: offline
took 15.9 s
About half reply within 1 s (uniform 0.1-2 s). 1,000 checks at 50 at a time, each at most 1 s, finish in about 16 s and never get stuck on partner 13.
Practice
1. Make the "never awaited" warning, then fix it.
Solution
import asyncio
async def check(partner):
await asyncio.sleep(0.1)
return f"partner {partner}: online"
async def main():
print(await check(7))
asyncio.run(main())
Output:
partner 7: online
2. Time three awaits one by one, then one gather of the same three.
Solution
import asyncio, time
async def job(n):
await asyncio.sleep(1)
return n
async def main():
start = time.perf_counter()
for n in range(3):
await job(n)
print(f"one by one: {time.perf_counter() - start:.1f} s")
start = time.perf_counter()
await asyncio.gather(job(0), job(1), job(2))
print(f"gather: {time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
one by one: 3.0 s
gather: 1.0 s
3. Run 100 tasks of 0.1 s each, but at most 5 at once. How long should it take?
Solution
import asyncio, time
limit = asyncio.Semaphore(5)
async def job():
async with limit:
await asyncio.sleep(0.1)
async def main():
start = time.perf_counter()
await asyncio.gather(*(job() for _ in range(100)))
print(f"{time.perf_counter() - start:.1f} s")
asyncio.run(main())
Output:
2.0 s
100 ÷ 5 = 20 rounds × 0.1 s = 2.0 s.
4. Build the partner health checker above, then change the timeout to 2 s. What happens to the counts?
Solution
With timeout=2, everyone except partner 13 answers in time (their waits are at most 2 s), so you get online: 999 | offline: 1, and the run takes longer because each slot is held up to 2 s.
Quick quiz
async def job():
await asyncio.sleep(1)
async def main():
await asyncio.gather(job(), job(), job())
asyncio.run(main())
How long does it take? A. about 0 s B. about 1 s C. about 3 s D. it raises an error
Answer
B. gather runs all three on the loop together; their 1-second waits overlap.
Interview questions
- What is a coroutine? A function defined with
async def; calling it returns a coroutine object that runs only when awaited or scheduled as a task, and can pause at eachawait. - What is the event loop? The scheduler in
asynciothat runs tasks on one thread, switching to another ready task whenever the current one awaits. - async vs threads? Async switches only at
await(cooperative), so tasks are tiny and shared state is safer; threads are switched by the OS at any moment (pre-emptive) and cost more memory, but work with blocking libraries. - Does async make CPU-bound code faster? No. It's still one thread; heavy calculation blocks the loop. Use processes.
gathervscreate_task?create_taskschedules one coroutine and returns a Task;gatherschedules many and waits for all, returning results in order.TaskGroup(3.11+) is the structured version.- How do you call blocking code from async code?
await asyncio.to_thread(func, *args)(orloop.run_in_executor). - How do you limit concurrency?
asyncio.Semaphore(n)withasync with.
Glossary
| Term | Meaning |
|---|---|
| asynchronous | start a job and do other work instead of waiting for it |
asyncio |
Python's built-in library for asynchronous code |
| event loop | the scheduler that runs many tasks on one thread |
async def |
defines a coroutine function |
| coroutine | a function that can pause and resume; calling it gives a coroutine object |
await |
pause this coroutine until the result is ready; let others run |
asyncio.run() |
starts the event loop, runs a coroutine to completion, closes the loop |
asyncio.sleep() |
a non-blocking sleep |
| task | a coroutine scheduled to run on the event loop |
asyncio.gather() |
runs many awaitables together and returns their results in order |
asyncio.create_task() |
schedules one coroutine as a task |
TaskGroup |
a block that runs tasks and waits for all of them (3.11+) |
| awaitable | anything you can await: a coroutine, a task, a future |
| blocking call | a call that holds the thread while it waits (time.sleep, requests.get) |
asyncio.to_thread() |
runs a blocking function in a worker thread from async code |
| aiohttp | a third-party async library for HTTP requests |
| semaphore | a counter of permits; at most N holders at once |
async with |
a with block whose enter/exit can await |
asyncio.wait_for() |
awaits with a time limit; raises TimeoutError |
TimeoutError |
the exception raised when a time limit runs out |
| cooperative multitasking | tasks give up control themselves (at await) |
| generator expression | (expr for x in items): produces items one at a time |
| FastAPI | a popular Python web framework built on asyncio |
Summary
- Thousands of threads just to wait are heavy; one event loop can run thousands of coroutines on one thread.
async defmakes a coroutine; calling it doesn't run it.awaitit inside anotherasync def, and start everything withasyncio.run(main()).- Awaiting one by one is sequential;
asyncio.gatherruns the waits together and returns results in order. - Never block the loop: no
time.sleepor blocking libraries inside async code (useasyncio.sleep, async libraries orasyncio.to_thread). - Limit concurrency with
asyncio.Semaphore, and protect against endless waits withasyncio.wait_forandTimeoutError. - Lots of waiting → async; blocking libraries → threads; calculation → processes.
- Next episode: Raju's customer search is slow with no waiting at all. You'll learn Big-O, profiling and fast built-ins.