Modern Python Intermediate #7: Async intro (asyncio)

The last in the intermediate series — async. The tool built on top of the send/yield mechanism we briefly saw in #4 Iterables and generators is asyncio and async/await. This post organizes that first step in one place.

Sync vs async — one picture #

import time

def fetch(url: str) -> str:
    time.sleep(1)             # wait for network response (1s)
    return f"data from {url}"

def main():
    a = fetch("a")            # 1s
    b = fetch("b")            # 1s
    c = fetch("c")            # 1s
    print(a, b, c)            # 3s total

main()

Three requests in order. 1 second each, 3 seconds total.

The same job done asynchronously:

import asyncio

async def fetch(url: str) -> str:
    await asyncio.sleep(1)
    return f"data from {url}"

async def main():
    results = await asyncio.gather(
        fetch("a"),
        fetch("b"),
        fetch("c"),
    )
    print(results)            # 1s total

asyncio.run(main())

The three run concurrently. 1 second total.

That’s the value of async. When waiting — not CPU — is the bottleneck — networks, DBs, files — concurrency buys time.

Core concepts — gathered #

1) Coroutine functions and coroutine objects #

async def fetch(url):    # coroutine function
    return ...

c = fetch("a")            # c is a coroutine object
                          # body has not yet executed!

async def makes a function that returns a coroutine object instead of running the body when called. Same pattern as generator functions.

For a coroutine to actually run, the event loop has to drive it.

2) await — wait for a result #

async def main():
    data = await fetch("a")    # wait until fetch finishes
    print(data)

What await does:

• Yields the expression’s value (coroutine, Task, etc.) to the event loop
• When it finishes, gets the result and continues
• During the yield, other coroutines can run

await can only be used inside async def.

3) The event loop #

A scheduler that alternates between coroutines. It runs only one coroutine at a time, but when one yields with await, another takes its place. The mechanism that creates concurrency on a single thread.

asyncio.run(coro) creates the event loop, runs coro to completion, and cleans up. Typically called once at the program entry point.

Two standards for concurrent execution — gather and TaskGroup #

asyncio.gather — the classic form #

async def main():
    a, b, c = await asyncio.gather(
        fetch("a"),
        fetch("b"),
        fetch("c"),
    )

gather starts multiple coroutines concurrently and gives results as a list when all complete. The classic standard.

TaskGroup (3.11+) — the new standard #

async def main():
    async with asyncio.TaskGroup() as tg:
        a = tg.create_task(fetch("a"))
        b = tg.create_task(fetch("b"))
        c = tg.create_task(fetch("c"))

    print(a.result(), b.result(), c.result())

Same behavior, safer in two ways.

1. If an exception is raised mid-flight, other tasks are cleaned up — gather without options just keeps going
2. When multiple tasks fail concurrently, they’re bundled into Basics #6’s ExceptionGroup

For 3.11+ environments, TaskGroup is the recommended form.

Task — running a coroutine in the background #

When you want to start immediately and run in the background, make a Task.

async def main():
    task = asyncio.create_task(fetch("a"))
    # task is already running here

    other_work()              # do other things

    result = await task        # collect result when needed

A coroutine object alone is just a plain value that doesn’t run until something awaits it. Wrapping it in a Task registers it with the event loop and starts execution.

Common pitfalls — don’t forget #

1) Can’t await inside a sync function #

def main():
    data = await fetch("a")    # ✗ SyntaxError

await only inside async def. The entry point starts async from sync code via asyncio.run(...).

2) time.sleep blocks async #

async def fetch(url):
    time.sleep(1)              # entire event loop stops
    return ...

async def fetch(url):
    await asyncio.sleep(1)
    return ...

time.sleep puts the whole OS thread to sleep. Used inside async code, it breaks concurrency. Sync blocking functions (some DB drivers, requests, parts of file I/O) have the same problem — use async libraries (asyncpg, httpx, aiofiles).

3) Creating a coroutine without awaiting it #

async def main():
    fetch("a")                # created but never awaited — doesn't run
    print("done")

You’ll get a warning (coroutine 'fetch' was never awaited). A coroutine is just a value — without something await-ing or making a Task, it never runs.

4) Don’t call from sync code as if it were sync #

async def fetch(url): ...

result = fetch("a")    # coroutine object. Not the result.

To unwrap, use asyncio.run(...) at the entry point.

Mixing with sync code #

When a sync function takes too long — to_thread #

import asyncio

def heavy_calc(n: int) -> int:
    # CPU-bound or blocking I/O
    ...

async def main():
    result = await asyncio.to_thread(heavy_calc, 100)

asyncio.to_thread(fn, *args) runs a sync function in a separate thread and gives an awaitable result. Often appears when you must call a sync library from async code.

Async context manager — async with #

Briefly seen in #3.

import httpx

async def main():
    async with httpx.AsyncClient() as client:
        resp = await client.get("https://api.example.com")
        data = resp.json()

An object with __aenter__/__aexit__ instead of __enter__/__exit__. You can build them from a function with @asynccontextmanager.

from contextlib import asynccontextmanager

@asynccontextmanager
async def db_transaction(conn):
    tx = await conn.begin()
    try:
        yield tx
        await tx.commit()
    except Exception:
        await tx.rollback()
        raise

Async iterator — async for #

async def stream_lines(url: str):
    async with httpx.AsyncClient() as client:
        async with client.stream("GET", url) as resp:
            async for line in resp.aiter_lines():
                yield line

__aiter__/__anext__ instead of __iter__/__next__ — the pattern of receiving the next value asynchronously. Common in streaming.

Timeouts — asyncio.timeout (3.11+) #

async def main():
    try:
        async with asyncio.timeout(5):
            data = await fetch("https://slow.example.com")
    except TimeoutError:
        print("5초 안에 못 받음")

Before 3.10, asyncio.wait_for(coro, timeout=5) was standard. New code uses the asyncio.timeout context manager.

Common patterns — collected #

Limit concurrency to N — Semaphore #

async def fetch_all(urls: list[str], concurrency: int = 10):
    sem = asyncio.Semaphore(concurrency)

    async def fetch_one(url):
        async with sem:
            return await fetch(url)

    return await asyncio.gather(*[fetch_one(u) for u in urls])

Firing 1000 concurrent requests at once will anger servers or exhaust memory/connections. Semaphore limits to at most N at a time.

Take only the first finisher — as_completed #

async def race(tasks):
    for coro in asyncio.as_completed(tasks):
        result = await coro
        if result.is_winner:
            return result

When you want to handle the first finishers of many concurrent jobs.

Cancellation #

task = asyncio.create_task(fetch("a"))
await asyncio.sleep(0.5)
task.cancel()    # raises CancelledError inside the task

Timeouts, user cancellations, and parent cancellations are all built on this. Async code is best written with cancellation in mind — try/except CancelledError for cleanup is common.

Where does intro end? #

This post is a conceptual introduction. In real practice, more topics come up.

• Internal behavior and policies of the event loop
• The exact difference between Future vs Task
• Choosing and integrating sync/async libraries
• Performance debugging (where await stretches)
• Background jobs, worker patterns
• Integration with web frameworks (FastAPI, Starlette)

These are covered properly in the next series Modern Python Advanced’s async-depth post and the final Modern Python in Practice (FastAPI) series.

Wrap-up #

What this post covered:

• Async is a tool for when waiting, not CPU, is the bottleneck — I/O concurrency
• async def = coroutine function; calling alone doesn’t run it; needs await or Task
• await yields to the event loop; resumes when result arrives
• asyncio.run(coro) — once at the entry point
• Concurrent execution: gather (classic), TaskGroup (3.11+ recommended)
• create_task — start immediately in the background
• Pitfalls: no time.sleep (use asyncio.sleep), coroutines without await, no await in sync functions
• Use asyncio.to_thread for sync functions
• async with, async for — async context/iterator
• asyncio.timeout (3.11+) — context-manager timeout
• Concurrency cap with Semaphore, first finisher with as_completed, termination with cancel()

Looking back at the series #

Across seven posts the Modern Python Intermediate toolbox is filled.

• Data shape — @dataclass, __slots__
• Types — Generic, Protocol, TypedDict, Literal
• Resource management — with, contextlib
• Flow expression — iterables, generators, yield from
• Wrapping functions — every decorator pattern
• Branch expression — match-case in depth
• Concurrency — async intro

The next series Modern Python Advanced goes into the depths you meet in libraries/frameworks — magic methods, descriptors, metaclasses, async depth, GIL/concurrency, advanced typing, performance. On top, the final In Practice (FastAPI) series gathers the tools into a single project.