Modern Python Intermediate #3: Context managers (with, contextlib)

In Basics #6 we briefly noted that with is cleaner than try/finally. This is that full post. We cover every pattern of with, plus user-defined context managers and the tools in contextlib.

with — resources cleaned in one line #

The most basic case — file handling.

f = open("data.txt")
try:
    data = f.read()
finally:
    f.close()

with open("data.txt") as f:
    data = f.read()
# f.close() runs automatically here

with triggers cleanup automatically when leaving the block. Whether the exit is normal, an exception, or a return, cleanup happens.

Resources that fit with #

The common pattern is: “enter some state briefly, then revert on exit”. That’s exactly what context managers are for.

Multiple resources at once #

with open("input.txt") as src, open("output.txt", "w") as dst:
    dst.write(src.read())

From 3.10, you can also wrap in parentheses and split lines.

with (
    open("input.txt") as src,
    open("output.txt", "w") as dst,
    Lock() as lock,
):
    dst.write(src.read())

The shape is like a function with many parameters — easy to read.

Building your own — __enter__ / __exit__ #

Objects that work with with implement two methods.

• __enter__(self) — called on entry; returns the value bound after as
• __exit__(self, exc_type, exc_value, traceback) — called on exit

Simplest example — a temporary chdir #

When you want to change the working directory briefly.

import os

class chdir:
    def __init__(self, path: str):
        self.path = path

    def __enter__(self):
        self.old = os.getcwd()
        os.chdir(self.path)
        return self.path

    def __exit__(self, exc_type, exc_value, tb):
        os.chdir(self.old)

with chdir("/tmp"):
    print(os.getcwd())     # /tmp
print(os.getcwd())          # original directory

__exit__’s return value — whether to absorb exceptions #

__exit__ suppresses exceptions if it returns a truthy value. Returning True without realizing this makes every exception silently disappear, which is hard to debug.

class Bad:
    def __enter__(self): return self
    def __exit__(self, *args):
        return True   # ✗ swallows every exception

In most cases, return nothing or False/None.

To intentionally absorb only specific exceptions, check exc_type.

class IgnoreFileNotFound:
    def __enter__(self): return self
    def __exit__(self, exc_type, exc_value, tb):
        return exc_type is not None and issubclass(exc_type, FileNotFoundError)

with IgnoreFileNotFound():
    with open("missing.txt") as f:
        ...
# passes without exception

This common pattern is already in the standard library (see suppress below).

@contextmanager — keep it short with one function #

Building a class with __enter__/__exit__ every time is tedious. There’s a way to create a context manager from a single function using yield.

from contextlib import contextmanager

@contextmanager
def chdir(path: str):
    old = os.getcwd()
    os.chdir(path)
    try:
        yield path        # the with block runs here
    finally:
        os.chdir(old)

This single function does exactly what the class version above does.

How to read:

• Before yield = what __enter__ does
• The yielded value = what gets bound after as
• After yield = what __exit__ does
• Wrap in try/finally to ensure cleanup on exceptions

To handle exceptions too #

@contextmanager
def transactional(conn):
    tx = conn.begin()
    try:
        yield tx
    except Exception:
        tx.rollback()
        raise
    else:
        tx.commit()

If the with block raises an exception, it is re-raised at the generator’s yield point. You can then catch it with a normal try/except.

Other helpers in contextlib #

suppress — ignore specific exceptions #

from contextlib import suppress
import os

with suppress(FileNotFoundError):
    os.remove("maybe-missing.txt")
# Pass even if the file doesn't exist

The IgnoreFileNotFound class we built above is in the standard library as-is.

closing — wrap an object that only has close() #

from contextlib import closing
from urllib.request import urlopen

with closing(urlopen("https://example.com")) as resp:
    data = resp.read()
# resp.close() automatic

Common in older Python versions when urlopen’s response object lacked __exit__. These days most response objects support with directly.

redirect_stdout — capture stdout #

import io
from contextlib import redirect_stdout

buf = io.StringIO()
with redirect_stdout(buf):
    print("hello")
print("captured:", buf.getvalue())
# captured: hello

Useful for tests, capture, log conversion. redirect_stderr is the same.

nullcontext — conditional with #

from contextlib import nullcontext

def process(path: str | None):
    ctx = open(path) if path else nullcontext()
    with ctx as f:
        if f is None:
            print("no file")
        else:
            print(f.read())

For “open the file if one is provided, otherwise do nothing.” with always needs a context manager; nullcontext is the harmless no-op you put in the empty slot.

ExitStack — managing many resources dynamically #

When the resource count is decided at runtime.

from contextlib import ExitStack

def merge(paths: list[str]) -> str:
    with ExitStack() as stack:
        files = [stack.enter_context(open(p)) for p in paths]
        return "\n".join(f.read() for f in files)
# every file auto-closes

When you can’t write N separate with open(p) as f statements because the list length is dynamic, ExitStack solves it. Every registered resource is cleaned up in reverse order.

ExitStack can also register plain cleanup callbacks — not just __enter__-able resources.

def setup_temp_resources():
    stack = ExitStack()
    tmp_dir = create_tmp_dir()
    stack.callback(remove_dir, tmp_dir)
    # ... register more resources
    return stack

with setup_temp_resources() as stack:
    ...

async context managers — quick preview #

For async resources, use async with.

async with open_async_db() as conn:
    await conn.execute(...)

Methods are __aenter__/__aexit__. There’s also @asynccontextmanager, with the same usage as the sync version. Details in #7 Async intro.

User-defined — class vs @contextmanager #

When in doubt, start with @contextmanager and move to a class if it grows complex.

Common patterns — gathered #

Temporary environment variables #

import os
from contextlib import contextmanager

@contextmanager
def set_env(**kwargs: str):
    old = {}
    for k, v in kwargs.items():
        old[k] = os.environ.get(k)
        os.environ[k] = v
    try:
        yield
    finally:
        for k, v in old.items():
            if v is None:
                del os.environ[k]
            else:
                os.environ[k] = v

Commonly used when testing code that depends on environment variables.

Timing #

import time
from contextlib import contextmanager

@contextmanager
def timer(label: str):
    start = time.perf_counter()
    yield
    elapsed = time.perf_counter() - start
    print(f"{label}: {elapsed:.3f}s")

with timer("query"):
    do_expensive_thing()
# query: 0.123s

Locks #

import threading

lock = threading.Lock()

with lock:
    # critical section
    update_shared_state()

Lock, RLock, and Semaphore all support with. You almost never call acquire() / release() directly.

Wrap-up #

Tools this post covered:

• with — clean resource entry/exit in one line, cleanup guaranteed on exceptions
• Multiple resources — with A, B: or parenthesized multi-line
• Building your own: __enter__ / __exit__; __exit__ typically returns False
• @contextmanager — the standard short form using a function; before/after yield = enter/cleanup
• contextlib helpers: suppress, closing, redirect_stdout, nullcontext, ExitStack
• ExitStack — for dynamic resource counts; can register callbacks too
• Short side effects (chdir, env, timer) are the natural fit for @contextmanager
• Async uses async with + @asynccontextmanager

In the next post (#4 Iterables/generators/yield from) we cover the tools beyond comprehensions — user-defined iterables, generator functions, and yield from. The @contextmanager you saw above is also built on top of generators.