When Your Python Server Is Slow — Finding the Bottleneck in 5 Minutes with py-spy

Python Performance Backend
5 min read

Your Python API server is slow. Even after the API diagnosis post’s segment-splitting, sometimes “other logic” dominates — or you simply don’t know what the process is doing at all. Time to look inside the code. The traditional tool, cProfile, requires wrapping the code and re-running it, with overhead too heavy for production. This is exactly the situation py-spy was built for: a sampling profiler that attaches to a running process, with no code changes and no restart, at negligible overhead.

How it works — spying from outside #

py-spy is not code running inside your process. A separate process reads the target’s memory from outside (Linux’s process_vm_readv) and reconstructs the Python interpreter’s call stacks tens to hundreds of times per second. Being statistical sampling — “where does the time go” — it barely slows the target, unlike cProfile’s record-every-call approach, and it’s written in Rust so its own cost is small. That structure is why it’s production-safe.

Installation is one line; since it reads another process’s memory, Linux typically requires sudo or the SYS_PTRACE capability.

install 
$ uv tool install py-spy   # or: pip install py-spy

dump — “what is it doing right now,” in one shot #

The cheapest, most-used command first: dump prints every thread’s call stack at this instant.

py-spy dump 
$ sudo py-spy dump --pid 4321
Process 4321: gunicorn: worker [api]
Thread 4321 (idle): "MainThread"
    _worker (psycopg_pool/pool.py:128)
    wait (threading.py:320)
Thread 4380 (idle): "ThreadPoolExecutor-0_0"
    acquire (psycopg_pool/pool.py:203)   ← waiting on the connection pool
    ...

A process that looks frozen, a worker that stopped answering — one page reveals it. All threads standing in pool.acquire means connection pool exhaustion (the symptom from the database post); lock.acquire means lock contention; a read on an external API means upstream waiting. This command is the Python-side entrance to the “descend into off-CPU analysis” step from Why Your Server Is Slow #1. For a hung process, take dumps a few seconds apart, two or three times — stacks parked in the same place are the verdict.

top — live, per-function consumption #

top is exactly what it sounds like: the function-level version of Linux top, aggregating samples per second into a live view of which functions (and their callees) eat the time.

py-spy top 
$ sudo py-spy top --pid 4321
Total Samples 3200, GIL: 62%, Active: 71%, Threads: 4

  %Own   %Total  Function (filename)
 24.0%   24.0%   _serialize_row (app/serializers.py)
 11.5%   38.2%   render_items (app/views.py)
  8.1%    8.1%   loads (json/decoder.py)

The GIL and Active header numbers are the Python-specific hints. Low Active means the process mostly waits (I/O, locks). A GIL pinned near 100% with several threads means the threads are fighting over the GIL on CPU-bound work — in which case adding threads won’t help and more worker processes are the answer. The structural conclusion comes straight from the header.

record — flame graphs as evidence #

To turn observation into shareable evidence, record collects samples over a period and writes a flame graph SVG.

py-spy record 
$ sudo py-spy record -o profile.svg --pid 4321 --duration 60
# when waiting (not CPU) is the suspect: include off-CPU time
$ sudo py-spy record -o profile-idle.svg --pid 4321 --duration 60 --idle

Reading flame graphs works as covered in Hardware Advanced #1: width is time share, wide peaks are bottlenecks. The option to know is --idle. Default recording focuses on CPU-using samples, so time spent waiting — on the DB, on locks — stays invisible. With --idle, waiting stacks are included, and the “CPU is idle but it’s slow” server’s missing time appears in the picture. Remember it as: CPU bottleneck → default mode; waiting bottleneck → --idle. Async (asyncio) servers show stacks organized around the event loop, a known limit — interpret alongside --gil and --threads when needed.

The five-minute routine #

Entry points by symptom:

  1. No response at all / hung — take dump two or three times. All threads parked at the same wait (pool, lock, external call) is the answer.
  2. Slow with high CPU — check top functions with top, then take a flame graph with record (default mode). Serialization, parsing, regex — the CPU eaters show as peaks. The GIL number settles the thread-strategy question too.
  3. Slow with low CPU — use record --idle. With off-CPU included, where it waits (DB, external API, locks) becomes visible.
  4. In containers — no py-spy inside the container needed: attach from the host using the container process’s PID, or from a sidecar/ephemeral container granted SYS_PTRACE. On Kubernetes, kubectl debug’s ephemeral containers are the standard route.

For development-stage precision (call counts, exact cumulative times), cProfile remains the right tool — the tool landscape is in Modern Python Advanced #7. py-spy’s niche is “the truth about a running process, right now.”

Summary #

  • py-spy is a sampling profiler that attaches to running Python processes with no changes or restarts — reading memory from outside makes it production-safe.
  • dump reveals a hung process in one page: wherever all threads are parked is the bottleneck.
  • top’s GIL and Active numbers tell you CPU-bound versus waiting, and whether your thread strategy is even right.
  • Flame-graph CPU bottlenecks with default record; catch waiting with --idle.
  • Linux needs SYS_PTRACE; on Kubernetes, kubectl debug ephemeral containers are the standard path.
X