Why Your Server Is Slow #2: More Memory Didn't Help — Page Cache, Swap, Working Set

Infrastructure Hardware Performance
6 min read

The first prescription anyone writes for a slow server is “add memory.” And yet it’s common to double a machine from 32GB to 64GB and watch response times stay exactly where they were. The money is spent — why didn’t it get faster? Part 1 covered the illusion in CPU utilization; this time it’s the illusion in memory capacity.

The conclusion up front: adding memory helps only when memory is the bottleneck, and the set of situations where memory is the bottleneck is narrower than it looks. The goal of this post is a method for judging whether you’re in that narrow set before paying for the upgrade.

Spare memory is not idle — the page cache #

Start with how Linux accounts for memory. Look at free -h.

free -h 
$ free -h
               total        used        free      shared  buff/cache   available
Mem:            31Gi        12Gi       1.2Gi       0.5Gi        18Gi        18Gi
Swap:          8.0Gi       2.1Gi       5.9Gi

Only 1.2GB of free looks alarming, but most of that 18GB of buff/cache is page cache: files the kernel has read once and keeps in memory so the next read skips the disk — and memory it hands back immediately when a process asks for it. So the real headroom is available (18GB). This accounting is covered in detail in Hardware Intermediate #3.

This structure is the first fork in the road for upgrade impact. Adding memory grows the page cache, and things get faster as disk reads turn into cache hits. Flip that around: if the data you read frequently already fits in the cache, growing the cache doesn’t raise the hit rate. Give a system at 98% hit rate twice the memory and all you can claw back is part of the remaining 2%.

The working set — the one condition where upgrades work #

The concept that decides the verdict is the working set: the amount of data the system actually touches repeatedly. If the total dataset is 500GB but the part touched daily is 10GB, the working set is close to 10GB.

  • Working set > memory: data keeps getting evicted from the cache and re-read from disk. This is the one regime where adding memory is dramatic.
  • Working set < memory: the cache already holds everything that matters. An upgrade produces nothing you can feel.

Whether the working set exceeds memory is measurable. The signal is sustained disk reads. Writes have to reach the disk eventually anyway, but in steady state, reads should be absorbed by the cache. If vmstat’s bi (blocks in) stays high throughout the load, or iostat -x shows steady read IOPS, the cache can’t hold the working set. Conversely, if the system barely reads from disk, memory is not the cause of the slowness — go back to part 1’s diagnostic order and look at other resources.

Swap — movement matters, not existence #

Seeing 2.1GB of swap used in free invites the verdict “we’re swapping, memory is short,” but that too is an illusion. The kernel proactively pushes long-untouched pages to swap when it has idle time. The problem is not pages being in swap; it’s pages moving through it.

vmstat 1 — si/so columns 
$ vmstat 1
procs -----------memory---------- ---swap-- -----io----
 r  b   swpd   free   buff  cache   si   so    bi    bo
 2  1 2201M  1.1G   218M   17.9G  840  1120  2400  3100

If si (swap in) and so (swap out) stay nonzero under load, active pages are shuttling between swap and memory — thrashing. That’s when a memory upgrade is exactly the right prescription. If swpd is large but si/so sit at zero, cold pages are simply parked there, and an upgrade changes nothing.

Turning vm.swappiness down to 0 to banish swap is a popular tweak, but the order matters. If the thrashing comes from a working set that exceeds memory, lowering swappiness just shifts the reclaim pressure onto the page cache — or worse, invites the OOM killer. Swap settings are fine-tuning for after the working-set verdict, not before.

Two common reasons the upgrade never arrives #

Separate from the working-set question, there are two cases where the new memory simply never reaches the process.

Container memory limits — growing host memory does nothing if the container has a memory limit; the ceiling inside stays the same. It’s easy to be fooled because free inside a container shows host-wide numbers. Read the real limit and usage from the container runtime (docker stats, or kubectl top pod on Kubernetes) or from the cgroup files. This distinction is covered in Hardware Intermediate #3.

Ceilings the application sets for itself — the JVM’s -Xmx, database buffer pools (shared_buffers, innodb_buffer_pool_size), language runtime heap settings. While these are fixed, you can bolt any amount of memory into the server and the process keeps living under its old ceiling. After an upgrade, raise these settings to match the new capacity — that’s what actually delivers it.

When OOM fires with memory to spare #

One adjacent symptom worth separating, since it gets mixed into upgrade discussions: if available is comfortable but the OOM killer still killed a process, it’s usually a cgroup (container) limit being exceeded, not the system running out. The OOM message in the kernel log (dmesg) includes the cgroup path — that one line tells you whether it was a system OOM or a container OOM. For a container OOM, the fix is that container’s limit, not more host memory.

The diagnostic order #

When the symptom is “we added memory (or plan to) and it isn’t helping (or won’t),” the order is:

  1. available in free -h — read the true headroom first. It’s available, not free.
  2. si/so in vmstat — sustained swap traffic under load means the working set exceeds memory. This is where upgrades work.
  3. bi in vmstat / read IOPS in iostat — steady disk reads throughout the load mean the cache can’t hold the working set. Also an upgrade candidate.
  4. Neither? Memory is innocent — return to part 1’s diagnostic order: CPU queuing, storage, locks.
  5. Already upgraded? Check the delivery path — container limits and application heap/buffer pool settings must be raised to match.

Summary #

  • Spare memory is already working as page cache. Read headroom from available, not free.
  • There is essentially one condition where an upgrade helps: the working set is bigger than memory. The signals are sustained swap traffic (si/so) and steady disk reads.
  • Judge swap by movement, not by usage. Adjusting swappiness comes after the working-set verdict.
  • Container limits, JVM heaps, and DB buffer pools are separate ceilings — upgrades are not delivered to them automatically.
  • OOM with memory to spare is usually a container limit. The cgroup path in dmesg settles it.

Next is storage: why a machine with an SSD can still be slow, traced through write amplification, fsync, and queue depth.

X