Hardware Basics #4: Storage ① Devices — HDD / SSD / NVMe and IOPS / Throughput / Latency

In #3 we said the slow place the system spills to when memory runs short is the disk. That disk is the subject here. Database files, logs, and uploaded images all live on storage, and service speed varies widely by its type and performance metrics. We’ll sort out a single disk first; layering and connecting many disks is carried forward in #5.

Where this post sits in the Hardware Basics series:

• #1 Four resources that run a computer — CPU / Memory / Storage / Network
• #2 CPU — Cores / Threads / Clock / Cache, and what a vCPU really is
• #3 Memory — RAM, the hierarchy, and what happens when swapping starts
• #4 Storage ① Devices — HDD / SSD / NVMe and IOPS / Throughput / Latency ← this post
• #5 Storage ② Layout and connection — RAID and DAS / NAS / SAN
• #6 Network — Bandwidth and latency, from the NIC to the data center
• #7 Virtualization and containers — How one physical server becomes many
• #8 Cloud — From owning to renting, from on-prem to IaaS / PaaS / SaaS
• #9 Reading cloud instance specs — Choosing to match the workload

The basics of storage #

Storage differs from #3’s memory in two ways. It’s non-volatile — contents survive a power-off — and it’s far slower than RAM. What it trades in speed it makes up in size and cost. So permanent data lives on storage, and only the data in use right now climbs into memory.

A disk reads and writes data in fixed-size units called blocks. The fact that one file is spread across blocks is the basis for understanding the performance metrics later.

HDD — a spinning magnetic disk #

An HDD (Hard Disk Drive) spins magnetic platters fast and moves a head over them to read and write. The presence of physically moving parts explains everything about its performance.

To read a piece of data, the head moves to that position (seek) and the platter rotates to that point (rotation). Because of this wait, random access — reading scattered data — is especially slow. Sequential access of contiguous data is comparatively fast. It’s cheap and large, so it’s still used for backups and bulk archiving.

SSD — no moving parts #

An SSD (Solid State Drive) stores data electronically in flash memory. With no moving parts there’s no head movement and no rotational wait. So random access is hundreds of times faster than an HDD, with almost no speed difference by location.

It’s the default choice for servers and databases today. It costs more than an HDD, but the speed gap is so large that latency-sensitive workloads effectively assume SSD.

NVMe — putting the SSD on a faster path #

As SSDs got fast, the path connecting them became the bottleneck. Early SSDs used the SATA path from the HDD era, and that path’s limit held the SSD back.

NVMe (Non-Volatile Memory express) connects an SSD directly to a high-speed path close to the CPU (PCIe). The same flash, connected over NVMe, is several times faster than a SATA SSD. The local disks on high-performance instances today are mostly NVMe.

HDD        : spinning platter + head     slow, cheap, large
SATA SSD   : flash + old path            fast
NVMe SSD   : flash + high-speed path     fastest

Three metrics that set storage apart #

At equal capacity, disk performance varies wildly. Three metrics set it apart.

IOPS — input/output operations per second #

IOPS (Input/Output Operations Per Second) is how many read/write requests are served in a second. Workloads that read and write small pieces all over, frequently — a database, for example — are sensitive to it. The more scattered small requests, the more IOPS matters.

Throughput — amount of data per second #

Throughput is how much data you move in a second (MB/s). Workloads that read or write one big file whole — a video file or a backup — are sensitive to it.

Latency — time per single operation #

Latency is the time to serve one request. The lower it is, the more immediate the response. HDD is in milliseconds; SSD and NVMe in microseconds.

The point: capacity and speed are different axes. A 1 TB disk isn’t necessarily faster than a 100 GB one. Speed is set by type and by IOPS / throughput / latency.

In the cloud — you buy capacity and performance separately #

The key thing about cloud disks is that you choose capacity and performance separately. Take AWS’s EBS as an example.

You can see why gp3 and io2 diverge: even within the same SSD category, the guaranteed IOPS level and price differ. When a database hits an IOPS limit, the answer is to raise IOPS or move to io2, not to grow capacity. The instance store — NVMe attached directly to the instance — is very fast but loses its data when the instance stops; that distinction is sorted out in #5.

Common pitfalls #

“The disk is full and slow, so grow the capacity” #

Out of capacity and out of speed are different problems. If IOPS is the limit, increasing capacity leaves it just as slow. With a disk alarm, separate capacity from IOPS first.

“It’s an SSD, so it’s fast no matter what” #

The same SSD differs several-fold by SATA vs NVMe and by guaranteed IOPS. The single word “SSD” doesn’t tell you the speed.

“I chose by the benchmark’s sequential speed” #

A product’s listed sequential read/write is for large files. Workloads with lots of random access, like a database, are governed by IOPS and latency regardless of that number.

“I put important data on the instance store” #

The local disk attached directly to an instance is fast but is temporary — it vanishes when you stop or replace the instance. Permanent data needs EBS or separate storage.

Wrap-up #

What we covered:

• Storage is non-volatile and slower than memory. It reads and writes in blocks.
• HDD is slow at random access because of moving parts, SSD is fast because it’s electronic, and NVMe is faster still by putting the SSD on a high-speed path.
• Performance is set by IOPS (operations per second), throughput (data per second), and latency (time per operation).
• Capacity and speed are different axes. A bigger disk isn’t necessarily faster.
• The cloud buys capacity and performance separately. The difference between gp3 and io2 is the example.

Next — storage layout and connection #

A single disk has limits: capacity, speed, and failure. So operations layer many disks and connect them over a network. #5 Storage ② Layout and connection — RAID and DAS / NAS / SAN covers RAID, which bundles disks for speed and safety, and the three ways to attach disks to a server — DAS, NAS, SAN — and how all of it was repackaged into cloud storage.