Hardware Basics #6: Network — Bandwidth and Latency, from the NIC to the Data Center
In #5 we noted NAS, SAN, and the cloud’s EBS all cross the network. Now we step outside the server and sort out that network. The most common confusion here is treating “fast” as a single thing. Wide bandwidth can still mean slow responses, and not knowing why means spending money in the wrong place. Let’s split it into two axes.
Where this post sits in the Hardware Basics series:
- #1 Four resources that run a computer — CPU / Memory / Storage / Network
- #2 CPU — Cores / Threads / Clock / Cache, and what a vCPU really is
- #3 Memory — RAM, the hierarchy, and what happens when swapping starts
- #4 Storage ① Devices — HDD / SSD / NVMe and IOPS / Throughput / Latency
- #5 Storage ② Layout and connection — RAID and DAS / NAS / SAN
- #6 Network — Bandwidth and latency, from the NIC to the data center ← this post
- #7 Virtualization and containers — How one physical server becomes many
- #8 Cloud — From owning to renting, from on-prem to IaaS / PaaS / SaaS
- #9 Reading cloud instance specs — Choosing to match the workload
Bandwidth and latency are different axes #
Network performance splits two ways. Mix them up and your diagnosis misses.
- Bandwidth — how much you can send at once. The unit is Gbps.
- Latency — how long one piece of data takes to arrive. The unit is ms.
In road terms, bandwidth is the number of lanes and latency is the distance from origin to destination. No matter how many lanes there are, a long distance doesn’t shorten the time one car takes to arrive.
| Situation | Bandwidth | Latency |
|---|---|---|
| Sending one large file | bigger is faster | little effect |
| Exchanging small requests often | little effect | lower is faster |
| Video call | some needed | must be low to feel natural |
A web service’s response speed is generally more sensitive to latency than bandwidth, because loading one page exchanges many small requests.
NIC — the server’s network entrance #
The part that connects a server to the network is the NIC (Network Interface Card). The NIC’s speed sets the ceiling on that server’s bandwidth. A 1 Gbps NIC won’t exceed 1 Gbps no matter how good a line you attach.
Mind the units here. Network speed is in bits, file size is in bytes. One byte is 8 bits, so a 1 Gbps line’s actual transfer rate is about 125 MB/s. “It’s a gigabit line but downloads only hit 100 MB/s” is mostly this unit difference.
1 Gbps (gigabit/s) ÷ 8 ≈ 125 MB/s (megabytes/s)Latency comes from distance #
A large part of latency is physical distance. A signal can’t exceed the speed of light, and inside fiber it’s slower than that. So a signal from Seoul to US East takes a minimum amount of time no matter how good the line.
The time for one round trip is RTT (Round Trip Time) — the round-trip time from sending a request to receiving a response. The region-distance table from AWS Basics #1 is exactly an example of this latency.
| From → to | Rough RTT |
|---|---|
| Within the same data center | under 1 ms |
| Seoul → Tokyo | about 30–50 ms |
| Seoul → Singapore | about 70–100 ms |
| Seoul → US East | about 180–200 ms |
On top of this, each router and switch in between adds a little time. Each intermediate stop is a hop, and more hops pile up more latency.
Distance cuts throughput too #
High latency can slow even a large file transfer. With a structure that sends one batch, waits for an acknowledgement, then sends the next, a long round trip means the line sits idle waiting for the next batch.
Wide bandwidth + high latency
send ──────▶ (wait for ack ...) ──────▶ (wait again ...)
└ during this wait, the wide line sits emptySo sending bulk data to a faraway place doesn’t get faster just by increasing bandwidth. It’s more effective to tune things to send more at once, or to cut the distance in the first place.
Data-center network — same AZ, cross-region, internet #
In the cloud, communication differs in speed and cost depending on what it crosses. The regions and AZs from AWS Basics #1 gain meaning here.
| Segment | Latency | Cost |
|---|---|---|
| Within the same AZ | lowest | usually free |
| Same region, across AZs | low | small charge |
| Cross-region | as large as the distance | charged per GB |
| Out to the internet (egress) | varies by path | charged per GB, usually most expensive |
So servers and databases that talk to each other often are better placed in the same AZ, which helps both latency and cost. At the same time, surviving an AZ outage means spreading across AZs, so a trade-off arises between latency/cost and availability.
CDN — a way to cut the distance #
If latency comes from distance, one answer is to place content close to users in advance. That’s a CDN (Content Delivery Network). It replicates images, video, and static files to footprints around the world, and users fetch from the nearest one.
AWS’s CloudFront is the example. Instead of a Korean user fetching an image from a US server each time, they fetch from a nearby footprint, cutting RTT significantly. The detailed behavior is in AWS Intermediate #7 CloudFront.
Common pitfalls #
“It’s slow, so grow the bandwidth” #
If the traffic pattern exchanges small requests often, growing bandwidth does almost nothing. If the cause is latency, you cut distance (CDN / a closer region) or reduce the number of round trips.
“It’s a gigabit line but only hits 100 MB/s” #
That’s normal. 1 Gbps is about 125 MB/s, and minus overhead, slightly lower in practice. It’s the bit-vs-byte difference.
“Cross-region traffic is surely fast” #
Cross-region carries latency as high as the distance demands, plus a per-GB charge. A design that shuffles bulk data between far-apart regions is slow and expensive.
“I didn’t watch egress cost” #
Traffic leaving the cloud (egress) is usually the most expensive segment. Inbound is generally free but outbound is charged, so sending bulk data outside often runs up a large bill.
Wrap-up #
What we covered:
- Network performance is two different axes: bandwidth (how much at once) and latency (how long to arrive).
- The NIC sets the bandwidth ceiling. Speed is in bits, files in bytes — an 8x difference.
- Latency comes largely from distance. The round trip is RTT, and high latency cuts throughput too.
- Cloud communication grows in latency and cost in the order same AZ → across AZs → cross-region → internet.
- A CDN places content close to users to cut the distance.
Next — virtualization and containers #
We’ve now seen every resource that makes up one server. Yet the cloud sells one physical server as if it were dozens. How is that possible? #7 Virtualization and containers — How one physical server becomes many covers how a hypervisor splits hardware, the difference in how virtual machines and containers share resources, and how it leads into Docker and Kubernetes.