Hardware Advanced #5: Datacenter Power — The Real Reason You Can't Rack More Servers

Five slots are still empty in the rack, yet the request to install a new server gets rejected. The stated reason isn’t lack of space — it’s lack of power capacity. In a datacenter, the thing that stops you from racking more servers is usually electricity, not slots. To make sense of that situation, you have to follow the power cable plugged into the server to its other end: the power environment the server lives in.

Posts #1 through #4 stayed inside the server. CPU pipelines, eBPF, memory, ZFS — all of it happens under the case lid. From #5 through #7 we step outside. This post traces the path electricity takes to reach the server, and #6 covers what happens after all of that electricity turns into heat: cooling.

One server’s power path — two PSUs and A/B feeds #

Look at the back of a server and you’ll usually find two power supply units (PSUs). That’s a 1+1 redundant configuration. Either PSU can carry the server’s entire load on its own, while the other stands by ready to take the same load. If one fails, the other picks up the full load with no interruption.

For that redundancy to mean anything, the two PSUs must be connected to separate power paths. That’s why the datacenter brings two independent feeds — an A feed and a B feed — down to the rack. PSU 1 plugs into the A feed, PSU 2 into the B feed. Plug both PSUs into PDUs on the same feed and you’ve protected against a PSU failure, but a feed outage takes the whole server down. It’s a surprisingly common mistake during cabling work.

There’s an operational implication hiding here. In normal operation the A and B feeds each carry half the load, but if one feed dies, the survivor has to absorb 100%. That means each feed must stay below half utilization in normal operation. If you’ve been running one feed at 80%, your redundancy is already broken.

The rack’s limit is kW, not slots — the rack power budget #

A standard rack is 42U — forty-two slots. It looks like you could fit forty-two 1U servers, but in practice you hit the power limit long before that. A traditional datacenter delivers around 5–10 kW per rack. With 1U servers that draw 500 W each, a 6 kW rack tops out at twelve. The remaining thirty slots stay empty.

Open a colocation contract and this structure is right there on the page. The unit of contract isn’t “how many racks” but how many kW per rack. The same physical rack costs differently at 4 kW versus 10 kW. From the datacenter’s perspective, the real cost isn’t floor area — it’s the plant that delivers that power and removes the same amount of heat. Floor space is left over; power is not.

So the first question when evaluating a server install isn’t “how many U are free” but “how many kW are left in that rack’s power budget.” That’s exactly the rejection reason at the top of this post.

PDU — not a power strip, a measurement device #

The device that takes the feed coming into the rack and distributes it to the servers is the PDU (Power Distribution Unit). With its long vertical strip of outlets it looks like a power strip, but a datacenter-grade PDU’s real job is measurement and control.

• Metered — measures current and power in real time, per rack or per outlet, and exports it via SNMP and the like. It’s the device that answers “how many kW is this rack drawing right now?”
• Switched — supports remote on/off per outlet. A hung server can be power-cycled without anyone driving to the datacenter.

Operationally, PDU metering is the eyes of power budget management. If the rack is allocated 6 kW and the PDU reads 5.2 kW, the remaining budget is 0.8 kW — a 500 W server might just barely fit. Without that number, adding up nameplate ratings will throw the budget far off. We’ll come back to this in the pitfalls below.

UPS — the bridge between the outage and the generator #

Utility power can drop, and generators take time to start. The device that fills those seconds to minutes with batteries is the UPS (uninterruptible power supply). The UPS’s job is not to ride out a long outage — it’s the bridge until the generator can take the load. That’s why datacenter UPS batteries are typically sized for 5–15 minutes.

There are two main designs.

Online double-conversion always rectifies incoming power to DC, then inverts it back to AC on the way out. The load only ever sees clean power generated by the UPS, so at the moment of an outage there is no transfer event at all. The conversion costs some efficiency, but because it also filters out voltage sags and noise, datacenters use this design.

The generator and the ATS — who’s responsible a few minutes out #

What happens during the minutes the UPS buys is generator startup. The diesel generators sitting outside the building or on a dedicated floor take roughly 10 seconds to a minute to start and ramp up to rated output.

The switch-over between utility power and the generator is handled by the ATS (Automatic Transfer Switch). When the ATS detects loss of utility power, it sends a start signal to the generator, and once the generator’s output stabilizes it transfers the load over. Transferring back when utility power returns is also the ATS’s job.

Putting it together, the timeline of an outage looks like this: power drops, the UPS takes the load immediately (or with no transfer at all), the ATS starts the generator, tens of seconds later the generator takes over, and from there it runs as long as there is fuel. That’s why datacenter tier ratings include generator fuel runtime and refueling contracts. However big the UPS batteries are, if the generator can’t come up, they’re nothing more than a few minutes’ reprieve.

PUE — the tax on getting electricity to IT #

The metric that comes up most often when talking about datacenter efficiency is PUE (Power Usage Effectiveness). The definition is one line.

PUE = total facility power / IT equipment power

If servers, storage, and network gear draw 1 MW while the facility pulls 1.5 MW from the grid, PUE is 1.5. The 0.5 MW difference goes to non-IT loads — most of it to cooling. UPS conversion losses and lighting are in the numerator too, but the main thing inflating it is the cooling plant. The ideal value is 1.0; the industry average sits around 1.5, and the hyperscalers have pushed down to about 1.1.

For an operator, PUE is a cost metric. In a facility with a PUE of 1.5, if the servers’ electricity costs 100, the actual bill corresponds to 150. Cutting one server’s power draw doesn’t stop there — it cuts cooling power along with it. Why cooling inflates the numerator so much, and how that’s been reduced over time, is the subject of #6.

GPU servers and power density — the era of tens of kW per rack #

GPU servers have completely broken the assumptions racks were designed around in the 5–10 kW era. The 8-GPU training server we saw in Intermediate #8 draws around 10 kW per machine. In a traditional rack, one of those uses up the whole budget. The latest integrated GPU racks can exceed 100 kW per rack — the power of ten ordinary server racks concentrated into one.

This is why every conversation about AI datacenters turns into a conversation about power. Even if you have the money for the GPUs, without the power to feed them and the plant to remove the equivalent heat, there is nowhere to plug them in. Bringing GPU racks into existing datacenter space starts with upgrading the incoming power capacity, and for new construction, securing a substation and transmission access comes before the server order — at the site-selection stage. As power density climbs, air cooling itself runs into its limits, and we’ll pick that up in #6 as well.

Common pitfalls #

• Budgeting from nameplate power — the server nameplate or PSU rating (say, 750 W) is a maximum, and actual draw is often far below it. Summing nameplates overstates the budget and leaves racks half-empty; conversely, budgeting from measured averages alone trips breakers at peak. The right basis is the peak value from PDU measurements.
• Plugging both PSUs into the same feed — PSU redundancy survives, but feed redundancy is gone. It turns up surprisingly often after cable cleanups and equipment moves.
• Loading one feed past half — the premise of an A/B configuration is that either side can carry everything. A feed that runs above 50% in normal operation becomes an overload during a failure.
• Treating the UPS battery as the whole outage plan — the UPS is a bridge to the generator. A UPS without generator start tests and fuel management is a few minutes of insurance, nothing more.

Wrap-up #

The picture we built in this post:

• Server power is made redundant with 1+1 PSUs and A/B feeds, on the premise that each feed stays below 50% load in normal operation.
• A rack’s real limit is its power budget, not its slot count. That’s why colocation contracts are written in kW.
• The PDU is a measurement and control device, and its readings are the basis of power budget management.
• Outage response runs on a timeline where the UPS buys minutes and the ATS hands off to the generator.
• PUE is the ratio of total facility power to IT power, and the main thing inflating it is cooling.
• GPU servers pushed per-rack power into the tens of kW and moved the bottleneck of AI infrastructure to power and cooling.

Next — datacenter cooling and racks #

Every watt a server receives turns into heat once the work is done. A 6 kW rack is also a 6 kW space heater. The next post, “Hardware Advanced #6: Datacenter Cooling and Racks,” covers the machinery that carries that heat away — cold aisles and hot aisles, the limits of air cooling, and the liquid cooling that GPU racks have pulled forward. It’s the fight to shrink the PUE numerator we met in this post.