SRE in Practice #2 Operating the Error Budget: The Policy Document and Burn Rates

Error Budgets covered what the budget is and how to compute it. This part is about making the budget function as an organizational decision-making tool. A budget is not a number on a dashboard; it only has power when attached to a prior agreement of the form “when this number does that, we do this.”

The error budget policy: a document agreed before the incident #

The error budget policy pre-defines actions for each budget state. The critical thing is timing: agree while budget remains, before anyone has a stake in the outcome. Debate “so are we actually freezing deploys?” after the budget is gone, and the debate is always won by whoever has more power.

The skeleton of the policy is a staged action table. For example:

The numbers and actions can differ per organization. Three things matter:

1. Actions must be concrete. “Exercise caution” is not a policy. “Halt all deploys except emergency fixes and reliability improvements” is.
2. It needs executive signatures. A feature freeze halts the product organization’s roadmap. As an engineering-internal document it will not hold; signed in advance by product leadership, it holds when the moment comes.
3. Put the exception path in the document too. Define who approves a deploy during a freeze, and exceptions get handled inside the policy instead of dissolving it.

Burn rate: the speed at which budget disappears #

Watching only remaining budget has a hole. “80% remaining” cannot distinguish “the past three weeks were calm” from “an incident that started ten minutes ago is burning 10% per hour.” So you watch speed.

Burn rate is the multiple of the budget-consumption speed, where 1x = the speed that exhausts the budget exactly at the end of the window.

• SLO 99.9% with a 28-day window means an allowed error rate of 0.1%.
• An actual error rate of 0.1% is burn rate 1x: budget gone exactly at day 28.
• Error rate 1% is burn rate 10x: budget gone in 2.8 days.
• Error rate 0.05% is 0.5x: the window ends with half the budget unspent.

The arithmetic is simple: burn rate = actual error rate / (1 - SLO). This one number is what lets you decide “does this problem justify waking someone up right now?”

Fast burn and slow burn are different problems #

Through the burn-rate lens, budget threats come in two kinds, with different responses.

• Fast burn: a bad deploy or infrastructure failure torching budget in a short time. Humans must intervene within hours — this is a page.
• Slow burn: an error rate hovering just above the SLO for days on end. Not tonight’s problem, but left alone it empties the budget late in the window — this is a ticket for business hours.

Carrying this split directly into the alerting system gives you multiwindow burn-rate alerts, the centerpiece of the next part. Here, only the principle matters: the same “budget threat” routes to page or ticket depending on speed.

Wiring the budget into release decisions #

With policy and burn rate in place, the budget becomes an input to the deploy pipeline.

• Deploy gates: when remaining budget drops below a threshold, the pipeline automatically holds non-urgent deploys. No human has to remember the policy.
• Combined with progressive delivery: if the burn rate spikes during the canary stage, roll back automatically. The budget becomes the verdict on “is this deploy safe.”
• Risk budgeting: allocate in advance — “this migration may spend up to 20% of the budget” — and risky work gets executed deliberately instead of postponed indefinitely.

This is the real value of error budgets: the budget dissolves the fight between velocity and reliability. While budget remains, you have the right to ship fast; when it is spent, you have the duty to slow down. Both sides are justified by numbers instead of emotions.

Common failure patterns #

• A budget without a policy: the dashboard shows a budget, it hits zero, and nothing happens. After the second repetition, nobody looks at the budget again.
• The hollow freeze: a freeze is declared, then “but this one is important” exceptions queue up. Put the exception-approval path in the policy and record the count of exceptions itself.
• The SLO is wrong: a chronically exhausted budget may signal not a careless team but an SLO set above what the system can currently deliver. The correct response is lowering the SLO or increasing reliability investment — not ignoring the alerts.

Summary #

• The error budget policy fixes actions per budget state before the incident. Actions are concrete, executive-signed, with an exception path included.
• Burn rate is the consumption-speed multiple: actual error rate / (1 - SLO), with 1x defined as exact exhaustion at window’s end.
• Fast burns page; slow burns ticket. Speed decides the response to the same threat.
• The budget earns its keep wired into deploy gates, canary rollbacks, and risk allocations.
• A chronically exhausted budget indicts the SLO itself. Next part: implementing burn rates as alerts.