#Infrastructure
380 posts
Choosing a GPU Cloud: Hyperscaler vs Specialist GPU Cloud vs Serverless Inference
A working standard for where to rent GPUs for AI workloads: why H100 hourly rates run 2〜5x apart between hyperscalers (AWS) and specialist GPU clouds (Lambda, RunPod), what you accept when spot drops below half, where serverless inference removes GPU management with per-token billing, and the integration, network, and regulatory requirements that flip the price difference.
The Core of Incident Handling — What Matters Most When Things Break
Making incident response run on procedure instead of heroics: recovery before root cause, separating command, communications, and operations, severity levels and declaration criteria, writing blameless postmortems that still name causes, and the conditions that keep on-call sustainable.
Error Budgets — The Concept and Math of a Failure Allowance
The error budget that falls out of any SLO: converting budget size into time and request counts, why burn-rate alerts beat threshold alerts, the policy that fires when the budget runs dry, and running the speed-versus-stability conflict by rule instead of by meeting.
LLM Quantization Compared: FP16, INT8, INT4 and GGUF, AWQ, GPTQ
A working standard for choosing an LLM quantization method: how precision (FP16, INT8, INT4) halves VRAM, the math that takes a 70B model from 140GB to 35GB, where the three formats (GGUF, AWQ, GPTQ) diverge by use (local, GPU serving, accuracy), where INT4 degrades quality on reasoning and code tasks, and how to match the format to the serving framework.
SLI, SLO, SLA — From Metric to Objective to Contract
Sorting out the easily confused trio in working order: choosing good SLIs (the good-events-ratio method), setting SLO numbers realistically, why an SLA must be looser than the SLO, a feel for the nines, and the common mistakes.
vLLM vs Ollama vs SGLang: Choosing an LLM Serving Framework
A working standard for choosing an LLM serving framework when self-hosting: where local execution (Ollama) and production throughput (vLLM, SGLang) diverge, why PagedAttention and continuous batching open a 3〜4x throughput gap, the 2026 shift with TGI entering maintenance mode, differences in quantization and structured-output support, and a selection order by team size.
LLM API vs Self-Hosting: Doing the Break-Even Math
A working standard for deciding whether to use an LLM API or self-host: the structural difference between per-token and per-GPU-hour billing, how to compute break-even from the tokens one GPU produces in a day, where utilization, batching, and quantization move the crossover, and the factors outside price — data privacy, latency, operational burden — that decide when the math is close.
SRE vs. DevOps — Where They Overlap and Where They Split
What SRE and DevOps each are, where they overlap, and where they diverge: DevOps as culture and philosophy, SRE as its concrete implementation, the tools unique to SRE (SLOs, error budgets, toil caps), how the roles actually split in organizations, and how to read job postings.
Lambda vs Fargate: Choosing Serverless Compute
A working standard for choosing between Lambda and Fargate: the structural difference between per-invocation and per-task billing, the unit-price math that makes Lambda over twice as expensive when running continuously and the utilization threshold that follows, Lambda's hard limits (15 minutes, 10GB, cold starts), the hidden costs where API Gateway and log ingestion outgrow the compute bill, and a selection order driven by workload shape.
Where Kubernetes Gets Slow — API Server, etcd, Scheduler
Diagnosing the situation where workloads are fine but kubectl crawls and rollouts lag — a slow control plane. Excessive LIST/WATCH traffic hammering the API server, etcd fsync latency and database bloat, scheduler delays, and the metrics and fixes for each.
ECS vs EKS: Choosing a Container Orchestrator
A working standard for choosing between ECS and EKS: why the real cost is bigger than the free-vs-$73/month control plane difference, the EKS upgrade cycle and the extended-support fee that multiplies by six when neglected, what Auto Mode takes off your plate and what it leaves, the conditions where the Kubernetes ecosystem is genuinely required, and how team size, portability, and hiring shape the decision.
How to Choose requests and limits — Rules That Hold Up in Operations
Not what requests and limits mean, but what numbers to write: measurement-based request sizing, the CPU limit throttling debate, why memory limits are the default, QoS classes and eviction order, using VPA in recommendation mode, and the two failure modes — over-reservation and unlimited memory.