When HPA Doesn't Behave — 6 Common Configuration Mistakes

Infrastructure Kubernetes Troubleshooting
4 min read

You set up an HPA, but traffic surges and nothing scales — or it scales after the damage is done, or it bounces up and down all day. The concepts (HPA vs. VPA vs. Cluster Autoscaler) are covered in K8s Intermediate #6; this post organizes the configuration mistakes that actually bite in practice, by symptom.

The status command first — most HPA diagnosis starts with this one line.

kubectl describe hpa 
$ kubectl describe hpa api-server
...
Metrics:              resource cpu on pods (as a percentage of request): <unknown> / 70%
Conditions:
  AbleToScale      True
  ScalingActive    False   FailedGetResourceMetric
Events:
  Warning  FailedGetResourceMetric  unable to get metrics for resource cpu

1. It can’t read the metric — <unknown> #

If the current value shows <unknown> as above, the HPA is frozen with nothing to decide on. Two causes cover most cases: metrics-server missing or down (if kubectl top pod fails, it’s this), and the next item — missing requests.

2. No requests, so no percentage to compute #

CPU averageUtilization: 70 means “70% of requests.” The denominator is requests, so with no CPU requests on a container, the HPA cannot compute a percentage. A manifest that sets limits but forgets requests is the classic cause.

  • Check: every container in the target Deployment (sidecars included) has CPU requests. One missing sidecar breaks the math.
  • Fix: set requests. How to choose the values is the next post’s topic.

For the same reason, the target percentage’s meaning is also request-relative. With requests of 500m and a 70% target, scaling starts at 350m. If you assumed it meant limits (say, 2 cores), you get the opposite confusion: “why is it scaling already?”

3. Pinned at the min/max boundary #

If the symptom is “it won’t scale up,” eliminate the most mundane cause first: it’s already at maxReplicas. If kubectl get hpa shows REPLICAS equal to MAXPODS, this is capacity planning, not configuration. The mirror image: a high minReplicas prevents scale-down, which gets misread as “scale-down is broken.”

4. It reacts too slowly — surges and scale-up lag #

An HPA doesn’t react instantly. Metric collection intervals, the HPA evaluation loop (15 seconds by default), and new-Pod startup time (image pull + app start + readiness pass) add up: from surge to actual capacity takes minutes. Second-scale spikes can’t be stopped by an HPA.

  • Fix: for slow-starting apps, make scale-up aggressive (several Pods at once via the scaleUp behavior policy) and hold peacetime headroom in minReplicas. Pre-scale for predictable peaks (scheduled events). Fundamentally, shrinking Pod startup time itself — image size, readiness design (K8s Intermediate #5) — is the real substance of scaling responsiveness.

5. Flapping — up, down, up, down #

Add replicas and per-Pod load drops, pulling the metric below target; scale down and it pops back above — oscillation. HPA has a stabilization window against this (scale-down: 5 minutes by default), but with a target set too tight (say 90%) or a naturally noisy metric, the default isn’t enough.

  • Fix: raise behavior.scaleDown.stabilizationWindowSeconds and rate-limit downscaling by policy (say 10% per minute). Set the target with more margin than the metric’s natural swing.

6. Fighting other controllers #

If the HPA scales up and things revert minutes later, something else is putting replicas back. The classic setup: replicas hardcoded in a manifest that GitOps (ArgoCD and friends) keeps syncing — HPA raises to 5, GitOps sees drift and restores 2. Running VPA in auto mode on the same metric (CPU) interferes similarly.

  • Fix: remove the replicas field from manifests of HPA-managed workloads, or have the GitOps tool ignore that field. Keep VPA in recommendation mode or separate the metrics.

One more thing — the limits of CPU-only scaling #

Even with everything configured right, an HPA idles if CPU isn’t the bottleneck. Memory-bound workloads, and workloads whose real load metric is connections or queue length (webhook consumers, job queues), get slow long before CPU reaches 70%. The right tools there are custom-metric scaling (requests per second, queue depth) or an event-driven autoscaler (KEDA). Figuring out what the bottleneck actually is uses the diagnostic order from the Why Your Server Is Slow series as-is.

Summary #

  • Diagnosis starts at kubectl describe hpa. On <unknown>, check metrics-server and requests first.
  • The target percentage is request-relative. No requests, no HPA — and one missing sidecar breaks it.
  • HPA reaction is measured in minutes. Cover second-scale spikes with minReplicas headroom and faster Pod startup.
  • Fix flapping with stabilization windows and scale-down policies; fix GitOps reverts by removing/ignoring the replicas field.
  • If CPU isn’t the bottleneck, change the scaling signal itself: custom metrics or KEDA.
X