CloudWatch Metrics and Alarm Costs: The Next Leak After Logs

The fact that Logs ingestion is the main body of CloudWatch cost was the previous post. Once Logs is under control, the second line item comes into view. The conclusion up front: custom metrics cost about $0.30 each per month, which sounds cheap until you learn that every dimension combination counts as its own metric — one line of code can create thousands — and with alarms, dashboards, and query APIs stacked on top, hundreds of dollars a month can accrue without Logs involved at all. AWS-provided default metrics are free, so the inspection targets are the custom metrics you publish and whatever reads them. Prices are for us-east-1.

The pricing structure #

Custom metrics: dimensions create the count #

The billing unit is not the metric name but name × dimension combination. One request_latency metric with 50 endpoints × 5 status codes as dimensions is 250 metrics — about $75 a month. And the moment a high-cardinality value becomes a dimension, the count explodes. A single metric with 10,000 customer IDs as a dimension is roughly $3,000 a month; most incidents in this line item trace back to one casually added dimension.

Two directions cut it down. First, dimension design principles: keep only the axes actually split in alarms and dashboards, and leave individual identifiers (instance, request, customer) to logs. Second, consider the embedded metric format (EMF): publish structured log events and designate only the values to promote into metrics, so detailed data rides the log bill (queried only when scanned) while just the aggregate axes ride the metric bill. It is the standard route for high-cardinality analysis.

EC2 detailed monitoring (1-minute resolution) is the same family of quiet cost: billed per instance as a metric bundle, it becomes visible money across hundreds of instances. Enable it only for groups that genuinely need 1-minute resolution, such as those driving autoscaling decisions.

Alarms and dashboards: small rates, multiplied #

Standard alarms at about $0.10 a month mean a hundred alarms cost $10. The problem is the pattern: infrastructure-as-code that auto-creates “four alarms per instance” scales alarms with the fleet, and they linger undeleted. High-resolution ($0.30) and composite ($0.50) alarms price differently, so folding many standard alarms into composites is sometimes the cleanup direction. Dashboards are free up to three, then about $3 each — dozens of leftover experiment dashboards are the typical residue.

Inspecting these is list management, not engineering: put “alarms that never fire, dashboards nobody opens” on the standing checklist and prune quarterly.

Query APIs: the bill your tools generate #

The easily missed item is the reading side. GetMetricData is always billed — excluded from the free tier — at about $0.01 per 1,000. Human console use is negligible, but a third-party observability tool (Datadog, Grafana, and the like) polling the entire account’s metrics every few minutes grows this line into hundreds of dollars a month, because metric count multiplies by polling frequency.

With third-party integrations, check two things: filter polling by namespace so only metrics actually used get fetched, and if the tool supports CloudWatch Metric Streams (push via Firehose), switch from polling to streams — stream rates generally undercut polling-style API charges at volume.

Inspection order #

1. Decompose the CloudWatch line on the bill: split usage types into MetricMonitorUsage (custom metrics), CW:GMD-Metrics (GetMetricData), and AlarmMonitorUsage, and the dominant axis shows itself.
2. Hunt dimension explosions in the metric list: a larger-than-expected metric count almost always means a high-cardinality dimension. Demote that axis to logs or EMF.
3. Delete alarms that never fire and dashboards nobody opens: quarterly list pruning suffices.
4. Filter or stream third-party polling: accounts with large GetMetricData lines almost always find the cause here.
5. Narrow detailed monitoring: keep 1-minute resolution only where it drives decisions.

Summary #

• The custom-metric billing unit is name × dimension combination. One high-cardinality dimension can cost thousands a month.
• Individual identifiers belong to logs, not metrics. EMF splits the roles and is the standard route for high-cardinality data.
• Alarms (~$0.10) and dashboards (~$3 beyond three) are a count problem, not a rate problem. Put them on the quarterly pruning list.
• GetMetricData is always billed, and full-account polling by third-party tools is what grows it. Filtering or Metric Streams is the fix.
• AWS default metrics are free, so focus inspection on what you publish (custom, detailed monitoring) and what reads it (APIs, tools).