Choosing Where to Analyze Logs on AWS: Logs Insights vs Athena vs OpenSearch
The cost of collecting logs was covered in the CloudWatch Logs post. This post is the next question: where do you analyze what you collected? The conclusion up front: logs you only dig into occasionally go to Logs Insights; logs archived long-term in S3 and queried with SQL go to Athena; and only logs that need always-on dashboards, full-text search, or alerting justify OpenSearch. The three are not competitors but layers sorted by query frequency, and indexing everything into OpenSearch is this area’s most common overspend. Prices are for us-east-1.
Side-by-side comparison #
| Aspect | Logs Insights | Athena + S3 | OpenSearch |
|---|---|---|---|
| Data location | CloudWatch Logs | S3 | Dedicated cluster or collection |
| Billing | ~$0.005 per GB scanned | ~$5 per TB scanned | Instance hours or OCUs |
| Idle cost | 0 (pay per query) | 0 (pay per query) | Always-on (domain or OCU floor) |
| Query language | Its own syntax (plus PPL/SQL support) | Standard SQL | Query DSL, dashboards |
| Suited frequency | Occasional (investigation, debugging) | Periodic (reports, audits) | Constant (dashboards, alerting) |
| Latency | Seconds | Seconds to minutes | Milliseconds (once indexed) |
| Setup work | None | Partitioning and format design | Indexing pipeline, capacity design |
Logs Insights: query what is already there, in place #
If application logs already flow into CloudWatch, the first answer for investigation and debugging is Logs Insights: pick log groups and query, no pipeline, at about $0.005 per GB scanned. Incident digging that sweeps a few GB a few times a day closes the month at a few dollars.
The limits are equally clear. Because billing scales with scanned volume, the pricing model collapses the moment wide-time-range queries run constantly. The classic mistake is a dashboard built on Insights queries rescanning all logs every five minutes. Needing constant queries is the signal that you have outgrown this layer — that is when OpenSearch enters the conversation. The query habits (narrow time ranges, specific log groups) are the ones from the collection post.
Athena + S3: long retention meets SQL #
As log lifetimes stretch, the store should be S3 (the conclusion of the collection post), and Athena is the SQL engine over it: about $5 per TB scanned, pure per-query billing, zero idle cost. For logs that land in S3 natively — ALB access logs, CloudTrail, VPC Flow Logs — this pairing is effectively the default. Monthly reports, security audits, and “every request from this IP last quarter” are this layer’s work.
The bill is set by scan volume, and scan volume is set by storage design. Partition by date and service, and store Parquet instead of raw JSON, and scans shrink by orders of magnitude. The same question that scans 200GB of unpartitioned JSON ($1) scans 2GB of partitioned Parquet ($0.01). Put the other way: when Athena starts to hurt, fix the storage format before the queries. The storage tier itself is the S3 storage class comparison’s territory.
OpenSearch: paying a fixed cost for constancy #
What OpenSearch buys is millisecond search, live dashboards, full-text queries, and log-driven alerting. What it costs is being always on. Managed domains bill instance hours, with small production setups starting at hundreds of dollars a month; serverless (about $0.24 per OCU-hour) can carry a minimum-OCU floor at a similar level depending on collection type (newer collection generations can scale to zero when idle — confirm at adoption). Either way, unlike the two pay-per-query layers, this one charges for existing.
So the bar is constancy, not features. An ops team’s all-day log dashboard, a log search screen that must answer in hundreds of milliseconds, real-time pattern alerts — when constant querying is a real requirement, the fixed cost is justified. Indexing all logs “because it might be nice” pays collection and indexing twice.
Composition: the three stack as layers #
The practical setup is not choosing one but routing logs by query frequency. The typical shape: application logs land in CloudWatch and Insights covers investigations; past retention they export to S3 where Athena covers audits and reports; and only the services needing live dashboards get their logs subscription-delivered to OpenSearch. Sized this way, the OpenSearch cluster scales with constantly queried logs rather than all logs, and the three-layer total often lands at a fraction of the index-everything setup.
Decision order #
- Classify logs by query frequency: occasional (investigation), periodic (reports, audits), constant (dashboards, alerts). The classification is the architecture.
- Close the occasional bin with Logs Insights: no pipeline, just the narrow-range habit.
- Send the periodic bin to S3 + Athena: partitioning plus Parquet conversion is the setup, and that design sets the scan bill’s order of magnitude.
- Consider OpenSearch only for the constant bin: approve the fixed cost only when the constancy requirement can be written as a sentence, and index only those logs.
- Revisit the boundaries periodically: growing Insights bills signal constancy (consider OpenSearch); OpenSearch indices no dashboard reads signal demotion to S3.
Summary #
- The three are layers by query frequency, not rivals: occasional to Insights, periodic to Athena, constant only to OpenSearch.
- Insights (~$0.005/GB scanned) is the pipeline-free first answer, but its pricing collapses under constant querying.
- Athena’s bill (~$5/TB scanned) is set by storage design: partitioning plus Parquet shrinks scans by orders of magnitude.
- OpenSearch charges for existing. Index only logs whose constancy requirement can be stated as a sentence.
- Indexing everything into OpenSearch is the area’s biggest overspend; layering typically cuts the total to a fraction.