Kinesis vs MSK vs SQS: Choosing Between Streams and Queues
Choosing among SQS, SNS, and EventBridge is covered in the messaging three-way comparison. This post is the next fork. The conclusion up front: work that is processed once and done goes to SQS; streams that multiple consumers must re-read go to Kinesis; and MSK only when the Kafka protocol and ecosystem are actual requirements. Picking Kinesis without needing a stream, or MSK without a Kafka requirement, are the two common forms of overengineering in this space. Prices are for us-east-1.
Side-by-side comparison #
| Aspect | SQS | Kinesis Data Streams | MSK |
|---|---|---|---|
| Model | Queue (consumed = deleted) | Stream (re-readable within retention) | Stream (Kafka) |
| Ordering | FIFO queues only | Per shard | Per partition |
| Multiple consumers | Poor fit (needs SNS pairing) | Native | Native |
| Billing | ~$0.40 per million requests | ~$0.015 per shard-hour, or per GB | Broker instance hours + storage |
| Idle cost | 0 | Shard upkeep on provisioned | Always-on cluster |
| Operations | Nearly none | Shard management (eased by on-demand) | Highest (eased by Express and Serverless) |
| Ecosystem | AWS integrations | AWS integrations (Lambda, Firehose) | The whole Kafka ecosystem |
The model split: does data vanish on consumption, or get re-read? #
An SQS message ends when one consumer processes and deletes it. That matches work dispatch, load leveling, and async processing precisely, and as the Step Functions split covered, it is the default building block of backend processing. Billing is also the simplest: about $0.40 per million requests.
Streams are different. Data stays for the retention period (24 hours by default, extendable), and multiple consumers read the same data independently, each at its own position. A clickstream read simultaneously by a live dashboard, an S3 loader, and an anomaly detector; reprocessing from a point in time after an incident — these are the stream’s native territory. Conversely, with a single consumer and no replay requirement, a stream’s abilities are features you pay and operate for without using.
Two questions decide it: do two or more consumers read the same data? Must past data be re-readable? Both no: SQS, done. Either yes: move to the stream family.
Kinesis: the AWS-native stream, in two modes #
Kinesis Data Streams’ strength is AWS integration — Lambda triggers and Firehose delivery to S3 and warehouses attach at configuration level. It bills in two modes.
- Provisioned: about $0.015 per shard-hour. Each shard caps at 1MB/s in and 2MB/s out, so you manage shard count against throughput. Two shards run about $22 a month, the cheapest option for steady, predictable streams.
- On-demand: per-GB billing with no throughput management. The standard mode runs about $0.08 per GB in, and the Advantage mode added in late 2025 cut rates substantially to roughly $0.032 in and $0.016 out (recheck modes and rates at adoption). With big spikes or unknown throughput, starting on-demand and re-evaluating against provisioned once the pattern settles is the safe order.
The trap to know is the product of shards and consumers. Consumers share each shard’s read cap, so consumer-heavy designs need enhanced fan-out (extra cost) or more shards. Three or four consumers are fine; beyond that, run the math at design time.
MSK: only when Kafka is the requirement #
MSK is managed Kafka, and the reason to choose it is not performance but ecosystem and portability: migrating an existing Kafka estate, keeping Kafka Connect, Streams, or Flink tooling, multi-cloud or on-premises portability requirements, or building a streaming platform at thousands-of-partitions scale. Absent those, Kinesis delivers the same stream model with far less operations and is the default.
Within MSK there are branches. Standard brokers are the traditional shape — broker instance hours plus EBS storage, with sizes, counts, and storage managed by you. Express brokers remove storage management and raise per-broker throughput, with much faster scaling and recovery than Standard, and have become the recommended default for new provisioned clusters. MSK Serverless bills about $0.75 per cluster-hour plus partitions and throughput, which sets a floor around $550 a month — a serverless in name whose idle cost is not zero. For small workloads, Kinesis on-demand is structurally cheaper than serverless MSK.
Decision order #
- Check whether a queue suffices: no multi-consumer or replay requirement means SQS. A stream can be adopted later when actually needed.
- Default streams to Kinesis: pipelines that live inside AWS get most of their operations absorbed by the Lambda and Firehose integrations.
- Make the Kafka requirement explicit: existing Kafka migration, ecosystem tooling, portability, or very large scale — write down which applies. If none does, there is no reason to pick MSK.
- Split Kinesis modes on predictability: predictable throughput takes provisioned shards; uncertain takes on-demand first, re-evaluated on measurements.
- If MSK it is, look at Express brokers first: they are the new-cluster default — and if the workload is small, compare the Serverless floor (~$550/month) against Kinesis before committing.
Summary #
- Two questions decide: more than one consumer, and replay. Both no means SQS; either yes means a stream.
- The stream default is Kinesis: AWS integrations absorb the operations, and provisioned shards (~$0.015/hour) versus on-demand covers the scale range.
- MSK is for explicit Kafka requirements — ecosystem, portability, very large scale. Chosen without them, it is buying the most expensive operations available.
- Within MSK, Express brokers are the provisioned default and Serverless carries a ~$550/month floor, below which Kinesis on-demand wins.
- Consumer-count math and on-demand GB rates belong in the design phase, with rates rechecked at adoption time.