Choose sync vs async communication and the right transport—queues or streams—for coupling, resilience, and scale.
1. Synchronous vs. Asynchronous Communication
A. Synchronous Communication (REST/gRPC)
- Mechanism: Client blocks awaiting immediate response.
- When to Use: Time-critical interactions; simplicity; easier transactional control.
- Trade-offs: High temporal/data coupling; client impacted by server latency/failures.
B. Asynchronous Communication (Queues/Streams)
- Mechanism: Client emits message/event and proceeds; receiver processes later.
- When to Use: Decoupling; resilience; fan-out; long-running workflows.
- Trade-offs: Eventual consistency; harder debugging; requires idempotency and DLQ policies.
2. Queues vs. Streams (Message Broker Selection)
A. Message Queues (e.g., SQS, RabbitMQ)
- Consumption: Messages consumed and removed.
- Characteristics: Scales by consumer count; ordering not guaranteed across consumers; durable until consumed.
- Fan-Out: Limited; use SNS or similar for broadcast.
- Best Use: Task distribution and load leveling (emails, image resize jobs).
B. Data Streaming Platforms (e.g., Kafka, Kinesis)
- Logs/Topics: Immutable ordered log; consumers read from offset.
- Characteristics: Ordering within partition; durable with retention; replay history.
- Fan-Out: Strong; multiple independent consumer groups.
- Best Use: Event sourcing, EDA, pipelines, strict ordering needs.
| Feature | Message Queues | Data Streams |
|---|---|---|
| Ordering | Not guaranteed across consumers. | Guaranteed within a partition. |
| Message Removal | Removed after successful consumption. | Persists; consumers read from offset. |
| Consumer Count | Single logical consumer. | Multiple consumer groups (excellent fan-out). |
| Use Case Focus | Work distribution; short-lived tasks. | Data history; event sourcing; analytics; EDA. |
| Complexity | Lower ops (PaaS friendly). | Higher ops (brokers, partitions, ZooKeeper/Quorum mgmt). |
Decision Matrix: Queues vs Streams
Heatmap score (1–5) across ordering, durability, fan-out, replay, ops complexity.