Link copied to clipboard!
Group 8: Analytics & Big Data
Data warehousing, processing, and analytics services: Redshift, EMR, Kinesis, Athena, QuickSight.
Analytics Principle: Match processing paradigm (batch vs stream vs interactive) to data velocity, volume, and query complexity requirements.
Services & Roles
Redshift
Petabyte-scale data warehouse with columnar storage & MPP architecture.
- Columnar storage
- MPP queries
- Redshift Spectrum
EMR
Managed big data framework (Hadoop, Spark, Presto) with elastic clusters.
- Hadoop ecosystem
- Spot instances
- Auto scaling
Kinesis
Real-time data streaming platform for ingestion and processing.
- Data Streams
- Data Firehose
- Data Analytics
Athena
Serverless interactive query service using standard SQL on S3 data.
- Serverless queries
- Standard SQL
- Pay per query
QuickSight
Business intelligence service with ML-powered insights and dashboards.
- Auto-generated insights
- SPICE engine
- Embedded analytics
Key Differences
| Dimension | Redshift | EMR | Kinesis | Athena | QuickSight |
|---|---|---|---|---|---|
| Processing Type | OLAP/Data warehouse | Batch processing | Stream processing | Interactive queries | Business intelligence |
| Data Volume | Petabyte scale | Unlimited | Real-time streams | S3 datasets | Moderate |
| Latency | Seconds to minutes | Minutes to hours | Milliseconds | Seconds | Interactive |
| Cost Model | Cluster hours | Instance hours | Shard hours + PUT | Data scanned | User sessions |
| Use Case | Data warehousing | ETL/ML training | Real-time analytics | Ad-hoc queries | Dashboards/reports |
Selection Model
0–10 sliders weight analytics requirements, data characteristics, and processing preferences.
Score_Redshift = 0.30*C_dataWarehouse + 0.24*C_olapQueries + 0.18*C_petabyteScale + 0.14*C_structuredData + 0.08*C_complexAnalytics + 0.06*(10 - C_realTimeRequirements)
Score_EMR = 0.28*C_bigDataProcessing + 0.24*C_batchProcessing + 0.18*C_hadoopEcosystem + 0.14*C_mlTraining + 0.10*C_etlWorkloads + 0.06*C_costOptimization
Score_Kinesis = 0.32*C_realTimeRequirements + 0.26*C_streamProcessing + 0.18*C_dataIngestion + 0.12*C_eventDriven + 0.08*C_lowLatency + 0.04*(10 - C_batchProcessing)
Score_Athena = 0.30*C_adhocQueries + 0.24*C_serverlessPreference + 0.18*C_s3DataLake + 0.14*C_standardSQL + 0.08*C_costOptimization + 0.06*(10 - C_complexETL)
Score_QuickSight= 0.32*C_businessIntelligence + 0.26*C_dashboards + 0.18*C_mlInsights + 0.12*C_embeddedAnalytics + 0.08*C_userFriendly + 0.04*C_visualizations
Redshift {{vm.scores.redshift|number:2}}
EMR {{vm.scores.emr|number:2}}
Kinesis {{vm.scores.kinesis|number:2}}
Athena {{vm.scores.athena|number:2}}
QuickSight {{vm.scores.quicksight|number:2}}
Primary Emphasis: {{vm.recommended.name}} ({{vm.recommended.score|number:2}})
Heuristics
- High C_dataWarehouse + C_olapQueries → Redshift for traditional BI.
- Strong C_bigDataProcessing + C_batchProcessing → EMR for Hadoop workloads.
- High C_realTimeRequirements + C_streamProcessing → Kinesis for real-time.
- C_adhocQueries + C_serverlessPreference → Athena for S3 analysis.
- C_businessIntelligence + C_dashboards → QuickSight for visualization.
Anti-Patterns
- Using Redshift for real-time analytics (high latency).
- Using EMR for simple ETL jobs (Glue is more appropriate).
- Using Kinesis for batch processing (EMR is better).
- Using Athena for complex ETL workflows (use EMR/Glue).
Summary
Choose Redshift for data warehousing, EMR for big data processing, Kinesis for real-time streams, Athena for ad-hoc queries, QuickSight for BI.