Link copied to clipboard!
Group 5: Object & File Storage
Scalable storage systems for unstructured data: S3, EFS, FSx, S3 Glacier, Storage Gateway.
Storage Principle: Match storage tier to access pattern; optimize for durability vs cost vs performance triangle based on data lifecycle.
Services & Roles
S3
Infinite object storage with multiple storage classes & lifecycle.
- 99.999999999% durability
- Multiple storage classes
- Lifecycle policies
EFS
Managed NFS file system for EC2 with POSIX semantics.
- Concurrent access
- Auto scaling
- POSIX compliance
FSx
High-performance file systems (Lustre, Windows, NetApp, OpenZFS).
- High throughput
- Low latency
- Protocol support
S3 Glacier
Long-term archive storage with flexible retrieval options.
- Deep archive
- Cost optimized
- Retrieval tiers
Storage Gateway
Hybrid cloud storage integration for on-premises environments.
- File gateway
- Volume gateway
- Tape gateway
Key Differences
| Dimension | S3 | EFS | FSx | S3 Glacier | Storage Gateway |
|---|---|---|---|---|---|
| Access Pattern | REST API | NFS mount | File system | Archive retrieval | Hybrid protocol |
| Performance | High throughput | General purpose | Ultra high | Retrieval delay | Variable |
| Cost | Per GB + requests | Per GB provisioned | High per GB | Very low per GB | Gateway + storage |
| Use Case | Web scale apps | Shared file access | HPC workloads | Long-term backup | Hybrid integration |
Selection Model
0–10 sliders weight storage requirements and access patterns.
Score_S3 = 0.28*C_webScale + 0.22*C_durability + 0.18*C_lifecycle + 0.14*C_apiAccess + 0.10*C_globalReach + 0.08*(10 - C_fileSystem)
Score_EFS = 0.30*C_fileSystem + 0.24*C_concurrent + 0.18*C_posixCompat + 0.12*C_autoScale + 0.10*C_sharedAccess + 0.06*(10 - C_performance)
Score_FSx = 0.32*C_performance + 0.26*C_hpcWorkload + 0.16*C_lowLatency + 0.12*C_protocol + 0.08*C_throughput + 0.06*(10 - C_costSensitive)
Score_Glacier = 0.34*C_archive + 0.28*C_costSensitive + 0.16*C_longTermRetention + 0.12*C_compliance + 0.06*(10 - C_frequentAccess) + 0.04*(10 - C_performance)
Score_Gateway = 0.30*C_hybridIntegration + 0.24*C_onPremises + 0.18*C_cloudMigration + 0.14*C_protocolBridge + 0.08*C_caching + 0.06*(10 - C_cloudNative)
S3 {{vm.scores.s3|number:2}}
EFS {{vm.scores.efs|number:2}}
FSx {{vm.scores.fsx|number:2}}
S3 Glacier {{vm.scores.glacier|number:2}}
Storage Gateway {{vm.scores.gateway|number:2}}
Primary Emphasis: {{vm.recommended.name}} ({{vm.recommended.score|number:2}})
Heuristics
- High C_webScale + C_durability → S3 for application data storage.
- Strong C_fileSystem + C_concurrent → EFS for shared file access.
- High C_performance + C_hpcWorkload → FSx for high-performance computing.
- C_archive + C_costSensitive → S3 Glacier for long-term retention.
- C_hybridIntegration + C_onPremises → Storage Gateway for hybrid clouds.
Anti-Patterns
- Using S3 for POSIX file operations (use EFS instead).
- Storing frequently accessed data in Glacier (use S3 Standard).
- Using FSx for simple file sharing (EFS is more cost-effective).
- Bypassing Storage Gateway for on-premises integration.
Summary
Choose S3 for web-scale object storage, EFS for shared file systems, FSx for high-performance workloads, Glacier for archival, and Storage Gateway for hybrid integration.