Buying Storage for Local AI
What this article covers
- The role storage plays in AI workloads.
- Differences between SATA, NVMe, and HDD drives.
- Storage requirements for models and datasets.
- TLC vs. QLC, DRAM cache, and drive longevity.
- Recommendations for different use cases.
Introduction: Buying storage for local AI
AI models can grow surprisingly large. A single 70B model at full precision easily exceeds 100 GB. Even quantized models, embeddings, and vector databases add up quickly. If you regularly load models, fast NVMe SSDs are worth the investment. If you maintain multiple models or backups, capacity becomes equally critical. Storage deserves more attention in local AI setups than most people give it.
This article walks you through choosing the right storage solution for AI applications.
Key terms
- SSD: Solid State Drive, fast flash storage.
- NVMe: Protocol for SSDs over PCIe.
- SATA: Older, slower interface.
- TLC: Triple-Level Cell, three bits per cell.
- QLC: Quad-Level Cell, four bits per cell, cheaper but slower.
- DRAM cache: Buffer for write operations.
- TBW: Terabytes Written, the lifespan metric for SSDs.
- IOPS: Input/Output Operations per second.
How much storage does an AI PC need?
| Content | Estimated size |
|---|---|
| Operating system and applications | 50 to 100 GB |
| Small models (7B, Q4) | 4 to 8 GB per model |
| Medium models (13B, Q4) | 8 to 15 GB per model |
| Large models (70B, Q4) | 40 to 50 GB per model |
| Embedding models | 1 to 5 GB |
| Vector database and data | Variable, 10 to 100+ GB |
| Backups and snapshots | Depends on strategy |
Plan for at least 1 to 2 TB in a typical AI PC. If you collect many models, you’ll quickly need 4 TB or more.
SATA vs. NVMe
| Type | Speed | Best for |
|---|---|---|
| SATA SSD | ~550 MB/s | Entry-level, data archival |
| NVMe PCIe 3.0 | up to 3,500 MB/s | Solid all-around choice |
| NVMe PCIe 4.0 | up to 7,000 MB/s | Fast loading of large models |
| NVMe PCIe 5.0 | up to 14,000 MB/s | High-end, expensive |
| HDD | 100 to 200 MB/s | Archive only |
For frequent loading of large models, a PCIe 3.0 or 4.0 NVMe SSD pays for itself.
TLC vs. QLC
- TLC: Three bits per cell, strikes a good balance between speed, price, and longevity.
- QLC: Four bits per cell, cheaper per gigabyte, but slower on writes and less durable.
TLC is better if your models are frequently written or modified. QLC works fine for read-heavy workloads.
DRAM cache
SSDs with onboard DRAM cache perform faster on random access and last longer. Without it, write speed drops noticeably once the SLC cache fills up. For AI workloads involving many small files or frequent writes, DRAM cache is worth having.
M.2 or 2.5 inch?
- M.2: Compact, usually NVMe, plugs directly into the motherboard.
- 2.5 inch: SATA SSDs or smaller NVMe drives with adapters.
- PCIe card: Additional SSDs via a PCIe slot.
M.2 NVMe is the modern and preferred standard.
Organizing models and datasets
- Store models on a fast NVMe drive.
- Keep large archives on a second SSD or HDD.
- Back up to external storage or NAS.
- Place Docker volumes on fast storage.
- Vector databases benefit from fast SSDs.
SSD lifespan
TBW indicates how many terabytes can be written before the SSD becomes unreliable. For AI workloads with frequent downloads and deletions of large models, prioritize high TBW ratings. Enterprise SSDs or consumer drives with high TBW ratings handle continuous operation better.
Recommendations
Entry-level
- 1 TB NVMe PCIe 3.0.
- TLC.
- DRAM cache.
Mid-range
- 2 TB NVMe PCIe 4.0.
- TLC.
- High TBW.
High-end / Workstation
- 4 TB NVMe PCIe 4.0 or 5.0.
- Enterprise SSD or high-end consumer model.
- Additional HDD or NAS for archives.
Common purchasing mistakes
- Insufficient capacity: Models and datasets don’t fit.
- QLC for heavy write workloads: Slow and less durable.
- No DRAM cache: Speed plummets during large write operations.
- SATA instead of NVMe: Noticeably longer load times.
- No secondary drive for backups: Risk of data loss.
- Too little headroom: SSDs slow down when nearly full.
Further reading and resources
- BotServ.de Buying RAM for local AI
- BotServ.de Building an AI PC
- BotServ.de Buying a motherboard for local AI
- BotServ.de Docker Backup
FAQ: Storage for local AI
How large should my SSD be? At minimum 1 TB, preferably 2 TB to fit multiple models.
Do I need NVMe? Recommended. Large models load significantly faster than from SATA.
Is QLC a bad choice? Not bad, but slower and less durable if you do heavy writes.
How long do SSDs last? Usually many years, provided you don’t run them constantly at full capacity.
Should I store models on a second drive? A second drive or NAS makes sense for archival and backup.
Sources and further reading
- SSD specs guide: https://www.techpowerup.com/ssd-specs/
- NVMe basics: https://nvmexpress.org/
- CrystalDiskMark: https://crystalmark.info/
Summary: Buying storage for local AI
A fast NVMe SSD with adequate capacity is essential for local AI. Models, embeddings, and databases grow rapidly, so plan for at least 1 to 2 TB upfront. TLC SSDs with DRAM cache and solid TBW ratings offer the best combination of speed and durability. A second drive or NAS for archives and backups is a worthwhile investment. Spending wisely on storage saves time during model loading and spares you an upgrade later.