AI Hardware Configurator

What this article covers

• How to properly size hardware for local AI.
• Which components matter for different use cases.
• How RAM, GPU, CPU, and storage work together.
• Typical configurations for beginners, home servers, and workstations.

Introduction: AI Hardware Configurator

Buying hardware for local AI is one of the most important decisions you’ll make. Too little power prevents models from running at all. Too much power wastes money and electricity. A configurator helps you find the right balance for your specific needs.

If you know what you want to do, you can choose components deliberately. A text chatbot has different requirements than image generation or training your own models. The recommendations below give you a solid starting point.

Why do you need a hardware configurator?

Without planning, you often end up with GPUs or CPUs that don’t fit the job. Many beginners buy an expensive graphics card, only to discover they don’t have enough RAM. Others underestimate power consumption or storage space for models and data. A configurator prevents these costly mistakes.

Hardware Configurator basics

The essential components:

• GPU/VRAM: Critical for fast model execution and image generation.
• RAM/System Memory: Important for the operating system, containers, vector databases, and large datasets.
• CPU: Relevant when models run on the CPU or when many threads work in parallel.
• Storage: SSD or NVMe for fast access to models and documents.
• Power Supply/Cooling: Ensure stable operation under load.

Different use cases call for different priorities.

Who is this configurator for?

• Beginners building their first AI machine.
• Home server operators looking for suitable hardware.
• Teams planning AI workstations.
• Anyone who wants to avoid buying the wrong components.

Key terms for AI hardware

• VRAM: Graphics card memory.
• Bandwidth: Memory bandwidth of the GPU.
• Core Count: Number of processing cores.
• TDP: Maximum heat dissipation and power draw.
• NVMe: Fast storage medium via PCIe.
• ECC-RAM: Memory with error correction, relevant for servers.

Practical examples of AI hardware

Minimal Chatbot PC

• CPU: Modern 6-core CPU
• RAM: 16 GB DDR4/DDR5
• GPU: Not mandatory; run small models on CPU
• Storage: 500 GB NVMe SSD
• Use case: Small models up to 7B parameters in Q4

Home Server with GPU

• CPU: 8-core CPU
• RAM: 32 GB DDR5
• GPU: 12 GB VRAM, such as RTX 3060 12 GB or RTX 4070
• Storage: 1 TB NVMe SSD
• Use case: Chatbots, RAG, occasional image generation

Advanced Workstation

• CPU: 12 to 16 cores
• RAM: 64 GB DDR5
• GPU: 24 GB VRAM, such as RTX 4090 or workstation GPU
• Storage: 2 TB NVMe SSD, additional drive for data
• Use case: Large models, image generation, training, multiple parallel services

Common pitfalls when buying hardware

• Focusing only on GPU: RAM and power supplies are often overlooked.
• Case too small: Workstation GPUs are long and need physical space.
• Underpowered supply: High-end GPUs require sufficient power and proper connectors.
• Slow storage: Models and data benefit greatly from fast NVMe storage.
• No upgrade path: Check RAM slots and PCIe slots early on.

Useful links and tools for the configurator

• BotServ.de VRAM Calculator
• BotServ.de RAM Calculator
• BotServ.de Power Cost Calculator
• BotServ.de AI Hardware

FAQ: AI Hardware Configurator

Do I absolutely need a GPU? No, but it’s recommended. Many models run on the CPU, but much slower.

Is more RAM better than more VRAM? For model inference alone, VRAM is more important. For RAG, multiple containers, and large datasets, RAM is equally critical.

Can I use a used GPU? Yes, with caution. Check utilization history, warranty, and physical condition.

Is 16 GB RAM enough for Ollama? For small Q4 models, yes. For larger models, RAG, and multiple services, 32 GB is better.

What CPU is sufficient for AI? Most workloads run on the GPU. A modern 6- to 8-core CPU is usually fine.

Sources and further reading

• BotServ.de AI Hardware: /ki-hardware/
• Ollama Hardware Requirements: https://ollama.com/
• Puget Systems AI Workstations: https://www.pugetsystems.com/

Summary: AI Hardware Configurator

The AI hardware configurator helps you weigh components based on your use case. Chatbots need minimal resources, while image generation and training demand much more. Your GPU, RAM, CPU, storage, and power supply must work together. Plan carefully, and you’ll save money and get a more stable system.