Local AI Software
What This Article Covers
- Software that powers local AI.
- What Ollama is used for.
- Links to software guides.
Introduction
Local AI requires a suitable runtime environment. Ollama is one of the simplest solutions for downloading models, launching them, and accessing them via API. This section contains guides for working with Ollama.
Content
- Ollama Basics - Installation, API, and model management in the Ollama section.
- LM Studio - Graphical interface for local models, no terminal required.
- Open WebUI - ChatGPT-like interface for Ollama with RAG and multi-user support.
- llama.cpp - The C++ inference engine powering Ollama and LM Studio.
- vLLM - High-performance inference for servers and production workloads.
- KoboldCpp - Local roleplay, storytelling, and creative writing.
- AnythingLLM - Desktop RAG app for chatting with documents without coding.
- Jan - Offline AI client for desktop, simple and privacy-focused.
Key Topics
| Topic | Purpose |
|---|---|
| Installation | Set up Ollama across different systems |
| API | Query models via HTTP requests |
| Model Management | Pull, list, and delete models |
FAQ
Is Ollama free?
Yes, Ollama is open source and free. Costs only come from electricity and hardware.
Are there alternatives to Ollama?
Yes, LM Studio, llama.cpp, and vLLM are common alternatives.