Installing Ollama

Introduction

Ollama is the fastest way to run large language models locally. It combines model management, runtime, and a local API into a single tool. This guide shows you how to install Ollama on Linux, macOS, Windows, and Docker.

Ollama Installation in a Nutshell

On Linux and macOS, Ollama installs directly via a setup script. Windows has an official installer. Docker lets you run Ollama as a container, which is especially useful for server or NAS environments. Once installed, start your first model with ollama run llama3.

Key Terms and Components

Term	Definition
Ollama	Runtime environment for local language models
Model	The actual AI file that Ollama loads and executes
Ollama CLI	Command-line tool for control
Ollama API	Local HTTP interface on port 11434

Practical Relevance

Who is this guide for?

You want to install Ollama and start your first model locally. This guide is aimed at beginners who need a working setup without spending time reading through lengthy documentation.

Installation on Linux

For most Linux distributions, run this command:

curl -fsSL https://ollama.com/install.sh | sh

The script downloads Ollama, sets up a systemd service, and starts it. Check the status afterward:

ollama --version

For GPU support on Nvidia cards, make sure the appropriate drivers and CUDA toolkit are installed. Ollama detects the GPU automatically if everything is configured correctly.

Installation on macOS

Download Ollama from the official website and drag the app into your Applications folder. Alternatively, use Homebrew:

brew install ollama

Apple Silicon is natively supported. Intel Macs work too, but are significantly slower. Once launched, Ollama appears in the menu bar. You can then use it from the terminal or via API.

Installation on Windows

Download the installer from ollama.com and run it. The installer sets up Ollama as a service. After installation, open PowerShell or a terminal window and check:

ollama --version

Windows users without WSL should note that GPU support and certain features depend on your driver version. For advanced scenarios, running Ollama in WSL2 is often worth considering.

Installation with Docker

Docker is ideal for servers, NAS devices, or isolated environments. The official container starts like this:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

For GPU support with Nvidia, add the appropriate runtime parameter to the container. On a Synology or Ugreen NAS, you can use the same image via Docker Compose or the graphical Docker interface.

Testing Ollama

After installation, download your first model:

ollama pull llama3
ollama run llama3

Ollama starts a chat session in the terminal. Exit with /bye.

Configuration

By default, Ollama stores models in ~/.ollama. You can adjust paths, ports, and other settings via environment variables. For more details, see Ollama Configuration.

More AI Info and Topics

• Ollama is available on Linux, macOS, Windows, and Docker.
• After installation, start a model with ollama run modelname.
• GPU support is detected automatically by Ollama when drivers are properly installed.
• Docker works especially well for servers and NAS.

Learn more about the API in Using Ollama API and about model management in Managing Ollama Models.

FAQ - Common Installation Questions

Do I need to restart Linux?

No. The service usually starts automatically after installation. A restart can help if you run into issues.

Does Ollama work without a GPU?

Yes, it runs on the CPU. Performance is significantly lower, but sufficient for testing.

Where are downloaded models stored?

By default in ~/.ollama/models on Linux and macOS, or in your user directory on Windows.

Can I run Ollama on a NAS?

Yes, with Docker or Podman. Make sure you have enough RAM and CPU power, since many NAS devices have limited compute resources.

Tools and Further Reading

The official Ollama documentation and Docker image are the best starting points. For graphical interfaces, Open WebUI is a good option.

Sources

• Ollama Installation Guide
• Ollama Docker Hub
• Docker GPU Documentation