Docker Compose for Local AI

What this article covers

• What Docker Compose is and when to use it.
• Building a compose.yaml file.
• How to define services, networks, and volumes.
• Practical examples with Ollama, Open WebUI, and databases.
• Tips for running local AI stacks.

Introduction: Docker Compose for local AI

Docker Compose lets you describe multiple containers in a single file and start them together. For local AI setups, this is especially useful because several services often work in tandem: Ollama for models, Open WebUI as a frontend, PostgreSQL with pgvector for RAG, maybe a Redis cache or web server too. Instead of launching each container individually, you define everything in compose.yaml and manage the entire stack with one command.

This article walks through composing files and provides concrete examples for AI stacks.

Key terms

• Compose: Docker tool for multi-container applications.
• compose.yaml: Configuration file for the stack.
• Service: A container that is part of the stack.
• Volume: Persistent storage for containers.
• Network: Virtual network connecting containers.
• Profile: Option to start certain services conditionally.
• Override: Additional compose file that extends values.

Basic structure of a compose.yaml

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama-data:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama-data:

Essential keys

• image: Which image to use.
• container_name: Name of the running container.
• volumes: Mounts for persistent data.
• ports: Port mapping from host to container.
• environment: Environment variables.
• depends_on: Control startup order.
• restart: Behavior on crashes.
• networks: Define custom networks.

Networks

Compose creates a network for the stack by default. Containers can then reach each other by service name:

open-webui:
  image: ghcr.io/open-webui/open-webui:main
  environment:
    - OLLAMA_BASE_URL=http://ollama:11434

Alternatively, define your own network:

networks:
  ki-net:
    driver: bridge

Volumes

Volumes ensure data persists after a container restart:

volumes:
  ollama-data:
  pg-data:

Ollama and Open WebUI

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    runtime: nvidia
    volumes:
      - ollama-data:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama-data:

With database for RAG

services:
  postgres:
    image: pgvector/pgvector:pg16
    container_name: postgres
    environment:
      - POSTGRES_USER=rag
      - POSTGRES_PASSWORD=ragpass
      - POSTGRES_DB=ragdb
    volumes:
      - pg-data:/var/lib/postgresql/data
    ports:
      - "127.0.0.1:5432:5432"
    restart: unless-stopped

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama-data:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - DATABASE_URL=postgresql://rag:ragpass@postgres:5432/ragdb
    depends_on:
      - ollama
      - postgres
    restart: unless-stopped

volumes:
  ollama-data:
  pg-data:

Common commands

Command	Purpose
docker compose up -d	Start stack in the background.
docker compose down	Stop and remove stack.
docker compose down -v	Remove including volumes.
docker compose ps	Show status.
docker compose logs -f	Follow logs in real time.
docker compose pull	Fetch latest images.
docker compose build	Rebuild images.
docker compose restart	Restart services.

Environment variables from file

Externalize secrets and configuration in .env:

services:
  open-webui:
    env_file:
      - .env

Profiles

Start services only when needed:

services:
  monitoring:
    image: prom/prometheus
    profiles:
      - monitoring

Launch:

docker compose --profile monitoring up -d

Tips for local AI

• Assign container names for easier access.
• Avoid binding to 0.0.0.0, prefer 127.0.0.1.
• Use volumes for models, databases, and configurations.
• Use depends_on thoughtfully for startup order.
• Store secrets and paths in .env files.
• Use restart: unless-stopped for long-running services.

Common pitfalls

• YAML indentation: Use spaces, not tabs.
• Port in use: Stop another container or service.
• No network: Containers cannot reach each other.
• GPU unavailable: Check runtime: nvidia and deploy.resources.reservations.
• Forgotten volumes: Data is lost after restart.
• Wrong environment variables: Watch for spaces, equals signs, and quotes.

Further reading and resources

• BotServ.de Docker commands
• BotServ.de Docker basics
• BotServ.de Ollama commands
• BotServ.de Ollama security
• BotServ.de Proxmox LXC vs. VM

FAQ: Docker Compose for local AI

What is the difference between Docker and Docker Compose? Docker manages individual containers, Compose orchestrates multiple containers together.

Do I need Compose for Ollama alone? No, a single docker run command is sufficient. Once multiple services are involved, Compose becomes worthwhile.

Where does compose.yaml go? Usually in your project directory. The command docker compose up looks for it in the current working directory.

Are volumes secure? Yes, as long as the volume is not deleted. Regular backups are still important.

Can I use Compose with GPU? Yes, via runtime: nvidia or the newer deploy.resources.reservations.devices.

Sources and further reading

• Docker Compose Docs: https://docs.docker.com/compose/
• Compose Specification: https://compose-spec.io/
• Ollama Docker: https://hub.docker.com/r/ollama/ollama

Summary: Docker Compose for local AI

Docker Compose is the ideal tool for orchestrating local AI stacks composed of multiple services. With compose.yaml, you can launch Ollama, Open WebUI, vector databases, and other services reproducibly. Clean YAML structure, persistent volumes, sensible port bindings, and proper environment variables are key. Using Compose saves you from lengthy docker run commands and gives you a maintainable setup.