Ollama Behind a Reverse Proxy

What This Article Covers

• Why a reverse proxy is useful for Ollama.
• Configurations for nginx, Traefik, and Caddy.
• TLS and DNS.
• Timeouts and streaming.
• Authentication at the proxy.

Introduction: Ollama Behind a Reverse Proxy

If you want to make Ollama accessible to other devices or the internet, exposing the Ollama port directly is a bad idea. A reverse proxy sits in front and provides TLS, authentication, logging, and simpler management. Popular proxies include nginx, Traefik, and Caddy. With the right configuration, Ollama runs securely and reliably behind the proxy.

This article walks through typical reverse proxy setups for Ollama.

Key Terms

• Reverse Proxy: Forwards client requests to a backend server.
• TLS: Encryption.
• Upstream: Backend server, in this case Ollama.
• Timeout: Time limit for responses.
• Load Balancing: Distribution across multiple instances.
• Middleware: Additional functionality like authentication.
• Stream: Successive delivery of tokens.
• Proxy Buffering: Caching responses.

Why Use a Reverse Proxy?

• TLS encryption.
• Centralized authentication.
• Logging and monitoring.
• Simple DNS names.
• No direct exposure of the Ollama port.
• Load balancing across multiple Ollama instances.

nginx

server {
    listen 443 ssl;
    server_name ollama.example.com;

    ssl_certificate /etc/letsencrypt/live/ollama.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ollama.example.com/privkey.pem;

    location / {
        proxy_pass http://localhost:11434;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_read_timeout 300s;
        proxy_buffering off;
    }
}

Setting proxy_buffering off is essential for streaming.

Caddy

ollama.example.com {
    reverse_proxy localhost:11434
}

Caddy automatically fetches and renews TLS certificates.

Traefik

services:
  ollama:
    image: ollama/ollama
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.ollama.rule=Host(`ollama.example.com`)
      - traefik.http.routers.ollama.tls=true
      - traefik.http.routers.ollama.tls.certresolver=letsencrypt
      - traefik.http.services.ollama.loadbalancer.server.port=11434

networks:
  proxy:
    external: true

Authentication at the Proxy

Traefik BasicAuth

labels:
  - traefik.http.middlewares.ollama-auth.basicauth.users=admin:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/
  - traefik.http.routers.ollama.middlewares=ollama-auth

nginx BasicAuth

htpasswd -c /etc/nginx/.htpasswd admin

location / {
    auth_basic "Ollama";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass http://localhost:11434;
}

Timeouts for Streaming

Long responses require generous timeouts:

proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_connect_timeout 60s;

CORS

If a web frontend accesses the proxy:

add_header Access-Control-Allow-Origin "https://frontend.example.com";
add_header Access-Control-Allow-Headers "authorization, content-type";

Load Balancing

For multiple Ollama instances:

upstream ollama {
    server ollama1:11434;
    server ollama2:11434;
}

server {
    location / {
        proxy_pass http://ollama;
    }
}

Tips

• Always enable TLS.
• Set timeouts high.
• Disable buffering for streaming.
• Set up authentication.
• Configure DNS correctly.
• Check logs regularly.
• Add rate limiting.

Common Pitfalls

• Streaming doesn’t work: proxy_buffering off is missing.
• Timeout errors: proxy_read_timeout is too low.
• No TLS: Unencrypted communication.
• CORS errors: Missing headers.
• Weak authentication: BasicAuth password is too simple.
• Incorrect forwarding: Path not set correctly.

Further Reading and Resources

• BotServ.de Ollama Remote Access
• BotServ.de Docker Reverse Proxy
• BotServ.de Docker TLS Certificates
• BotServ.de Ollama API Troubleshooting

FAQ: Ollama Behind a Reverse Proxy

Do I absolutely need TLS? Yes, if access happens over the internet.

Which proxy is easiest to use? Caddy for automatic TLS, Traefik for Docker integration.

Why does the response cut off? Likely the timeout is too low.

Does streaming work through the proxy? Yes, if buffering is disabled and timeouts are set high.

How do I authenticate users? Best through proxy middleware like BasicAuth or Authelia.

Sources and Further Reading

• nginx reverse proxy: https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/
• Caddy: https://caddyserver.com/docs/quick-starts/reverse-proxy
• Traefik: https://doc.traefik.io/traefik/getting-started/quick-start/

Summary: Ollama Behind a Reverse Proxy

A reverse proxy makes Ollama secure, accessible, and easy to use. nginx, Traefik, and Caddy are established solutions that provide TLS, authentication, and logging. For Ollama, the key points are high timeouts and disabled buffering to ensure streaming works properly. By putting a proxy with TLS and access control in front, you avoid the risk of exposing an unprotected API.