Reverse Proxy for AI Services: Nginx and Caddy

What this article covers

• How to place a reverse proxy in front of Ollama and other AI services
• The differences between Nginx and Caddy, and when each tool makes sense
• TLS encryption with Let’s Encrypt, both manual and automatic
• Authentication with Basic Auth and API keys to prevent unauthorized access to your models
• Common pitfalls, hardware considerations, and a complete practical example

Introduction: Understanding reverse proxies

When you run AI services like Ollama locally, you quickly face a question: how do you access the service from the outside without leaving the door wide open? Ollama listens on localhost:11434 by default. That’s secure as long as you only work from the same machine. But once you want to access your models from your phone, a laptop in the garden, or another location, you need a solution.

A reverse proxy is exactly that. It sits between the client and your AI service, handles TLS encryption, manages authentication, and forwards requests cleanly. Unlike a forward proxy that works for the client, a reverse proxy acts on behalf of the server. To the client, it looks like they’re talking directly to the proxy.

This article is part of the Secure Operation section and specifically covers Network Security. If you want to start with basic hardening, see the article on Firewall. For Ollama context, the articles on Ollama Network Access and the Ollama API are helpful.

Why do you need a reverse proxy?

Imagine you’re running Ollama on a small home server. You want to access your models from anywhere using a domain like ki.example.de, perhaps to run your own app or give colleagues an endpoint.

Without a reverse proxy, you face multiple problems at once:

1. Ollama speaks plain HTTP. Everything you send and receive travels unencrypted across the network.
2. Anyone who reaches the port can call the API. There’s no authentication.
3. You must bind Ollama to 0.0.0.0 to make it reachable from outside. That significantly expands the attack surface.
4. You have no TLS certificate, so the browser complains, and modern browser features are blocked.

A reverse proxy solves all of this in one step. It terminates TLS, checks authentication, forwards only legitimate requests, and keeps Ollama bound to localhost. You get a clean, public interface while the actual service stays in the background.

Reverse proxy explained briefly

A reverse proxy accepts requests from clients and forwards them to one or more backend servers. Only the proxy is visible to the client. The backend server sees the proxy as the sender, not the original client.

The main responsibilities of a reverse proxy:

• TLS termination: encryption outbound, plain traffic inbound
• Authentication: credential verification before forwarding
• Routing: distribution to multiple backends based on path or domain
• Load balancing: spreading across multiple instances
• Caching and compression: performance optimization
• Header manipulation: for example, CORS or security headers

For AI services, the first three are most critical. You need TLS and authentication almost always, and routing when you offer multiple models or services behind one domain.

Who this article is for

This article is for people running AI services locally or on their own server who want to access them securely from the outside. You should have basic Linux knowledge, ideally on Ubuntu, and know how to navigate your server via SSH. Familiarity with Docker helps but isn’t required, since the examples work without it.

If you already have Ollama running and want to take the next step toward secure accessibility, this is the right place.

Key terms

Term	Definition
Reverse Proxy	Server that accepts requests and forwards them to a backend server
Nginx	Performant, widely-used web server and reverse proxy
Caddy	Modern web server with automatic TLS via Let’s Encrypt
TLS	Transport Layer Security, standard for encrypted connections
SSL	Predecessor of TLS, still commonly used in casual speech
Certificate	Digital certificate that confirms the identity of a domain
Basic Auth	Simple authentication with username and password
Proxy Pass	Nginx directive that forwards requests to a backend server
Upstream	Definition of a backend server in Nginx, can include multiple targets
Let’s Encrypt	Free certificate authority for TLS certificates

Nginx as a reverse proxy for Ollama

Nginx is the classic reverse proxy. It’s performant, well-documented, and available on nearly every system. Configuration happens through text files, which takes some getting used to but is very flexible.

A minimal example that exposes Ollama on localhost:11434 to the outside on port 80:

server {
    listen 80;
    server_name ki.example.de;

    location / {
        proxy_pass http://localhost:11434;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Important for streaming responses from Ollama
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}

The proxy_pass line is the heart of it. It tells Nginx where to send requests. The proxy_set_header lines ensure that Ollama knows where the request originally came from. proxy_buffering off is important for AI services because Ollama often delivers responses as a stream. With buffering enabled, Nginx would collect the entire response before sending it, destroying the streaming effect.

Caddy as a reverse proxy for Ollama

Caddy takes a different approach. The configuration is much shorter, and TLS certificates from Let’s Encrypt are fetched and renewed automatically. For smaller setups and anyone who wants to get started quickly, Caddy is often the better choice.

The equivalent to the Nginx example looks like this in Caddy:

ki.example.de {
    reverse_proxy localhost:11434
}

That’s it. Caddy listens on port 443, automatically obtains a certificate for ki.example.de, and forwards to Ollama. Streaming works out of the box because Caddy passes HTTP/1.1 streaming to the backend.

If you want to set headers explicitly, that’s possible too:

ki.example.de {
    reverse_proxy localhost:11434 {
        header_up Host {host}
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-For {remote_host}
    }
}

The price for this simplicity is that Caddy offers less fine-tuning in very complex setups compared to Nginx. For most AI setups, that’s not an issue.

TLS Certificates

TLS is the standard today, and rightfully so. Without TLS, your API calls and credentials travel across the network in plaintext. With TLS, they’re encrypted and protected from eavesdroppers.

Let’s Encrypt provides free certificates valid for 90 days with automatic renewal. The standard client is certbot.

For Nginx, grab a certificate like this:

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d ki.example.de

Certbot asks for your domain, verifies it via HTTP challenge, automatically updates the Nginx configuration, and sets up a timer for renewal. Your site is then accessible over HTTPS.

With Caddy, this step is unnecessary. Caddy uses Let’s Encrypt automatically once you have a domain in your configuration. Just ensure port 443 is reachable from outside and your domain points to your server.

For internal networks without a public domain, you can set up your own Certificate Authority using mkcert. This makes sense when the service is only accessible on your LAN and you don’t have a public domain.

Adding Authentication

TLS alone isn’t enough. Anyone who knows the endpoint can still call it without credentials. You need authentication.

Basic Auth with Nginx

Basic Auth is the simplest approach. Create a file with usernames and password hashes, then reference it in your Nginx configuration.

sudo apt install apache2-utils
sudo htpasswd -c /etc/nginx/.htpasswd meinuser

Add to your Nginx location block:

location / {
    auth_basic "Geschuetzer Bereich";
    auth_basic_user_file /etc/nginx/.htpasswd;

    proxy_pass http://localhost:11434;
    # ... weitere proxy_set_header Zeilen
}

When you access the endpoint, the browser prompts for username and password. For API calls, send credentials in the header, like Authorization: Basic base64(user:pass).

Basic Auth with Caddy

In Caddy, use the basicauth directive:

ki.example.de {
    basicauth {
        meinuser $2a$14$...
    }
    reverse_proxy localhost:11434
}

Generate the hash with caddy hash-password. Only the hash goes in the configuration, not the password itself.

API Keys

For programmatic access, API keys are often more convenient than Basic Auth. Set a header like X-API-Key and verify it in the proxy.

In Nginx with a map:

map $http_x_api_key $api_key_valid {
    default 0;
    "dein-geheimer-key" 1;
}

server {
    # ...

    location / {
        if ($api_key_valid = 0) {
            return 401;
        }
        proxy_pass http://localhost:11434;
    }
}

In Caddy, use header checks or a small plugin. For more complex scenarios with multiple keys or roles, consider a dedicated tool like Authelia or Authentik, which offers Single Sign-On and fine-grained policies.

Example: Ollama with Nginx and TLS

Here’s a complete, step-by-step example. Assume a fresh Ubuntu server with Ollama already running and listening on localhost:11434.

Step 1: Install Nginx.

sudo apt update
sudo apt install nginx

Step 2: Create a configuration file at /etc/nginx/sites-available/ollama.

server {
    listen 80;
    server_name ki.example.de;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name ki.example.de;

    ssl_certificate /etc/letsencrypt/live/ki.example.de/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ki.example.de/privkey.pem;

    auth_basic "Ollama";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location / {
        proxy_pass http://localhost:11434;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_buffering off;
        proxy_read_timeout 300s;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Step 3: Enable the site and test Nginx.

sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Step 4: Obtain a certificate.

sudo certbot --nginx -d ki.example.de

Step 5: Create a user.

sudo htpasswd -c /etc/nginx/.htpasswd meinuser

Step 6: Check your firewall. Ports 80 and 443 must be open, port 11434 stays closed. See the Firewall article for details.

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

Now Ollama is accessible at https://ki.example.de, encrypted and authenticated. The direct port 11434 remains hidden from the outside.

Common Pitfalls

1. Ollama bound to 0.0.0.0. If you configure Ollama to listen on all interfaces, you bypass the reverse proxy protection. Keep Ollama on localhost and let the proxy bridge to the outside. See Ollama Network Access for more.

2. Streaming broken by buffering. Without proxy_buffering off in Nginx, the proxy accumulates the entire response before sending it. This is undesirable for token streams from AI models. Check this if you don’t see progressive output.

3. Timeouts on long generations. Long prompts can take minutes. Nginx’s default timeout is often too short. Increase proxy_read_timeout to 300 seconds or more.

4. Certificate renewal forgotten. Let’s Encrypt certificates expire after 90 days. With certbot, a timer handles it, but verify it’s running. With Caddy, this happens automatically in the background.

5. HTTP/1.0 as backend protocol. Nginx defaults to HTTP/1.0 with the backend, preventing Keep-Alive. Set proxy_http_version 1.1 and clear the Connection header as shown above.

6. CORS issues with web frontends. If your frontend runs on a different domain, you need CORS headers in the proxy. In Nginx, use add_header Access-Control-Allow-Origin *, in Caddy use header Access-Control-Allow-Origin *.

7. Credentials in plaintext in configuration. With Basic Auth, only the hash is in the file, which is correct. With API keys in the Nginx map, the key is in plaintext. Use file permissions to restrict access.

8. Firewall open for the backend port. If port 11434 is open to the outside, anyone can bypass the proxy. Close it, the proxy communicates internally via localhost.

Hardware, Costs, and Security

A reverse proxy is resource-efficient. Nginx and Caddy run fine on a Raspberry Pi or a small VPS with 512 MB RAM. CPU load is minimal as long as you don’t have very high traffic with TLS termination.

Costs are minimal. Let’s Encrypt is free, Nginx and Caddy are open source. If you don’t have your own server, small VPS instances start at a few euros per month. For pure AI workloads at home, you don’t even need that since the proxy can run on the same device as Ollama.

From a security perspective, a reverse proxy is not a panacea, but it significantly reduces your attack surface. Ollama remains bound to localhost, only the proxy is visible to the outside. TLS protects transmission, authentication protects access. Combined with a solid firewall and regular updates, you have a robust foundation.

Additional measures to consider:

• Rate limiting to make brute-force attacks on Basic Auth harder
• Fail2ban to block repeated failed login attempts
• Separate keys for different users so you can revoke individual access
• Monitoring access logs to detect unusual activity

Further Reading

• Secure Operation overview
• Network Security as the parent category
• Firewall for port hardening
• Ollama Network Access for base configuration
• Ollama API for API details
• Docker Basics if you’re running Ollama in Docker
• Ubuntu as your base system

External resources:

• Nginx Documentation: nginx.org/en/docs/
• Caddy Documentation: caddyserver.com/docs/
• Let’s Encrypt: letsencrypt.org
• Certbot: certbot.eff.org

FAQ

Do I need a reverse proxy if I only use Ollama locally? No. If you’re accessing Ollama exclusively from the same machine, a reverse proxy isn’t necessary. Ollama listens on localhost and isn’t exposed to the outside. A proxy becomes relevant only when you want to access it from other devices or outside your network.

Nginx or Caddy, which should I choose? For quick setups and beginners, Caddy is the better choice because its configuration is minimal and TLS works automatically. For complex setups with multiple domains, load balancing, and fine-tuned performance, Nginx offers more flexibility. Both work well with Ollama.

Can I run multiple AI services behind a single proxy? Yes. You can either use separate domains like ollama.example.de and stable-diffusion.example.de, or paths within one domain, such as example.de/ollama and example.de/sd. The latter requires path rewriting, which both proxies support.

What does Let’s Encrypt cost? Nothing. Let’s Encrypt is free. Certificates are valid for 90 days and renew automatically. You only need a domain pointing to your server.

Is Basic Auth secure enough? Basic Auth over TLS is acceptable for small setups with few users. For better security and multiple users, consider switching to API keys or a gateway tool like Authelia or Authentik. Crucially, Basic Auth must always run over TLS, otherwise credentials are exposed in plaintext.

How do I test whether my reverse proxy works? Open the domain in your browser and verify that you see the authentication prompt. Then test an API call, for example curl -u user:pass https://ki.example.de/api/tags. If you get a JSON response from Ollama, the proxy is working.

What if my provider blocks port 443? Some providers block incoming ports for residential connections. In that case, you can use an alternate port like 8443, but then you’ll need to include the port in the URL, like https://ki.example.de:8443. Alternatively, a tunnel service such as Cloudflare Tunnel can route traffic through an outbound tunnel and requires no incoming ports.

Can I run Ollama in Docker with a reverse proxy? Yes, that’s common practice. You define Ollama and the proxy in the same Docker network and configure the proxy to forward to the container name, for example proxy_pass http://ollama:11434. See the Docker Basics article for more.

Do I need to restart Ollama when I configure the proxy? No. Ollama continues running unchanged. You only restart the proxy, for example with sudo systemctl reload nginx, after you’ve updated the configuration.

How do I renew certificates manually? Run sudo certbot renew to check all certificates and renew any that are expiring soon. Use sudo certbot renew --dry-run to test the process without making changes. With Caddy, renewal is automatic.

Sources

• Nginx HTTP Proxy Module Documentation, nginx.org/en/docs/http/ngx_http_proxy_module.html
• Caddy reverse_proxy Directive, caddyserver.com/docs/caddyfile/directives/reverse_proxy
• Let’s Encrypt Documentation, letsencrypt.org/docs/
• Certbot Documentation, certbot.eff.org/docs/
• Ollama Documentation, github.com/ollama/ollama/blob/main/docs/api.md