Firewall for AI Services: Securing Ports

What this article covers

How to set up and secure a firewall for local AI services like Ollama
The difference between ufw and iptables, and when to use each tool
Step-by-step guide to securing port 11434 and other service ports
Best practices for default-deny policies, VPN access, and minimal attack surface
Common pitfalls that sabotage firewall rules, and how to avoid them

Introduction: Understanding Firewalls

When you run local AI services like Ollama on your server, you’ll eventually have a service listening on a port that other devices on your network can reach. That’s often convenient because you can query models from other machines. But it also introduces risk if the service doesn’t require authentication and anyone on the network can send requests.

A firewall is your first line of defense. It decides which network traffic can even reach your services. In secure operations and specifically in network security, firewall configuration is foundational knowledge you should have before running AI services in production.

This article shows you how to build a firewall with ufw and iptables, which rules make sense for Ollama, and how to spot common mistakes.

Why do you need a firewall?

Here’s a concrete example: Ollama listens on port 11434 by default. If you set the environment variable OLLAMA_HOST to 0.0.0.0, making the service available across your network, Ollama accepts requests from any device that can reach your server. No authentication. No TLS. No rate-limiting.

In a home network with three devices, this might seem acceptable. In an office, coworking space, or public WiFi, it becomes a problem. Anyone who knows or guesses your server’s IP address can load models, run prompts, and consume compute time. In the worst case, someone uses your server to generate inappropriate content under your IP address.

A firewall restricts who can even establish a connection to port 11434. Instead of trusting everyone on your network, you allow only specific IP addresses or VPN access. All other requests are dropped before they reach Ollama.

Firewall basics

A firewall is software that filters network traffic based on rules. Each rule describes what traffic is allowed or blocked, based on criteria like source IP, destination port, protocol (TCP or UDP), and network interface.

On Linux systems, the firewall operates in the kernel via the Netfilter framework. The tools ufw and iptables are frontends you use to define rules without directly communicating with the kernel.

ufw (Uncomplicated Firewall) is the recommended frontend on Ubuntu. It simplifies the syntax and works well for most use cases.
iptables is the classic, more powerful tool. It offers fine-grained control over each stage of packet processing, but requires deeper knowledge of internal tables and chains.

Both tools ultimately write to the same Netfilter tables. What you configure with ufw is translated internally into iptables rules.

Who is this article for?

This article is for you if you run local AI services like Ollama on a Linux server and want to ensure only authorized devices can access them. You should have basic command-line skills and know how to SSH into your server. Firewall experience is helpful but not required, since each step is explained.

Key terms

Term	Definition
Firewall	Software that filters network traffic based on rules
Port	Numbered address where a service is reachable, e.g. `11434` for Ollama
ufw	Uncomplicated Firewall, simplified frontend for firewall rules on Ubuntu
iptables	Classic tool for direct configuration of Netfilter rules in the kernel
`0.0.0.0`	Bind address that includes all network interfaces
`127.0.0.1`	Loopback address, only reachable locally, not from the network
Allow	Rule that permits specific network traffic
Deny	Rule that blocks specific network traffic
Rule	Individual firewall rule with defined criteria
Default Policy	Behavior when no rule matches, typically Allow or Deny

Ollama default behavior

Ollama binds to 127.0.0.1:11434 by default. This means only processes on the same machine can access the service. Other devices on your network cannot reach port 11434. In this configuration, you don’t need firewall rules for Ollama because the service doesn’t listen externally.

This changes when you want Ollama available across your network. You set the environment variable OLLAMA_HOST to 0.0.0.0:11434, and Ollama listens on all network interfaces. Now any device that can ping your server can reach the service. This is where the firewall comes in.

For details on configuration, see the Ollama configuration and Ollama network access articles.

Firewall with ufw

ufw comes preinstalled on Ubuntu but is often disabled. The following steps show you how to enable the firewall and define rules for Ollama.

Step 1: Install ufw and check status

sudo apt update && sudo apt install ufw
sudo ufw status

The output shows Status: inactive as long as the firewall is not running.

Step 2: Set default policy

Before you enable the firewall, define your default policy. The safest approach is to block incoming traffic by default and allow outgoing.

sudo ufw default deny incoming
sudo ufw default allow outgoing

Step 3: Allow SSH access

If you enable the firewall now, all incoming connections are blocked, including SSH. This locks you out if you’re working remotely. First, allow SSH.

sudo ufw allow ssh

If you run SSH on a different port, say 2222, specify it explicitly:

sudo ufw allow 2222/tcp

Step 4: Enable the firewall

sudo ufw enable

ufw warns you that existing SSH connections might be interrupted. Confirm with y if you allowed SSH in the previous step.

Step 5: Allow Ollama only for localhost

If Ollama listens on 127.0.0.1, you don’t need an additional rule because loopback traffic is allowed by ufw by default. If Ollama listens on 0.0.0.0 but you want to restrict access to a specific device, allow the port only for that IP address.

sudo ufw allow from 192.168.1.50 to any port 11434

Replace 192.168.1.50 with the IP address of your workstation. All other devices on your network cannot connect to port 11434.

Step 6: Verify the Rules

sudo ufw status verbose

The output lists all active rules with their directions and source IP addresses. Verify that only your intended ports and addresses are permitted.

Firewall with iptables

If you’re not using ufw or need more granular control, you can configure iptables directly. The syntax is more complex, but you’ll see exactly which rules are active in the kernel.

Display Existing Rules

sudo iptables -L -n -v

The -L flag lists the rules, -n skips DNS resolution of IP addresses, and -v shows additional statistics like packet counters.

Allow Loopback Traffic

sudo iptables -A INPUT -i lo -j ACCEPT

This rule permits all traffic on the loopback interface lo, allowing local processes like Ollama to function freely on 127.0.0.1.

Allow Established Connections

sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

Connections that are already open or related to an existing connection are accepted. This prevents responses to outgoing requests from being blocked.

Allow SSH

sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Allow Ollama for a Specific IP

sudo iptables -A INPUT -p tcp -s 192.168.1.50 --dport 11434 -j ACCEPT

Block Everything Else

sudo iptables -A INPUT -j DROP

This rule sits at the end of the chain and drops all packets that don’t match any previous rule. Order matters because iptables processes rules from top to bottom and stops at the first match.

Save Rules Permanently

iptables rules are lost on reboot. Save them permanently with iptables-persistent:

sudo apt install iptables-persistent
sudo netfilter-persistent save

Best Practices

Default Deny as Your Starting Point

Always set the default policy to deny for incoming traffic. Allow only what you explicitly need. This principle of minimal attack surface reduces the risk that a new service accidentally becomes accessible.

Be Specific, Not General

Restrict ports to concrete IP addresses or subnets rather than opening them globally. ufw allow 11434 exposes the port to the entire internet if your server has a public IP. ufw allow from 192.168.1.0/24 to any port 11434 restricts access to your local subnet.

VPN Over Port Forwarding

If you need to access Ollama from outside your network, don’t open the port in your router. Use a VPN like Tailscale instead, which creates an encrypted connection between your devices. The port remains invisible externally, and only authenticated VPN participants can connect.

Document Your Rules

Firewall rules grow over time. Comment each rule so you remember why it exists months later. With ufw, use descriptive names in the rule comment. With iptables, keep a separate document or use iptables-save with comments.

Review Regularly

Go through your rules every few weeks. Remove rules for services you no longer use. Every open rule is a potential attack vector.

Example: Securing Ollama Access

This example walks through the complete process of configuring Ollama so only your work machine can access it.

Step 1: Bind Ollama to 0.0.0.0

Set the environment variable in Ollama’s service file:

sudo systemctl edit ollama.service

Add these lines in the editor:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Save and reload the configuration:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Step 2: Set a Firewall Rule

Allow port 11434 only from your work machine:

sudo ufw allow from 192.168.1.50 to any port 11434

Step 3: Test Access

Test from your work machine:

curl http://192.168.1.10:11434/api/tags

Replace 192.168.1.10 with your server’s IP address. You should get a JSON response with the installed models.

Test from another device on the network. The connection should time out or be refused.

Step 4: Optional, Add a Reverse Proxy

For additional security and TLS encryption, put a reverse proxy in front of Ollama. The proxy terminates TLS, provides authentication, and forwards requests internally to Ollama. Your firewall then allows only port 443 to the proxy, while port 11434 stays completely closed.

Common Pitfalls

Locking yourself out of SSH: The most common mistake. If you enable the firewall without allowing SSH, you lose server access. Always allow SSH as your first rule.
Forgetting the default policy: Without an explicit default policy, ufw can end up in an ambiguous state. Set deny incoming and allow outgoing before enabling it.
Blocking loopback traffic: An overly strict rule can block loopback traffic, preventing local processes from communicating. Make sure 127.0.0.1 is always permitted.
Ignoring rule order: With iptables, order is critical. A DROP rule at the start of the chain blocks everything, even if an ACCEPT rule for the same port comes later.
Overlooking a public IP: If your server has a public IP and you run ufw allow 11434 without a source IP, Ollama becomes accessible from the entire internet. Always check whether your server sits behind a router or is directly exposed.
Not saving rules permanently: iptables rules vanish after a reboot. Use iptables-persistent to save them, or your server stands unprotected after the next boot.
Neglecting IPv6: If IPv6 is active, iptables rules for IPv4 don’t automatically apply to IPv6. Configure ip6tables separately or use ufw, which covers both protocols.
Docker bypassing the firewall: Docker manipulates iptables rules directly and can circumvent your manual rules. If you run Ollama in Docker, check whether the container port is exposed externally, and use Docker networks instead of port mappings.

Hardware, Costs, and Security

A firewall requires no additional hardware. It runs as software on your existing server. The tools ufw and iptables are open source and free. Your only cost is the time spent configuring and regularly reviewing it.

A firewall is not a silver bullet for security. It filters network traffic but doesn’t protect against attacks coming through allowed ports. If you open port 11434 to a device and that device is infected with malware, the firewall won’t help. It’s one layer in a multi-layered security strategy, complemented by authentication, encryption, and regular updates.

FAQ

What’s the difference between ufw and iptables?

ufw is a simplified frontend that generates iptables rules behind the scenes. It works well for standard configurations and is easier to read. iptables gives you direct control over Netfilter tables and chains, allowing finer-grained rules, but it requires more expertise.

Does a firewall also block outgoing traffic?

Not by default with ufw. The default policy for outgoing traffic is allow. You can change it to deny, but then you’ll need to explicitly allow each outgoing port, which significantly increases administrative overhead.

Do I need a firewall if Ollama is listening on 127.0.0.1?

No. If Ollama is only bound to 127.0.0.1, the service isn’t reachable from the network. A firewall rule for port 11434 isn’t necessary in that case. However, the firewall does protect other services on your server that listen externally.

How do I test whether my firewall rules are working?

Use curl or nc from another device to check if the port is accessible. A timeout or connection refused means the firewall is blocking the traffic. A successful response indicates the rule is in effect.

nc -zv 192.168.1.10 11434

What happens if I disable ufw?

sudo ufw disable removes all active rules from the kernel and stops blocking any traffic. The rules remain stored in the configuration and will be restored when you re-enable it.

Can I use ufw and iptables at the same time?

Yes, but carefully. ufw generates iptables rules, and manual iptables rules can conflict with them. If you use both, check the complete rule chain with iptables -L -n -v to ensure no conflicting rules exist.

How do I secure Ollama for remote access over the internet?

Don’t open port 11434 on your router. Use a VPN like Tailscale, which creates an encrypted tunnel between your device and the server. The port remains hidden from the outside, and only authenticated VPN participants can access it.

Why is my ufw rule for Ollama not working?

If Ollama runs in Docker, Docker can bypass ufw rules because it adds its own iptables rules for port mappings. Check with iptables -L -n -v whether Docker has directly exposed the port, and use Docker networks instead of port mappings.

Do IPv6 rules need to be configured separately?

With iptables, yes, because it only handles IPv4. You need to configure ip6tables separately. ufw handles IPv4 and IPv6 together as long as the option IPV6=yes is set in /etc/default/ufw.

What does it cost to run a firewall?

Nothing. ufw and iptables are open source and pre-installed on most Linux distributions. The only cost is your time for configuration and maintenance.

How often should I review my firewall rules?

Every few weeks or after any change to your services. Remove rules for services you no longer use, and check whether new services have accidentally opened ports.

Sources

Ubuntu Documentation, Uncomplicated Firewall
Ollama Documentation, Configuration and Environment Variables
Netfilter Project, iptables Documentation
Tailscale Documentation, Access Controls

Täglicher Artikel