Self-Hosting Fundamentals

IRC-Mania Die Seite rund um Sicherheit und Programmierung

Internet, Sicherheit , Internet Relay Chat

Gesponsert

What This Article Covers

• What self-hosting means and how it differs from cloud services.
• The benefits and responsibilities you take on when running your own infrastructure.
• What hardware, operating system, and software you need to get started.
• How Docker, Proxmox, and Linux connect, and where local AI fits in.
• Common pitfalls beginners face and how to avoid them.

Introduction: Understanding Self-Hosting

Self-hosting means running software and services on your own hardware instead of renting them from a cloud provider. Rather than using AWS for a database, Slack for chat, or OpenAI for AI, you install these services on your own server at home or in a data center you control. You keep full control over your data, configuration, and costs.

This article is aimed at newcomers exploring self-hosting for the first time. You don’t need deep system administration skills, but you should be willing to learn about Linux, containers, and basic networking. If you’re looking to learn programming fundamentals, IRC-Coding.de has tutorials on Python, C#, and more. Security topics are covered on IRC-Security.de.

Why Self-Host?

Imagine relying on a cloud service for your notes, contacts, or AI models. One day the provider raises prices, discontinues a feature, or closes your account. Your data vanishes or becomes accessible only at steep cost. It happens more often than you’d think.

Self-hosting solves this because you retain ownership of your data. You decide where it’s stored, who can access it, and how long to keep it. This matters especially for sensitive information like customer data, health records, or internal documents.

There’s also the cost argument. A modest GPU instance in the cloud easily runs hundreds of euros per month. A self-owned server with used GPU cards pays for itself within months, then runs at just the cost of electricity. If you’re running local AI with Ollama, self-hosting is unavoidable anyway since the models need to run on your own hardware.

Self-Hosting in a Nutshell

Self-hosting is the practice of installing and maintaining software on your own hardware. That could be a Raspberry Pi under your desk, an old PC in the basement, or a full server rack. The software typically runs in containers that bundle all dependencies and are simple to manage.

The core principle is straightforward: you own the hardware, you control the software, you’re responsible for the data. No one can remotely disable a feature or access your data as long as you secure your systems properly.

Who Should Self-Host?

Self-hosting appeals to several groups:

• Individuals who don’t want to hand their data to tech giants and enjoy experimenting.
• Small and medium businesses that need GDPR compliance and want to cut cloud costs.
• Developers building local AI, RAG pipelines, or agent workflows and requiring their own infrastructure.
• Hobbyists experimenting with Home Assistant, Nextcloud, and custom chatbots.

Prior Linux experience is helpful but not mandatory. If you’ve never used a command line, start with a Linux tutorial covering the basics. The learning curve is steep but rewarding.

Key Concepts in Self-Hosting

• Docker - Container platform that packages software in isolated environments. Use it for: almost every service you want to self-host.
• Proxmox VE - Virtualization platform for VMs and containers on a single host. Use it when: you need multiple isolated systems on one server.
• Linux - Operating system powering most servers. Use it as: the foundation for any self-hosting setup.
• Ollama - Local model server for AI models. Use it when: you want to run local AI.
• Tailscale - VPN service for secure remote access. Use it when: you need to reach your services from outside your network.
• Reverse Proxy - Service that forwards external requests to internal services. Use it when: you want multiple services accessible through a single domain.
• Kubernetes - Container orchestration for large deployments. Use it when: managing many containers across multiple servers.

Hardware for Getting Started

Your hardware determines what you can host. A simple note-taking server runs fine on a Raspberry Pi with 4 GB RAM. Local AI with 7B models needs a GPU with at least 8 GB VRAM, larger models require more.

Raw compute power matters, but so does reliability. Servers run 24/7, so you need stable power, adequate cooling, and ideally a UPS (Uninterruptible Power Supply) to bridge brief outages. Find details in Planning an AI Cluster.

Operating System: Which Linux for Self-Hosting?

Most self-hosting servers run Linux because it’s free, stable, and resource-efficient. Three distributions have proven themselves:

• Ubuntu Server - Beginner-friendly, enormous community, long support cycles. Great for your first server.
• Debian - Even leaner than Ubuntu, extremely stable. The standard for many production servers.
• Proxmox VE - Built on Debian but includes a web interface for virtualization. Ideal if you want to run multiple VMs or containers on a single host.

For pure AI workloads without virtualization, Ubuntu or Debian suffice. Once you need to cleanly separate multiple services, Proxmox makes sense because you can create a separate VM or LXC container for each purpose. See Proxmox LXC vs. VM for guidance on choosing.

Docker: The Standard for Self-Hosting

Docker is the critical technology for self-hosting because it packages software into containers. A container bundles the application, all its libraries, and configuration into a single unit. Instead of installing individual packages on your system, you spin up a container that brings everything it needs.

Benefits of Docker:

• Isolation: Each service runs in its own environment and doesn’t affect others.
• Simple updates: A new container replaces the old one, keeping your system clean.
• Portability: A container runs anywhere Docker is installed.
• Reproducibility: Your configuration lives in a docker-compose.yml file that you can version control.

The typical setup is straightforward: create a docker-compose.yml, define your services, and start everything with docker compose up -d. Learn more in Docker Compose optimization.

services:
  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama-data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  ollama-data:

This example runs Ollama in Docker with GPU support. The ollama-data volume persists your models across restarts.

Networking and Security in Self-Hosting

When you host services yourself, you must secure them too. A server exposed directly to the internet becomes a target within minutes. Three core principles help:

1. Don’t open all ports to the world. Only expose services that need external access, like a web server on port 443. Keep everything else on your local network.
2. Use a reverse proxy. A reverse proxy like Nginx or Caddy receives external requests and forwards them to internal services. This gives you one place to handle TLS encryption and access control.
3. VPN for remote access. Instead of exposing services directly online, use a VPN like Tailscale or WireGuard. You can reach your services securely from anywhere without making them publicly available.

If you handle sensitive data, also manage secrets properly, configure a firewall, and maintain regular backups. The guide on secure operation basics covers these in detail.

Local AI and Self-Hosting

Local AI is one of the most compelling use cases for self-hosting. Instead of relying on ChatGPT, Claude, or Gemini, you run your own models on your own hardware. This approach offers several advantages:

• Privacy: Your data never leaves your network.
• Cost: After buying hardware, you only pay for electricity.
• Independence: No API limits, no price increases, no service shutdowns.
• Customization: Fine-tune models, adjust system prompts, and integrate your own tools.

A typical local AI stack includes:

• Ollama as your model server.
• Open WebUI as your chat interface.
• AnythingLLM for RAG with your own documents.
• n8n for workflow automation with AI.
• A vector database like Chroma or Qdrant for embeddings.

To build AI agents, you’ll also need a framework like LangGraph or CrewAI and possibly an MCP server so agents can access tools. All of this runs best in containers on your own server.

Real-World Example: A Small Self-Hosting Server

Imagine setting up a server for Nextcloud, Ollama, and a reverse proxy. Here’s how it goes:

1. Get the hardware: A mini PC with an N100 CPU, 16 GB RAM, and 1 TB NVMe handles Nextcloud and smaller AI models. For larger models, add a dedicated GPU.
2. Install the OS: Flash Ubuntu Server 24.04 LTS onto the NVMe, enable SSH, and update the system (apt update && apt upgrade).
3. Install Docker: curl -fsSL https://get.docker.com | sh installs Docker and Docker Compose.
4. Define your services: Create a docker-compose.yml with Nextcloud, Ollama, and Caddy as your reverse proxy.
5. Secure the network: Enable the UFW firewall and open only port 443 and 22 (SSH, ideally just for your LAN).
6. Set up TLS: Caddy automatically fetches Let’s Encrypt certificates for your domain.
7. Configure backups: A cron job backs up your volumes to an external drive each night.

After an afternoon of work, you have a fully functional server running Nextcloud for files, Ollama for local AI, and encrypted remote access. Ongoing costs are minimal: just electricity (a few euros per month) and your domain registration.

Common Self-Hosting Pitfalls

• Forgetting to check ports: Many beginners open all ports in frustration to “finally get it working.” That’s a massive security risk. Invest time in configuring your reverse proxy correctly instead.
• No backups: A disk failure without backups means data loss. Your volumes and configurations must live somewhere else. See Docker volume backups.
• Too many services on too little hardware: A Raspberry Pi with 4 GB RAM can’t handle local AI and Nextcloud at the same time. Be realistic about what your hardware can do.
• Neglecting updates: Self-hosted software still needs updates. Running docker compose pull && docker compose up -d monthly keeps everything current.
• Secrets in plain text: Passwords and API keys don’t belong in your docker-compose.yml. Use Docker Secrets or a .env file that stays out of Git.
• No monitoring: If you don’t notice a full disk or a crashed container, you’ll only find out when users complain. Docker monitoring helps prevent that.

Further Reading and Self-Hosting Resources

• Docker basics - How containers work and how to use them.
• Proxmox installation - Set up virtualization on a host.
• Linux basics - Ubuntu, Debian, and system administration.
• Networking and Tailscale - Secure remote access to your services.
• Local AI with Ollama - Run AI models on your own hardware.
• Secure operations - Firewalls, secrets, and privacy for self-hosting.

Key Takeaways:

• Self-hosting gives you control over your data, software, and costs.
• Docker is the standard for packaging and running services.
• Security starts with a firewall, reverse proxy, and VPN.
• Local AI is one of the most important self-hosting use cases.
• Backups and updates are mandatory, not optional.

FAQ: Self-Hosting Fundamentals - Common Questions

Resources and Further Reading

• Docker Documentation - Official Docker docs.
• Proxmox VE Wiki - Virtualization with Proxmox.
• Ubuntu Server Guide - Official Ubuntu Server documentation.
• Selfhosted Podcast - Weekly podcast on self-hosting.
• Ollama GitHub - Local model server.

Premium VPS Hosting

Schnelle und sichere VPS Server

Gesponsert