Managing Docker Secrets with Compose

IRC-Mania Die Seite rund um Sicherheit und Programmierung

Internet, Sicherheit , Internet Relay Chat

Gesponsert

What this article covers

• What Docker Secrets are and why they’re more secure than environment variables.
• Which methods exist for managing Secrets in Docker Compose.
• How to use external Secrets with Docker Swarm and Compose.
• How to properly use .env files and what to avoid.
• Common pitfalls with Secrets in Docker and how to solve them.

Introduction: Understanding Docker Secrets

Secrets are confidential data like passwords, API keys, tokens, and certificates. In Docker Compose, these Secrets need to reach containers without appearing in plain text in docker-compose.yml or on the command line. Docker Secrets is a mechanism that solves this securely.

This article is aimed at developers using Docker Compose who want to manage Secrets securely. You should understand how Docker Compose works and what Secret Management means. For security topics related to Docker, IRC-Security.de provides additional resources.

Why do I need Docker Secrets?

Imagine you manage a Docker stack running Nextcloud, PostgreSQL, and Ollama. Each service needs passwords: the database for the admin, Nextcloud for the database connection, Ollama for the API. If you put these passwords as environment variables in docker-compose.yml, they’re readable to anyone who sees the file.

Worse, environment variables are visible inside the container. Anyone who accesses the container can run env to read all the passwords. They end up in logs, crash dumps, and process lists that any system user can view.

Docker Secrets solve this problem by mounting Secrets as files visible only to the respective container. They don’t show up in environment variables, process lists, or logs.

Docker Secrets explained

Docker Secrets are confidential data that Docker manages and provides to containers as a file. The container reads the Secret from a file under /run/secrets/ instead of receiving it as an environment variable. The file exists only in the container’s memory and is never written to disk.

The core idea is: Secrets belong in files, not in environment variables. Files can be protected, rotated, and audited. Environment variables cannot.

Who Docker Secrets are for

• Developers operating Docker Compose stacks with sensitive data.
• System administrators managing Secrets in Docker environments.
• Security teams ensuring Secrets aren’t exposed in plain text.
• Self-hosters running their services securely.

Experience with Docker Compose and basic Secret Management is helpful. If you’ve never worked with Docker Compose, start with the fundamentals first.

Key terms around Docker Secrets

• Secret - Confidential data like passwords, API keys, tokens. Useful when: any service needs to authenticate.
• Docker Compose - Tool for defining multi-container stacks. Useful when: running any Docker stack.
• Docker Swarm - Container orchestration with built-in Secret Management. Useful when: you want to manage Secrets natively.
• .env file - File with environment variables that Docker Compose loads automatically. Useful when: values aren’t highly sensitive.
• HashiCorp Vault - Centralized Secret Management system. Useful when: managing many Secrets and rotation.
• Infisical - Open-source Secret Manager. Useful when: seeking an alternative to Vault.
• Mozilla SOPS - Tool for encrypting Secrets in files. Useful when: you want to version Secrets in Git.

Methods for Secrets in Docker Compose

Several methods exist for managing Secrets in Docker Compose. Each has pros and cons.

Method 1: Environment variables (not recommended)

The simplest but least secure approach. Passwords appear as environment variables in the container and are visible to anyone accessing it.

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: "mein-geheimes-passwort"

Use this method only for non-sensitive data. Passwords and API keys don’t belong here.

Method 2: .env file

A .env file sits in the same directory as docker-compose.yml, and Docker Compose loads it automatically. The variables are referenced in the Compose file.

.env:

POSTGRES_PASSWORD=mein-geheimes-passwort

docker-compose.yml:

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

Don’t commit the .env file to your Git repository. Add it to .gitignore. This approach is better than plain text in the Compose file, but passwords still land as environment variables in the container.

Method 3: Docker Compose Secrets (recommended)

Docker Compose has supported Secrets since version 1.27. You define Secrets in a secrets section and mount them into containers. Secrets become available as a file under /run/secrets/ in the container.

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
    secrets:
      - postgres_password

secrets:
  postgres_password:
    file: ./secrets/postgres_password.txt

The file ./secrets/postgres_password.txt contains the password in plain text. Docker reads the file and provides its contents to the container as a file under /run/secrets/postgres_password. The password isn’t in environment variables; the application reads it from the file instead.

Important: Many official images support the *_FILE convention. Instead of POSTGRES_PASSWORD, you use POSTGRES_PASSWORD_FILE and provide the path to the Secret file. This applies to PostgreSQL, MySQL, Redis, Nextcloud, and many others.

Method 4: External Secrets with Docker Swarm

With Docker Swarm, you can manage Secrets centrally and make them available to containers. Secrets are stored encrypted in the Swarm and only accessible to containers that need them.

# Create Secret
echo "mein-geheimes-passwort" | docker secret create postgres_password -

# Use Secret in Compose

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
    secrets:
      - postgres_password

secrets:
  postgres_password:
    external: true

This approach is the most secure because Secrets don’t live in files on disk; instead, they’re stored in the Swarm. The trade-off is that you need a running Swarm.

Method 5: Secret Management Tools

For larger deployments, a centralized secret management system like HashiCorp Vault or Infisical is worth the investment. These systems manage secrets centrally, rotate them automatically, and log every access.

Integration with Docker Compose typically happens through init containers or sidecar containers that fetch secrets and expose them as files.

Practical Example: A Stack with Docker Secrets

Imagine you want to deploy a stack with PostgreSQL, Nextcloud, and Ollama. All three need passwords. Here’s how to set it up with Docker Secrets:

1. Create a secrets directory: mkdir -p secrets
2. Generate secrets:

   openssl rand -base64 32 > secrets/postgres_password.txt
   openssl rand -base64 32 > secrets/nextcloud_password.txt
   openssl rand -base64 32 > secrets/ollama_api_key.txt
   chmod 600 secrets/*.txt

3. docker-compose.yml with secrets:

   services:
     postgres:
       image: postgres:16
       environment:
         POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
       secrets:
         - postgres_password
       volumes:
         - postgres-data:/var/lib/postgresql/data
   
     nextcloud:
       image: nextcloud:latest
       environment:
         POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
         NEXTCLOUD_ADMIN_PASSWORD_FILE: /run/secrets/nextcloud_password
       secrets:
         - postgres_password
         - nextcloud_password
       depends_on:
         - postgres
   
     ollama:
       image: ollama/ollama:latest
       environment:
         OLLAMA_API_KEY_FILE: /run/secrets/ollama_api_key
       secrets:
         - ollama_api_key
       volumes:
         - ollama-data:/root/.ollama
   
   secrets:
     postgres_password:
       file: ./secrets/postgres_password.txt
     nextcloud_password:
       file: ./secrets/nextcloud_password.txt
     ollama_api_key:
       file: ./secrets/ollama_api_key.txt
   
   volumes:
     postgres-data:
     ollama-data:

4. Update .gitignore: Add secrets/ so secrets never end up in version control.
5. Start the stack: docker compose up -d

Passwords won’t appear in your docker-compose.yml, environment variables, or logs. Each container reads only the secret it needs.

Rotating Secrets

Secrets should be rotated regularly, especially in long-running services. Here’s how secret rotation works with Docker Compose:

1. Generate a new secret: openssl rand -base64 32 > secrets/postgres_password.txt.new
2. Back up the old secret: cp secrets/postgres_password.txt secrets/postgres_password.txt.bak
3. Activate the new secret: mv secrets/postgres_password.txt.new secrets/postgres_password.txt
4. Restart the service: docker compose up -d postgres

Important: For databases, you must also change the password inside the database itself. The secret file alone isn’t enough; the password must be updated in the database. With PostgreSQL, use ALTER USER postgres PASSWORD 'new-password';.

For automated rotation, consider a tool like HashiCorp Vault or Infisical, which handles rotation for you. The article Secret Rotation automatisieren covers this in detail.

Common Pitfalls with Docker Secrets

• Secrets in Git: Your .env file and secrets directory must never be committed. Check your .gitignore.
• Incorrect file permissions: Secret files should have chmod 600 so only the owner can read them.
• Image doesn’t support *_FILE convention: Not every image supports reading passwords from files. Check the image documentation.
• Secrets in logs: Some applications log passwords when reading from files. Check the logs after startup.
• No rotation: Using the same password for years risks compromise. Regular rotation is mandatory.
• Secrets in environment variables as a fallback: If an image doesn’t support *_FILE, passwords often end up in environment variables anyway. Find a different image or build your own.
• No backup of secrets: If secret files are lost, services become inaccessible. Store secrets in a secure location.
• Secrets baked into the image: Some Dockerfiles embed passwords as environment variables. This is a massive security risk because anyone can inspect the image.

Further Reading and Resources on Docker Secrets

• Docker Compose optimieren - Best practices for Compose files.
• Docker Swarm - Orchestration with native secret management.
• Docker Sicherheit - Securing containers.
• Secret-Management - Overview of secret management tools.
• HashiCorp Vault - Centralized secret management.
• Secret Rotation automatisieren - Automate password rotation.
• Umgebungsvariablen - Pros and cons of .env.

Key Takeaways:

• Docker Secrets are more secure than environment variables because they’re mounted as files in the container.
• The *_FILE convention lets many images read passwords from files.
• .env files are better than plaintext in the Compose file, but still not ideal.
• External secrets with Docker Swarm offer the most secure approach.
• Rotation, backups, and .gitignore are non-negotiable.

FAQ: Docker Secrets with Compose - Common Questions

Sources and further reading

• Docker Secrets Documentation - Official Docker Secrets documentation.
• Docker Compose Secrets - Secrets in Compose.
• HashiCorp Vault - Centralized secret management.
• Mozilla SOPS - Encrypted secrets in Git.
• Infisical - Open-source secret manager.

Premium VPS Hosting

Schnelle und sichere VPS Server

Gesponsert