Secrets and API Keys in AI Projects

What this article covers

• Why secrets demand careful attention.
• How to store API keys properly.
• Which tools work well for self-hosting.
• Common mistakes to avoid.

Introduction: Secrets and API Keys

Nearly every AI project works with secrets. These might be API keys for external services, Discord bot tokens, database passwords, or credentials for AI models. If stored insecurely, they become a liability. A single exposed key in a public repository or log file can cause serious damage.

Local AI reduces the need for external keys, but you rarely escape them entirely. Even a purely local bot needs a token, and a RAG system typically connects to a database.

Why do you need secure secrets management?

An API key functions like a password. If someone obtains it, they can use your services, rack up charges, or steal data. With local projects, the risk often gets underestimated because the server isn’t directly public. Yet internal attackers, corrupted backups, or accidentally pushed files can still leak secrets.

Secure management means: secrets never live in code, logs, or backups. They’re loaded only at runtime and passed only to authorized processes.

Secrets management at a glance

Several security tiers exist:

• Environment variables: Simple, but visible at the host level.
• .env files: Convenient, but shouldn’t end up in Git.
• Docker Secrets: Designed for container environments.
• HashiCorp Vault / Infisical: Centralized vault solutions suitable for self-hosting.
• Password managers: Sufficient for personal development projects.

The best approach depends on how many secrets you manage and who needs access. For a small hobby project, environment variables suffice. For multiple users or production workflows, a proper secret manager makes sense.

Who should use secrets management?

• Developers running bots, agents, or RAG systems.
• Admins managing multiple containers and services.
• Users who want to ensure their API keys don’t leak.
• Teams collaborating on AI projects.

Key terminology around secrets

• .env: A file containing environment variables that shouldn’t be committed.
• Docker Secret: A secret managed within Docker Swarm or Compose.
• HashiCorp Vault: A scalable open-source solution for secret management.
• Infisical: A modern open-source alternative with strong self-hosting support.
• Principle of Least Privilege: Grant each process only the permissions it actually needs.

Practical examples for secrets management

Using .env correctly

Create a .env file:

DISCORD_TOKEN=dein_token
OLLAMA_HOST=http://localhost:11434
DB_PASSWORD=geheim

Load it in Python:

from dotenv import load_dotenv
import os

load_dotenv()
token = os.environ['DISCORD_TOKEN']

Make sure .env is in .gitignore:

.env

Using Docker Secrets

In docker-compose.yml, you can reference secrets:

services:
  app:
    secrets:
      - api_key

secrets:
  api_key:
    file: ./secrets/api_key.txt

The secret appears in the container at /run/secrets/api_key and never ends up in the image.

Self-hosted HashiCorp Vault

Vault makes sense when you have many secrets and multiple access levels. It runs as its own container and provides dynamic secrets, expiration policies, and detailed audit logs.

Common pitfalls with secrets

• Storing keys in code: Even private repositories aren’t safe.
• Committing .env: A frequent mistake that happens quickly.
• Outputting secrets in logs: Error messages or debug output can leak keys.
• Overly broad permissions: Every process gets access to all keys.
• No rotation: Once created, keys are never refreshed.

Further resources on secrets and API keys

• dotenvx: Secure management of .env files.
• HashiCorp Vault
• Infisical
• Docker Secrets

FAQ: Secrets and API Keys

Are .env files secure enough? For local development, yes, as long as they don’t enter Git. For production or team environments, a vault is worth the effort.

Which is better: .env or Docker Secrets? Docker Secrets win for container deployments because they don’t get baked into the image.

Should I rotate API keys? Yes. Regularly refreshing keys reduces risk if a leak occurs.

How do I know if a key has been exposed? Many providers alert you to unusual traffic. Git leak scanners help find keys already in repositories.

Is Vault worthwhile for a small project? Probably not. A few keys are fine with solid .env management. Once you have multiple users or sensitive data, Vault becomes the better choice.

Sources and further reading

• OWASP Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
• HashiCorp Vault Docs: https://developer.hashicorp.com/vault/docs
• Docker Secrets: https://docs.docker.com/engine/swarm/secrets/

Summary: Secrets and API Keys

Secure secrets management matters in every AI project. Environment variables and .env files get you started, as long as they don’t land in Git. For containers, Docker Secrets work well. For larger setups, HashiCorp Vault or Infisical offer more power. The essentials: keep keys out of code, keep them out of logs, and rotate them if you suspect a breach.