HashiCorp Vault for Local AI

What This Article Covers

• What Vault is and when it makes sense to use it.
• Running Vault locally with Docker.
• Storing, retrieving, and rotating secrets.
• How dynamic credentials work.
• Integrating Vault with AI agents and self-hosting tools.

Introduction: HashiCorp Vault for Local AI

HashiCorp Vault is an enterprise-grade tool for managing secrets, credentials, and encryption keys. It provides encrypted storage, dynamic credentials, automatic rotation, and fine-grained access control. For local AI projects, Vault can be valuable when you need to manage many API keys, database passwords, or certificates.

For small homelabs, Vault often feels like overkill. Once you’re running multiple agents, tools, and containers that need secrets, however, a centralized secret manager becomes worthwhile. Vault is particularly valuable when you need dynamic database credentials or time-limited access tokens.

Key Concepts

• KV Engine: Key-value store for secrets.
• Dynamic Secrets: Short-lived, automatically generated credentials.
• PKI Engine: Certificate issuance and management.
• Transit Engine: Encryption as a service.
• Policy: Controls who can access which paths.
• Token: Short-lived access key.
• AppRole: Authentication method for applications.
• Unseal: Unlock the Vault storage.

Why Vault for Local AI?

• Centralized management: One place for all secrets.
• No hardcoded keys: Applications fetch secrets at runtime.
• Audit logs: Track who accessed what secret and when.
• Rotation: Automatically replace database passwords.
• Time-limited tokens: Reduces risk if credentials leak.

Installation with Docker

A simple test setup runs with Docker Compose:

services:
  vault:
    image: hashicorp/vault:latest
    container_name: vault
    ports:
      - "8200:8200"
    environment:
      - VAULT_DEV_ROOT_TOKEN_ID=MEIN_DEV_TOKEN
      - VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200
    volumes:
      - vault-data:/vault/file
    cap_add:
      - IPC_LOCK
    command: server -dev -dev-root-token-id="MEIN_DEV_ROOT_TOKEN"

volumes:
  vault-data:

Start it:

docker compose up -d

The web UI is accessible at http://localhost:8200.

Production Setup

For production use, Vault should not run in dev mode:

• Use disk-based storage.
• Enable TLS.
• Create multiple unseal keys.
• Authenticate via AppRole, TLS certificates, or LDAP.
• Enable audit logging.

Storing Your First Secret

After startup, authenticate:

export VAULT_ADDR='http://localhost:8200'
export VAULT_TOKEN='MEIN_DEV_TOKEN'

Store a secret:

vault kv put secret/ollama-api-key key=MEIN_KEY

Retrieve a secret:

vault kv get secret/ollama-api-key

Application Integration

Applications can fetch secrets via the REST API or an SDK:

import requests

headers = {"X-Vault-Token": "MEIN_DEV_TOKEN"}
resp = requests.get("http://localhost:8200/v1/secret/data/ollama-api-key", headers=headers)
data = resp.json()
print(data["data"]["data"]["key"])

In production, use AppRole or a machine-authenticated path instead of the root token.

Policies

Vault uses paths and policies to control access:

path "secret/data/ollama/*" {
  capabilities = ["read"]
}

Policies are bound to tokens or roles.

Dynamic Database Credentials

One of Vault’s strongest features is generating short-lived database users:

vault secrets enable database
vault write database/config/my-postgres \
  plugin_name=postgresql-database-plugin \
  allowed_roles="readonly" \
  connection_url="postgresql://{{username}}:{{password}}@db:5432/postgres" \
  username="vaultadmin" \
  password="vaultpass"

Create a role:

vault write database/roles/readonly \
  db_name=my-postgres \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
    GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

Applications now request credentials that expire after one hour.

Vault and AI Agents

AI agents like OpenClaw or OpenHands often need API keys. Instead of storing them in .env files, they can fetch them from Vault:

• Agent starts with a Vault token.
• It reads API keys before use.
• Tokens are kept short-lived.
• Rotation happens centrally.

This simplifies managing multiple providers and reduces risk.

Common Pitfalls

• Dev mode in production: Data is lost on restart.
• Using the root token everywhere: Security risk.
• No TLS: Communication is unencrypted.
• Losing unseal keys: Vault cannot be unlocked.
• Overly permissive policies: Anyone can read everything.
• Missing backups: Vault data must be backed up.

Further Reading

• BotServ.de Secret Management
• BotServ.de Environment Variables
• BotServ.de Secret Rotation
• BotServ.de Secure API Key Management

FAQ: HashiCorp Vault Locally

Is Vault overkill for a small homelab? Often yes. For a few secrets, .env files or Infisical are sufficient. Vault pays off with many agents and dynamic credentials.

Can I install Vault without Docker? Yes, binaries are available for Linux, macOS, and Windows.

What happens if I lose data? In dev mode, all data is lost on restart. In production mode, backups and unseal keys must be maintained.

How secure is Vault? Very secure when TLS, strong authentication, and well-designed policies are used.

Can Vault rotate secrets automatically? Yes, especially for dynamic database credentials and some cloud providers.

Sources and Further Reading

• HashiCorp Vault Docs: https://developer.hashicorp.com/vault/docs
• Vault Docker Image: https://hub.docker.com/r/hashicorp/vault/
• Vault Learn: https://developer.hashicorp.com/vault/tutorials

Summary: HashiCorp Vault for Local AI

HashiCorp Vault is a powerful secret manager that adds real value to local AI setups. Docker makes it quick to test, but sustained use requires TLS, backups, unseal keys, and careful policy design. It shines when managing many API keys, agents, and databases through centralized management, audit logging, and dynamic credentials. Master the complexity, and you get professional secrets management in your own network.