Centralize Docker Logs

What this article covers

• Why centralized logging matters.
• How Docker handles logs by default.
• Forwarding logs to Loki, Fluentd, or Vector.
• Searching logs in Grafana.
• Tips for parsing, rotation, and security.

Introduction: centralize Docker logs

Containers generate a lot of logs. By default, Docker stores them locally on the host in JSON format. This works fine for small setups, but becomes unwieldy when running multiple services, analyzing failures, or tracing incidents. A centralized logging solution aggregates all logs in one place, enabling search, filtering, and alerting.

This article shows how to forward Docker logs to centralized systems like Loki, Fluentd, or Vector and visualize them in Grafana.

Key concepts

• Logging driver: Component that stores or forwards Docker logs.
• Log aggregation: Collecting logs from many sources into one location.
• Loki: Log database from Grafana.
• Fluentd: Log collector and forwarder.
• Vector: Modern log and metrics collector.
• Grafana: Visualization and search interface.
• Parsing: Breaking log lines into structured fields.
• Retention: How long logs are kept.

Docker logging basics

By default, Docker uses the json-file driver:

docker logs containername

This stores logs in JSON files within the container directory. For production or larger deployments, an external logging driver often makes more sense.

Loki with Docker

Grafana Loki is a popular log database that pairs well with Prometheus and Grafana.

Compose example

services:
  loki:
    image: grafana/loki:latest
    container_name: loki
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yaml:/etc/loki/local-config.yaml
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml
    restart: unless-stopped

  promtail:
    image: grafana/promtail:latest
    container_name: promtail
    volumes:
      - /var/log:/var/log:ro
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - ./promtail-config.yaml:/etc/promtail/config.yml
    command: -config.file=/etc/promtail/config.yml
    restart: unless-stopped

volumes:
  loki-data:

Promtail configuration

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: docker
    static_configs:
      - targets:
          - localhost
        labels:
          job: docker
          __path__: /var/lib/docker/containers/*/*.log

Switching the logging driver to Loki

Docker can send logs directly to Loki. Install the loki logging driver first:

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

Then enable it globally:

vim /etc/docker/daemon.json

{
  "log-driver": "loki",
  "log-opts": {
    "loki-url": "http://localhost:3100/loki/api/v1/push",
    "loki-batch-size": "400"
  }
}

Restart the Docker daemon:

systemctl restart docker

Fluentd as a collector

Fluentd is an established tool that collects, filters, and forwards logs.

services:
  fluentd:
    image: fluent/fluentd:v1.16-debian
    container_name: fluentd
    volumes:
      - ./fluent.conf:/fluentd/etc/fluent.conf
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
    ports:
      - "24224:24224"
    restart: unless-stopped

fluent.conf:

<source>
  @type tail
  path /var/lib/docker/containers/*/*.log
  pos_file /var/log/fluent-docker.pos
  tag docker.*
  <parse>
    @type json
  </parse>
</source>

<match docker.**>
  @type loki
  url http://loki:3100
</match>

Vector as a modern alternative

Vector is fast and easy to configure:

services:
  vector:
    image: timberio/vector:0.35.X-debian
    container_name: vector
    volumes:
      - ./vector.toml:/etc/vector/vector.toml
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
    restart: unless-stopped

vector.toml:

[sources.docker_logs]
type = "docker_logs"

[sinks.loki]
type = "loki"
inputs = ["docker_logs"]
endpoint = "http://loki:3100"
labels.job = "docker"

Searching logs in Grafana

After adding Loki as a data source, query with LogQL:

{job="docker"} |= "error"

Parsing and structuring

Raw logs are often messy. Promtail, Fluentd, and Vector can parse logs to extract fields like timestamp, level, or message.

Example Promtail pipeline_stages:

pipeline_stages:
  - json:
      expressions:
        level: level
        message: msg
  - labels:
      level:

Log rotation and retention

• Docker’s json-file driver has built-in rotation.
• Loki requires configured retention policies.
• Archive or delete logs regularly.
• Monitor storage requirements.

Security

• Logs may contain sensitive data.
• Restrict access to your centralized logging system.
• Mask or anonymize personally identifiable information.
• Encrypt logs at rest.
• Protect audit logs separately.

Common pitfalls

• Missing log rotation: Disk fills up quickly.
• Wrong paths: Promtail can’t find logs.
• Parser errors: JSON logs aren’t parsed correctly.
• High costs: Long retention requires significant storage.
• Log volume: Use filters and labels to improve visibility.
• No authentication: Grafana and Loki exposed publicly.

Further reading and resources

• BotServ.de Docker monitoring
• BotServ.de Monitoring basics
• BotServ.de Docker security
• BotServ.de Docker backup

FAQ: centralize Docker logs

Do I need Loki? No, Elastic, Fluentd, Vector, or simple syslog forwarding are alternatives.

Can I collect logs without an external service? Yes, using local tools like journald, rsyslog, or file tailing.

How much storage do logs require? It depends on the number of containers, log level, and retention policy. Several GB per week is easy to reach.

Should I use Loki directly as a logging driver? Possible, but a separate collector like Promtail or Vector is more flexible.

Can I analyze logs from Ollama? Yes, Ollama generates logs like any other Docker container.

Sources and further reading

• Grafana Loki: https://grafana.com/oss/loki/
• Fluentd: https://www.fluentd.org/
• Vector: https://vector.dev/
• Docker logging: https://docs.docker.com/config/containers/logging/configure/

Summary: centralize Docker logs

Centralized logging is essential for keeping Docker containers manageable. Loki, Fluentd, and Vector are proven tools for collecting, parsing, and searching logs in Grafana. Log rotation, clear labels, a thoughtful retention strategy, and protection of sensitive data are all important. Setting up logs systematically from the start saves time during troubleshooting and provides better visibility into security incidents.