Audit Logging: Tracking Agent Actions

What This Article Covers

• Why audit logging matters for AI agents
• What an audit trail is and what information it contains
• How to generate logs in a structured way with Python
• How to store, rotate, and analyze logs
• How audit logging works alongside human approval and guardrails

Introduction

AI agents make decisions and execute actions. They read files, run commands, and call APIs. When something goes wrong, you need to know what the agent did, who triggered it, and when. That’s where audit logging comes in.

Audit logging records relevant events in a way that’s tamper-evident and easy to trace. This article shows you how to build an audit log for AI agents using Python, since many agents are written in it.

If you need a refresher on agent system basics, check out Agent Systems. For safe operation practices, see Safe Operation.

Why Audit Logging Matters

AI agents often have far-reaching capabilities. If one deletes a file or makes a wrong API call, you need a complete record of what happened. Audit logs serve several purposes:

• Debugging: trace what led to a problem
• Security: detect unauthorized actions
• Compliance: provide evidence for auditors and regulators
• Trust: let users verify what the agent actually did
• Learning: identify patterns in agent behavior to improve it

Audit logging isn’t a substitute for protective measures like sandboxes or guardrails, but it’s a critical piece. It shows you what happened when something goes wrong.

Understanding Audit Trails

An audit trail is a time-ordered list of events. Each event typically contains these fields:

• Timestamp: when did it happen?
• Actor: who or what triggered the action?
• Action: what was done?
• Resource: what object was acted upon?
• Status: did the action succeed or fail?
• Metadata: additional details like IP address, request ID, or parameters

Unlike generic application logs, audit logging focuses on business or security events. It’s meant not just for developers but also for administrators and auditors.

A typical log entry in JSON looks like this:

{
  "timestamp": "2025-08-24T14:30:00Z",
  "actor": "agent-007",
  "action": "file_delete",
  "resource": "/data/tmp/ergebnis.txt",
  "status": "success",
  "metadata": {
    "user": "anna",
    "ip": "192.168.1.42"
  }
}

This structure is machine-readable and straightforward to analyze.

Who This Article Is For

This article is for beginners building or running their own AI agents. You need basic Python knowledge and should be comfortable with files and JSON. If you’re on a Linux system like Ubuntu, find foundational material in Ubuntu.

Read this if you:

• Develop AI agents that access tools
• Need to make agent actions auditable
• Want to store logs securely long term
• Must meet compliance or security requirements

Key Terms

Term	Definition
Audit Trail	A complete record of security-relevant events.
Log	A single log entry.
Timestamp	The point in time when an event occurred.
Actor	The initiator of an action, such as a user or agent.
Action	The operation performed, for example file_read or api_call.
Resource	The affected object, such as a file or API endpoint.
Rotation	The regular archiving and removal of old log files.
Retention	The period for which logs are kept.
Compliance	Adherence to regulations and standards.
Immutable Log	A log that cannot be modified after being written.

A Simple Audit Log in Python

Python includes a built-in logging module, but for audit logs you should use your own structure that’s machine-readable. Here’s a minimal example:

import json
import datetime
import pathlib

LOG_FILE = pathlib.Path("/var/log/ki-agent/audit.log")
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)

def audit_log(actor, action, resource, status, metadata=None):
    entry = {
        "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
        "actor": actor,
        "action": action,
        "resource": resource,
        "status": status,
        "metadata": metadata or {}
    }
    with LOG_FILE.open("a", encoding="utf-8") as f:
        f.write(json.dumps(entry) + "\n")

# Example call
audit_log(
    actor="agent-007",
    action="file_read",
    resource="/data/dokumente/bericht.pdf",
    status="success",
    metadata={"user": "anna"}
)

The advantage of this approach is simplicity and no external dependencies. The downside is that the log file is readable and modifiable in plain text. For production setups, you should add additional measures.

Storage and Rotation

Logs grow quickly, especially if an agent performs many actions per minute. You should rotate them regularly. Python provides the logging.handlers.RotatingFileHandler module:

import logging
import json
import datetime

class JsonFormatter(logging.Formatter):
    def format(self, record):
        data = {
            "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
            "level": record.levelname,
            "message": record.getMessage(),
            "extra": record.__dict__.get("audit", {})
        }
        return json.dumps(data)

logger = logging.getLogger("audit")
handler = logging.handlers.RotatingFileHandler(
    "/var/log/ki-agent/audit.log",
    maxBytes=10_000_000,
    backupCount=5
)
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)
logger.setLevel(logging.INFO)

logger.info("file_read", extra={"audit": {"actor": "agent-007", "resource": "/data/dokumente/bericht.pdf"}})

This handler creates new files once the current one reaches 10 megabytes. Old files are archived. Once written, logs should be stored on a separate server or write-once medium to prevent tampering.

Analyzing Audit Logs

Logs are only useful if you actually analyze them. Common approaches include:

• Searching for error patterns with grep or jq
• Setting up a log collector like Promtail, Fluent Bit, or Filebeat
• Visualizing them in Grafana or a SIEM tool
• Triggering automatic alerts on suspicious actions

With the jq command-line tool you can run simple queries:

jq 'select(.action == "file_delete")' /var/log/ki-agent/audit.log

This command shows all delete operations. For production environments, a SIEM system that automatically detects patterns and sends alerts is recommended.

Compliance and Retention

Many industries have requirements for log retention. In Germany, GDPR applies when processing personal data. ISO 27001 also mandates documented audit trails.

Key compliance considerations:

• Define a clear retention policy.
• Store logs in a secure, immutable location.
• Ensure only authorized personnel have access.
• Document which events are logged.
• Regularly test the recovery of archived logs.

If you log personal data, follow GDPR rules. Log only what’s necessary. Pseudonymization helps balance data protection with auditability.

Audit Logging and Human Approval

Audit logging becomes especially powerful when combined with human approval. Each approval is recorded as a separate event. This way, you can always prove that a person authorized a critical action.

An example entry:

{
  "timestamp": "2025-08-24T14:35:00Z",
  "actor": "anna",
  "action": "approve",
  "resource": "agent-007:file_delete",
  "status": "approved",
  "metadata": {
    "reason": "bereinigung temporaerer dateien"
  }
}

This combination builds trust and helps you identify mistakes. If an agent takes the wrong action despite approval, the log shows exactly who approved it and when.

Common Pitfalls

1. Unstructured logs: Plain-text logs are hard to parse. Use JSON or another structured format.
2. No timezone: Timestamps without timezone information cause confusion during analysis. Use UTC and a consistent format.
3. Missing context: A log without a request ID or user information makes it hard to trace operations.
4. Logs on the same server as the agent: If the agent is compromised, it can delete the logs. Store them externally.
5. No rotation: Log files grow indefinitely and fill the disk.
6. No access control: Not every user should be able to read all logs. Restrict permissions.
7. Sensitive data in logs: Avoid passwords, API keys, or personal data in log entries.
8. Logs never reviewed: Logs that are never analyzed don’t help with security.

Hardware, Costs, and Security

Audit logging requires minimal additional hardware. The logs themselves need storage space. A modern system with a few gigabytes of free storage is enough for small setups. For long-term retention, consider a central log server or object storage.

Costs mainly come from storage and backups. Open-source tools like Grafana Loki, Elasticsearch, and Graylog can be run for free. Cloud solutions simplify operations but come with a price tag.

From a security perspective, treat logs as sensitive. Whoever controls the logs can hide attacks. Store them in a separate location and limit access. On a Linux system, you can set permissions so only a specific user can read them. Ubuntu Basics will help you get started.

Further Reading

• Secure Operations
• Agent Security
• Human Approval
• Configuring Guardrails
• Agent Systems
• Ubuntu Server Setup

FAQ

What is audit logging?

Audit logging is the targeted recording of security-relevant events for later review and accountability.

What’s the difference between an audit log and an error log?

An error log records technical problems. An audit log documents actions, identities, and resources deliberately.

Why should I use JSON logs?

JSON is machine-readable and works well with tools like jq or SIEM systems for analysis.

How long should I keep logs?

That depends on your requirements. Often 30 to 90 days is sufficient. For compliance, retention of a year or longer may be necessary.

Can I generate audit logs with Python?

Yes. Python is excellent for this. The built-in logging module and JSON are enough to get started.

What is an immutable log?

An immutable log cannot be changed or deleted after creation. This makes tampering much harder.

Should I store logs on the same server?

No. For production setups, send logs to a separate server or object storage. Otherwise, a compromised agent can erase traces.

What is log rotation?

Log rotation regularly switches the active log file and archives old files. This prevents a single file from growing indefinitely.

What information belongs in an audit log?

At minimum: timestamp, actor, action, resource, and status. Optionally: IP address, request ID, client version, and approvals.

How do I analyze logs?

Use tools like jq, grep, or a SIEM. For visual dashboards, Grafana or Kibana work well.

Can audit logging cause data protection issues?

Yes, if you log personal data. Avoid unnecessary data and pseudonymize where possible.

What is a SIEM?

SIEM stands for Security Information and Event Management. It collects, correlates, and alerts on security events.

Sources

1. Python Logging Documentation - https://docs.python.org/3/library/logging.html
2. OWASP Logging Cheat Sheet - https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
3. NIST SP 800-92 Guide to Computer Security Log Management - https://csrc.nist.gov/publications/detail/sp/800-92/final
4. ISO 27001 Audit Logging Requirements - https://www.iso.org/standard/27001