Agent Audit/Blog/Why existing logs don't work
ESSAY · 7 MIN7 June 2026 · By Hak Turkel

Why Datadog and Splunk can't produce an Article 12 pack.

An engineer's-eye view of the structural reasons general-purpose observability and SIEM tooling don't capture the right shape of evidence for AI agent compliance.

The most common pushback we hear in early conversations is some version of: "we already log everything to Datadog (or Splunk, or Elastic). Why do we need a separate tool?" The honest answer requires understanding what a regulator-acceptable Article 12 pack actually needs to contain — and the structural mismatches between that and how general-purpose observability tools work.

What Datadog and Splunk are good at.

Both are excellent at ingesting high-volume HTTP and application traces, indexing them for ad-hoc search, and rendering operational dashboards for SRE teams. Datadog's APM gives engineers detailed visibility into service-to-service calls. Splunk's strength is at the security side: correlating events across diverse sources for incident detection.

Both have been load-tested at hyperscale and have mature ecosystems. Neither is the wrong choice for what they were built for.

What an Article 12 pack needs.

The pack is built around a specific question: "What did this AI agent do, on whose behalf and with what authority, with which data, between dates X and Y, and prove you can't have edited — or silently omitted — any of the record."

Six structural requirements follow:

1. Per-agent-action records, not per-HTTP-request.

When an agent makes a decision, it might involve one LLM call, several tool calls, a sub-agent spawn, a data lookup, and a final decision. In Datadog / Splunk, these surface as separate traces or events. The agent's reasoning — what tool was chosen, what classification was determined, what confidence the decision had — usually doesn't surface at all, because the observability layer doesn't know the agent's internal model.

2. Data classification at the action.

A regulator wants to know which actions touched PII, PCI, financial, biometric, or other classified data. This classification is a property of the data being accessed, decided at the application layer, not the transport layer. SIEM tools don't have that context.

3. Who authorised the action, at the instant it ran.

"On whose behalf" is not a name in a config file. It is the authority the agent actually held at the moment it acted — the scope it was granted, the delegation chain that scope descended from, and whether that grant was still in force and un-revoked when the action ran. Permissions change through a session; proving authorisation means pinning it then, at capture, not reconstructing it later from whatever the access system says today. General observability captures the request, not the authority behind it. (This is an evidentiary record of the authority in force at the time — not a legal opinion that the action was permitted.)

4. PII redaction at the source.

Article 12 logs are retained for years and made available to authorities. Storing raw PII in your log database is a privacy nightmare. The redaction must happen before the log is created, not as a post-processing step. Datadog and Splunk are designed to ingest everything and analyse it later — the opposite pattern.

5. Cryptographic integrity — and completeness.

Standard application logs are append-only by convention, not by guarantee. A database administrator with the right credentials can edit them. A regulator-grade record requires mathematically detectable tamper evidence — typically hash-chaining or Merkle-tree structures. Neither Datadog nor Splunk provides this as a property of the log itself.

Integrity is only half of it. Proving nothing was altered is not the same as proving nothing was hidden: an actor who cannot edit a record can still drop an inconvenient entry or backdate a period boundary. A defensible record has to be verifiable on both axes — nothing changed and no gaps, no silent omissions, no backdating. That completeness guarantee is demonstrated on a live record in the interactive proof.

6. Output format the regulator accepts.

An auditor or regulator does not want to log into your Datadog instance. They want a PDF with a defined structure: system inventory, operational summary, material event log, integrity proof, manifest, sign-off block. Building that PDF from Datadog data is weeks of work, every quarter.

Why a thin layer on top of Datadog isn't the answer either.

You could in theory build a thin transformer that ingests Datadog traces and produces an Article 12 pack. The problem is properties 1, 4 and 5 above — Datadog doesn't capture per-agent-action records, doesn't redact at source, and doesn't provide cryptographic integrity or completeness. A downstream transformer cannot retrofit these properties because the underlying data is the wrong shape.

The right answer is to capture the data correctly at the SDK boundary, ship it to a system designed for cryptographic integrity and regulator- readable export, and integrate that system with Datadog as a complement — not instead of it.

Where Agent Audit and Datadog overlap.

We integrate cleanly with Datadog (and Splunk, Sentinel, Elastic) via webhook. Receipts flow to both: Datadog for SRE / debugging context, Agent Audit for compliance evidence. Each system gets what it's good at doing without forcing the other to do something it wasn't designed for.

The bottom line.

The wrong way to think about Agent Audit is "alternative to Datadog." The right way is "complementary primitive in the compliance evidence layer, where Datadog wasn't designed to play."

Want to see how the two integrate? Get in touch →