AI agent audit logs: authority, tool calls and evidence

What should an AI agent audit log explain?

An AI agent audit log should connect who delegated an action, what the agent was allowed to do, which tool request it made, and what evidence exists of the result. A chat transcript alone does not establish these facts. A tool can execute after a client times out; a retry can produce another side effect; a model's description of its own action can disagree with the receiving system.

This guide follows one synthetic operation: an agent changing a ticket's priority in a test service. It is a design walkthrough, not a report of a production deployment or an executed test. Use the general AI audit-trail guide for cryptographic record design and the AI governance guide for responsibilities and policy.

Connect authority, tool calls and results

Record times with explicit units, clock quality and a defined representation. Do not silently convert a nanosecond integer to a floating-point number or rewrite signed timestamps. VCP's numeric and serialization proposals are research drafts; their existence does not change existing signed records.

A timeout must remain an unknown outcome

1. Delegation
2. Policy decision
3. Recorded intent
4. API request
5. Timeout
6. Reconciliation

Suppose an operator authorizes the test agent to change ticket T-42 from normal to high priority. The grant is limited to that ticket and operation. Before requesting the change, the system records the operation identity, authority, target and arguments. The receiving API then times out.

The available evidence supports “request attempted; outcome unknown”. It does not yet support “change failed”, “change succeeded” or “nothing happened”. Preserve the original attempt. If the API provides a trustworthy operation-status lookup, retain its later response as separate evidence linked to the attempt. An idempotency key can help only if the service implements and honors its documented semantics.

Now suppose the local intent cannot be saved. A separately designed execution gate may reject the outbound call, if the system requires durable recording before execution. An audit trail by itself does not provide that gate. Test alternate credentials, direct API access and other bypass paths separately; a log of the intended route cannot prove that every route was controlled.

Review integrity and evidence coverage separately

Agree the system, period, operations, observers, exclusions and expected evidence before evaluation. A declared scope alone is not proof of complete observation. Identify the independent records or committed manifest from which an expected set can be derived, and how the recording source is known to have closed.

Retain failed, unsupported and unknown checks individually. If a valid item proves a requirement was violated, an unknown total does not erase that violation. A successful signature is not a blanket pass for the operation, observer or submitted evidence collection.

Keep secrets out of the audit export

Do not log API keys, session tokens or full confidential prompts by default. Retain the minimum context required for the review, keep sensitive payloads in access-controlled storage, and define retention and disclosure rules. A digest of a predictable value is not anonymization: an observer may guess values and compare hashes.

When redaction is required, preserve the original evidence under appropriate access controls and describe the derived view. Changing signed bytes invalidates the original signature for those bytes; a redacted export must not be presented as the original verified object. Record what was withheld, unavailable or not collected, without making claims about unobserved real-world actions.

A small evaluation checklist

• Define one operation and its allowed authority; name the observation boundary and direct-call paths.
• Compare a normal operation, a denied operation, a duplicate retry and a timed-out operation.
• Check modified evidence, a missing required item, an unavailable source and a notification with no delivery receipt.
• Keep the source input, trusted-key policy, verifier version, evaluation time and expected conclusions.
• Record who ran the checks, who managed the environment and what support the implementer provided.
• Separate permission to run the evaluation from permission to publish results, raw evidence, names and affiliations.

Download the editable evaluation checklist (text). This is a planning aid, not a conformance test suite.

What VSO currently provides

The declared-scope research verifier evaluates a research adapter with synthetic inputs. It is not a verifier for native VCP v1.2 event/AnchorRecord wire formats or proof of full profile conformance. Its offline evidence checks are separate from the mock execution gate and OS-isolation experiments, which study execution control under their own assumptions.

Start with VAP's scope, read the current research status and source links, and agree exactly what a reviewer will execute. No inference of production suitability or independent reproduction follows from an internal test result.