How to monitor AI agents in production

AI agent observability is a different problem from service monitoring, and most teams discover this the hard way: every dashboard is green, no alert has fired, and the agent has been doing the wrong thing confidently for three days. A 200 response tells you the agent replied — not that it was right.

Trace the run, not the request

The unit of work is a run, not an HTTP call. One run may span a dozen model calls, several tool invocations, an approval that waits ten minutes for a human, and a retry after a rate limit.

So give every run an id and propagate it everywhere — model calls, tool calls, database writes, queue messages, log lines. When someone reports “the agent did something weird this morning,” a run id is the difference between an answer in thirty seconds and an afternoon of grep.

Within a run, span these:

• Model calls — prompt, response, model, tokens, latency
• Tool calls — name, arguments, result, duration, error
• Approvals — what was proposed, who decided, how long it waited
• Retries — which call, which attempt, why

Standard OpenTelemetry spans work fine for the shape. What’s specific to agents is the payloads, which is why an LLM-focused tracing layer on top earns its keep.

The four metrics that matter

Ignore the vanity ones. These four tell you whether the system is healthy.

Run success rate. Not HTTP success — task success. Did the run accomplish what it was asked to do? This needs a definition per task type, which is annoying and unavoidable. Without it you’re measuring whether your code crashed, not whether it worked.

Steps per run. The single best early warning. When average steps climbs, the agent is flailing — retrying, re-reading, going in circles. It usually moves before success rate does, so it’s the leading indicator.

Approval rate and approval latency. What share of proposed actions get approved, and how long they wait. A falling approval rate means the agent’s judgement is drifting. Rising latency means you’re gating too much and users are getting numb — which ends in rubber-stamping, and a gate that nobody reads is not a gate.

Cost per run. Track it per task type, not in aggregate. Aggregate cost hides the one workflow that quietly costs forty times the others.

Log the decision, not just the outcome

Standard logging captures what happened. For agents you need what nearly happened.

When the agent selects a tool, record the alternatives it considered and rejected. When it decides an action needs approval, record the rule that triggered it. When it stops, record why — done, blocked, out of budget, error.

This is what makes debugging tractable. “The agent didn’t send the email” has a dozen possible causes; “the agent chose draft over send because the recipient wasn’t in the allowlist” has exactly one, and you can fix it. The tool selection path is worth tracing in particular, since that’s where most confusing behaviour originates.

Redact at the boundary

Agent traces contain email bodies, documents and customer data by nature. Your tracing backend is now a copy of your most sensitive data, usually with weaker access controls than the primary store.

Decide deliberately. Redact at the point of capture, not in the backend — anything else means the raw data has already crossed the wire. Practical defaults: hash or truncate message bodies, keep structure and metadata, never record credentials or tokens in any span attribute, and set a shorter retention on traces than on your primary data.

Alert on shape, not on errors

Error-rate alerting misses the failure mode that matters, because a confidently wrong agent produces no errors at all.

Alert on these instead:

1. Steps per run above baseline — flailing
2. Approval rate falling sharply — judgement drift
3. Same tool failing repeatedly across runs — a broken integration, not a bad run
4. Cost per run above baseline — a loop, or a model change
5. Runs stuck in approval beyond a threshold — a human bottleneck

Baselines beat thresholds here. “20% above the seven-day median” survives growth; “more than 8 steps” fires constantly the moment a new task type ships.

Sample intelligently

Tracing every run at full payload is expensive, and cutting to a flat 1% throws away the runs you’d most want.

Keep 100% of runs that errored, that hit an approval, that exceeded a cost or step threshold, or that a user gave feedback on. Sample the boring successful ones at a few percent. You keep the interesting tail at a fraction of the storage.

The rollout order

1. Run ids propagated everywhere. Do this first; everything depends on it.
2. Spans for model and tool calls.
3. The four metrics on one dashboard.
4. Decision logging — alternatives and reasons.
5. Redaction at capture, with its own retention.
6. Baseline alerts on steps, approval rate, cost.
7. Tail-biased sampling once volume makes cost real.

Steps 1–3 take a day and catch most of what you’ll actually hit. Step 4 is what turns a three-hour investigation into a three-minute one.

See also: proving an agent doesn’t do things and where the money actually goes.