NewAgent Observability is live.
Back to directory

Stage 4 of 5 · Agent observability & evals

AI agent evaluation and tracing

Watching the agents themselves: hallucination, drift, eval scores and what a session actually did.

This is stage 4 of 5 in a guide to monitoring when AI coding agents do the engineering work. This page covers agent observability and evals: what changes, what to measure, where your team is on three levels, and which tools cover it.

What changes when agents do this work

The three stages before this one watch software that agents produced. This stage watches the agents themselves. It is a different data shape: a trace here is a chain of prompts, tool calls and model responses, and the failure modes are semantic rather than operational. Nothing crashed. The answer was wrong.

Agent traces almost always sit in a different product from the rest of your telemetry. So when a slow database makes an agent slow, the symptom is in one tool and the cause is in another, and the person joining them up is doing it by timestamp and memory.

What to measure

Eval score over time, per prompt version

A score without a version attached tells you something changed but not what. Attach the prompt version and the number tells you what changed.

Turn count and context growth per session

Both correlate with quality decline, and both are cheap to record.

Tool-call failure and retry rates

An agent that retries a failing tool six times is describing a broken integration, not a hard problem.

Whether the trace joins to your infrastructure trace

This is the most useful property here, and the one most often missing. Shared trace IDs across both are what turn two tools into one investigation.

Three levels, from doing it by hand to letting the system do it

Find the level your team is at today. The next one is the step to take.

  1. 01Manual

    You read the transcripts

    Someone opens the session log when a user complains. This finds the bug you were told about and none of the ones you were not.

  2. 02Instrumented

    Sessions are traced and scored

    Prompts, tool calls and responses are captured as spans with a stable schema. Automated evaluation runs against them. Be aware that some vendors bill for the objects their own evaluation features create.

  3. 03Autonomous

    Regressions are caught before release

    Evals run in the pipeline against a held-out dataset, and a score drop blocks the deploy. At this point the code stage, the CI stage and this stage are the same loop.

Tools that cover this stage

These are the tools that cover this stage. Each label says whether the tool has an MCP server of its own. The ones with a directory page link to it.