Demystifying evals for AI agents: Evaluation Structure and Margin Notes (#AI4SDLC)

Continuing the agent harness theme. In “Demystifying evals for AI agents”, the Anthropic team examines how to tell whether an agent has improved after another change. The article was published on January 9, 2026; its authors are Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares, and Jiri De Jonghe. This is an engineering overview of experience from Anthropic and its customers, with recommendations based on individual practical cases.

First, the article’s main sections.

🔸 Eval structure A task, an individual trial, a grader, a trajectory, and the final state of the environment. An agent saying “The ticket is booked” does not mean that a reservation has appeared in the database. There are also two harnesses: the agent harness enables the model to work with tools, while the evaluation harness runs trials and collects scores. We are evaluating the model together with its harness.

🔸 How to grade With code, an LLM judge, and humans. Tests are a natural fit for coding agents; research agents need checks for source support and completeness; conversational agents need evaluation of both outcomes and interaction quality. LLM judgments need to be calibrated against human assessments.

🔸 What to measure Capability evals show what the agent still needs to learn; regression evals protect capabilities it has already acquired. Meanwhile, pass@k and pass^k answer different questions: was there at least one success in k attempts, or did all k attempts succeed?

🔸 How to build and maintain evals The authors recommend starting with 20–50 real cases, defining success explicitly, isolating trials, and reading traces. The grader itself can be wrong. The suite then needs updating, and the picture needs to be supplemented with production monitoring, feedback, and human evaluation.

Now for my notes as I read, which are in the attached PDF.

1️⃣ Familiar engineering Isolated trials and an environment resembling production recall established testing practices; eval-driven development recalls TDD. Even the organizational setup is familiar: a dedicated team maintains the infrastructure, while product teams contribute tasks. Hello, platform teams and stream-aligned teams. Here, though, a green result on one run does not promise another green result on the next.

2️⃣ Expectations need to be defined too Early on, there may be no users yet, and quality criteria are still taking shape. Evals help establish that bar and maintain it. This is especially useful with frequent model updates: you have your own set of checks to inform the decision to switch.

3️⃣ The DOM and screenshots I used to think screenshots were more expensive and working through the DOM was cheaper. The authors give an example that goes the other way: extracting the entire DOM can consume more tokens. Here I would want to measure the volume of data transferred, latency, and the number of actions separately — this does not establish a universal winner.

4️⃣ OR, AND, and independence It is useful to think of the pass@k metric as a logical OR and pass^k as a logical AND. But calculating p^k requires independent attempts with the same probability of success. With p = 0.75, three consecutive successes have a probability of roughly 42% only under those assumptions. A shared environment, leftover files, or memory carried between attempts can violate them.

5️⃣Trajectories The user simulator reminded me of my breakdowns of “SWE-Together” and “SWE-Interact”. The article places two recommendations side by side: evaluate the process, and avoid prescribing a specific sequence of steps to the agent. I would reconcile them by checking properties of the process: an agent could start a bug fix with a test or by reading the code, but a claim that the fix was “verified” must refer to the final version. Access permissions and restrictions on writing outside the workspace should be enforced by the harness itself: passing an eval does not replace blocking a prohibited action.

#AI4SDLC #Agents #Evals #Research #Engineering #PlatformEngineering