Harness Engineering: Familiar Ideas in One Book (#Books)
O’Reilly has Harness Engineering by Nicole Koenigstein. It is an early release with just two chapters so far. After our Research Insights discussions of agent harnesses, a natural question arises: how much does the book add for someone who has already read the original articles?
My provisional answer: most of the conceptual foundation of these two chapters is already present in the material we have discussed. The book brings it together into a sequential explanation of how an agent system works. The quality of that synthesis still needs to be assessed.
Here is where the ideas overlap:
🔸 A model on its own is not yet a working agent. In Research Insights #33 on SWE-agent, we examined how tools, interfaces, and feedback change the same model’s results. The book opens with that same broad argument and illustrates it through an incident-response example.
🔸 The context runs out; the task continues. Episode #34 on long-running agents covered feature lists, progress logs, git, and handing work to the next session. The book’s second chapter gives this problem a more general vocabulary: task state must outlive a model invocation and a process.
🔸 The loop must be able to reject a result. In my Loop Engineering breakdown, we already followed the path from prompts and context to the harness, verification, and persistence. In the book, the model proposes a service rollback, while the execution code blocks the action without approval. A request to “follow the rules” becomes an enforceable constraint.
🔸 The engineer designs the environment and feedback. This overlaps with the OpenAI case study I examined for episode #35: knowledge is accessible to the agent, boundaries are enforced through checks, and failures feed back into the process.
This is a comparison of ideas. From two chapters, I cannot establish which specific articles the author used. But readers of our discussions will readily recognize the conceptual foundation.
The book does add something useful. Its second chapter treats checkpoints, event history, and records of external effects separately. Where to resume, what happened, and which action must not accidentally happen twice are three different questions. It also separates a task identifier from permission to access the task. That level of detail brings the overview closer to system design.
What I am still missing is an examination of when the recipes apply. In my longread on harness evolution, I specifically examined how its parts change alongside the model. In Anthropic’s follow-up, the author removes forced context resets and then sprints as he moves to stronger models. Access control remains the system’s responsibility; a particular planning procedure needs reassessment. And tomorrow’s episode #38, on An Empirical Study of Harness Design for Coding Agents, which I have already written about, offers an experimental approach to evaluating the different parts of a harness and their effect on the final result.
So far, the book mainly takes the perspective of improving the harness around a fixed model. Disabling checks against a recorded sequence of responses explains the mechanism well, but does not yet measure improvements on real tasks. The promised notebooks cannot be checked from the supplied version either: the chapters’ opening notes say the GitHub repository will become available later.
For now, I see a promising synthesis of familiar material. If you missed our series, the book may offer a useful entry point. If you followed the discussions, the first two chapters are more likely to help organize what you already know. From the next chapters, I want working code, difficult recovery cases, and an explanation of when another harness component justifies the added complexity.
#Books #AI4SDLC #Agents #Architecture #Engineering #ResearchInsights