Harness Engineering: the anatomy of coding agents (#AI4SDLC)

How much of a coding agent's work depends on the model, and how much on the environment around it? What interests me in the 80+ page paper "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents" is its attempt to examine that environment through source code. Paul Barbaste and colleagues from Wavestone AI Lab and Inclusive Brains study 11 agent systems, including Claude Code, Codex, OpenHands, and Pi, and separately the Omnigent meta-harness. Their snapshot is from July 2026. The authors examine agent architecture through source code; there is no shared experiment running all the systems on the same tasks here.

I would highlight four parts of the paper

🔸Harness anatomy Seven subsystems: the working loop, LLM integration, tools, memory and context, security, orchestration, and extensions. This map helps locate the differences between agents: how they apply edits, recover their work, and decide a task is complete.

🔸 Engineering mechanisms How to compress history without losing the goal; who should be allowed to write long-term memory; how to separate the stable and changing parts of the prompt for caching. Especially important is the shift from asking in text for a rule to be followed to enforcing it in executable code.

🔸Common patterns and trade-offs According to the authors' audit, the systems studied use neither general-purpose agent frameworks in their core runtime nor vector search over code. Instead of the latter, they search text, files, and program structure. A simple agent can get by with a small loop and a shell tool; product complexity adds security, recovery, and integrations.

🔸 The platform turn SDKs, skills, plugins, session transfer, and corporate policies turn the harness into an environment that other products are built on. The authors argue that competition is shifting toward ecosystems. Yet they also describe meta-harnesses that could make individual harnesses interchangeable.

Attached to this post is a PDF with my notes: questions in the margins about the authors' conclusions and ideas I would like to test. Here are a few points for discussion.

1️⃣ “What are the hypotheses?” — on the authors' conclusions (page 2) If the selected systems do not use LangChain or vector search, does that mean they are unnecessary? A technology's absence from a sample does not establish that it is useless. In their recommendations, the authors sometimes draw stronger conclusions than their research method supports.

2️⃣ “What should we measure besides success rate?” — on the benchmark results (page 58) My conjecture in the margins is that SWE-bench tasks may already be too easy for this kind of comparison. That still needs testing. But the cost of an accepted solution, the number of human interventions, and the ability to recover from failure certainly deserve separate measurement. The same percentage of tasks solved can conceal very different working experiences.

3️⃣ “Where are the network effects here?” — on the discussion of platform economics in harnesses (page 61) Accumulated memory and settings make it harder to switch products. More users, meanwhile, can attract extension developers and make a platform more useful. These are different mechanisms. Portable skills may strengthen the entire ecosystem while also reducing dependence on a particular harness.

4️⃣ On the co-evolution of the model and harness (pages 55-56) As the model improves, parts of the harness may be worth simplifying. A good subject for an experiment: which mechanisms actually help, and which continue to compensate for limitations that have already disappeared? We covered this in an earlier episode of Research Insights

Tomorrow, October 20, at 11:30 Moscow time, I will be live to discuss this whitepaper. We will go through the architecture, the debatable conclusions, and the notes in the PDF. Bring your questions.

#AI4SDLC #Agents #Architecture #Research #PlatformEngineering