Harness Design for Long-running Application Development
Episode participants
A solo episode without invited guests.
What we discussed on the recording
In episode 36, Alexander Polomodov reviews Anthropic Labs engineer Prithvi Rajasekaran’s post “Harness design for long-running application development.” The author combines two goals: a high-quality frontend and a working application from a single request. Context problems and generous self-evaluation require different mechanisms. Inspiration from GANs suggests a separate critic; this is an analogy for dividing roles, without training model weights.
For design, the evaluator receives four criteria and scored examples, inspects the live page through Playwright MCP, and returns criticism. The last iteration is not always the best. For a full application, a planner creates the specification, while the generator and evaluator agree on a sprint contract before implementation. A direct Opus 4.5 run on a retro game maker took about twenty minutes and cost nine dollars, but core features failed. The full harness spent six hours and two hundred dollars to produce a more complete result. Different budgets mean that the example cannot establish universal economic efficiency.
The checker also needs tuning: otherwise it can find a bug and persuade itself that the problem is unimportant. After moving to Opus 4.6, the author initially changes too many components and loses the previous quality, then tests their contributions one at a time. Sprints disappear, the planner remains, and the evaluator helps near the model’s capability limits. A digital audio workstation reveals the feedback boundary: the checker can inspect the interface, APIs, and database, but cannot hear audio.
The host closes by considering harness evolution and answering a question about external risk checks. Compensating for model weaknesses and controlling dangerous actions serve different purposes: a stronger worker does not remove the need to limit consequences. The epilogue mentions Managed Agents’ event log outside the harness. Chapter timestamps are based on the episode’s audio version and may differ slightly from YouTube.