Harness Engineering Is Not Enough: Why Software Factories Fail (#AI4SDLC)
Dex Horthy is an excellent speaker, and I watched his latest talk. Previously, the HumanLayer co-founder proposed addressing AI slop through context engineering and the Research → Plan → Implement loop (the talk “No Vibes Allowed”). In my review of “The New SDLC,” we arrived at the formula Agent = Model + Harness. In this new 19-minute keynote from AI Engineer World's Fair 2026, Dex adds an important qualification: a good harness improves execution dramatically, but does not by itself teach a model to preserve architectural quality over time.
This is an interesting and, at times, sobering assessment of the current state of AI-assisted development. It cannot be called an independent review, however. Horthy is a co-founder of HumanLayer, a company that sells an AI IDE and tools for collaboration between people and coding agents. In the written version, Dex himself warns of possible bias, and at the end of the talk he makes a direct product pitch. This is a substantive report from a practitioner-provider, not a neutral market study. Still, he does not reduce the method to his own product: for document review, he explicitly mentions GitHub, Notion, and Plannotator.
The starting point is a lights-off software factory. One agent writes code, other agents review changes and run regression tests, incidents and user feedback enter the queue immediately, and people stop reading the changes. According to Dex, HumanLayer tried exactly this approach in July 2025. A few months later, the team encountered a complex problem the agents could not fix: during a site-outage incident, people had to understand a codebase whose evolution they had stopped following. This is Horthy's retrospective account of his own team without independent data, but it is almost a textbook example of the loss of understanding from my Loop Engineering review.
Why, in Horthy's view, does another loop not solve this? Using SWE-bench Multilingual, he illustrates test-based evaluation: was the issue fixed, did the new tests pass, and did the old ones remain green? His hypothesis is that a similar reward signal does not penalize a model for an unnecessary try/catch, a dubious cast, or shotgun surgery that spreads one change throughout the system. The cost of poor program design appears months later, while a short training episode simply cannot see it.
Importantly, Dex is candid about the boundary of his argument: he cannot prove that maintainability degrades, because there is no good benchmark for it yet. Long-horizon evals are emerging and Frontier Code uses multi-PR tasks, but in his assessment, model-as-a-judge still does not turn architectural quality into a reliably measurable signal.
The author's proposal is not to abandon agents, but to “turn the lights on” and move human judgment to the beginning of the process:
1️⃣ Product requirements: what problem are we solving, for whom, and how will we know the result is useful? 2️⃣ System architecture: component contracts, data models, and constraints; 3️⃣ Program design: types, method signatures, code layout, and call graphs; 4️⃣ Vertical slices: implementation order and testable end-to-end slices instead of one enormous horizontal plan.
Small tasks can still go directly to an agent. For larger ones, Dex proposes agreeing on the decisions before implementation, then reading the code and checking the result in parts. His practical estimate is that roughly 30 minutes of up-front alignment can save hours of review; that is the team's experience, not an experimental result.
The commercial interest is again easy to see: the recipe clearly echoes HumanLayer's workflow—Questions → Research → Design → Structure → Plan → Implement. That does not erase the useful boundary. A harness helps an agent execute a stated task more effectively; it does not replace architectural taste, a mental model of the system, or responsibility for what the code will look like in six months.
#AI4SDLC #AI #Agents #Architecture #Evals #Engineering