When does a team need Warp’s “software factory”? (Series #AI4SDLC)
Why would a team that already has Codex or Claude Code also need a “software factory”? Warp founder Zach Lloyd has a good example. When an agent misses an instruction from the repository, the easiest thing for a developer to do is remind it in the chat. How do you find out how many colleagues have already had to do the same thing?
In his AI Engineer talk, Lloyd proposes describing a task’s entire journey in code. An engineer opens a Linear ticket or writes in Slack. The system follows rules in the repository to assign investigation, spec writing, implementation and review. For each step the team specifies the model, tools and instructions. Run history is saved, so you can change a rule for everyone and try it out on familiar tasks to see what happens.
In the demo Lloyd opened Warp’s internal dashboard. The system had counted 37 cases in which agents started implementation despite missing relevant instructions. Alongside it he showed a pull request correcting the factory’s own rules. That’s what he calls self-improvement. Subsequent runs will tell you whether the fix helped.
There was also a chart of old Warp frontend tasks. Different models got the same work. One configuration scored around 93% at about $2 per task on average. Another scored around 95% for three times the price. A separate language model did the scoring. I’d open the tasks where those scores diverged. What exactly is the team buying for those extra four dollars?
At what scale does this make sense? In the Q&A Lloyd gave a ballpark of ten people on the engineering team. I’d also look at how many times a week people repeat the same work. A developer taking a task through to completion may do just fine with an agent, instructions in the repo and CI tests. If several engineers regularly catch the same mistake in review, you have somewhere to start. How many hours a month would one shared fix give back to the team? And how much time would maintaining and checking the rule take?
Tags: #AI4SDLC #Agents #Evals #PlatformEngineering