No Vibes Allowed: Solving Hard Problems in Complex Codebases (AI column)
Interesting. speech Dex Horthy, founder of HumanLayer, a company that develops tools for AI-assisted development. His previous report "12 Factor Agents: Patterns of reliable LLM applications" (see mine analysis) June 2025 It was one of the most popular at the conference. He is credited with popularizing the term "context engineering".
Stanford Study Reveals Unpleasant Truth About AI Coding Tools (“Does AI Actually Boost Developer Productivity?” by Yegor Denisov-Blanch told) Most of the “additional code” written with AI is a reworking of the slop that was written before. Agents work great on new projects, but in large legacy codebases they often make developers less productive.
To solve these problems, the author talks about context engineering. LLM is a stateless system.. The only way to get the best result is to give the best context to the entry. With each call, the model chooses the next step based solely on what is in the current context. - "Dumb Zone". The context window has a practical limit. After ~40The percentage of completion begins to degrade the quality of responses. If you have a lot of MCP tools that fill the context with JSONs and UUIDs, you are constantly working in a dumb zone. Methodology: Research → Plan → Implement. Instead of the naive “ask → get slop → fight → ask again” approach, Dex’s team uses a frequent deliberate compaction of context: Research - understanding the system, finding the right files. The result is compressed into markdown with specific files and line numbers. Plan is a detailed plan with code snippets of what will be changed. The more specific the plan, the more reliable the execution.
- Implement - execution of the plan. If the plan is good, even a "dumb" model will cope. Finally, the author talks about the practical results of the type: 7 The hours of the Saturday session were sent 35,000 lines of code in the BAML project (300k LOC Rust) - It was usually a job. 1-2 weekdays
Practical advice from the author Sub-agents are not for roles, but for context control.. Do not create a front-end agent or a back-end agent. Use sub-agents to isolate heavy codebase reading operations, returning only a compressed result. Progressive Context Disclosureah Instead of one huge documentation file at the root of the repository, place contextual files on each level, loading only the relevant ones. On-demand condensed context is better than static documentation. The documentation is outdated and lying. Code is the source of truth. Generate research documents on the fly from real code. - Trajectory matters. If you 5 Once the model was scolded in one context - it "learned" that the next step = error + swearing. Better start a new context. Cultural change must come from above.. If you are a technical leader, choose one tool and stuff your practice. Do not jump between Claude Code, Cursor and Codex.
The main conclusion of the speech is about this
AI cannot replace thinking. It can only amplify the thinking you have done - or the lack of thinking you have done.
#Engineering #AI #Metrics #Software #DevEx #Productivity