Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel (AI column)
I saw it. 20minute report Kobie Crawford from Snorkel AI conference AI Engineer London (10 April 2026). This is a rare report that goes against the usual "new model bigger and smarter." The main idea is simple and inconvenient for the race of parameters: the capabilities of the model are limited not so much by the architecture and number of parameters, but by the quality of data and tasks on which it was trained. And for agents who go into tools, the bottleneck is often not reasoning, but tool discipline.
The most obvious example is financial analysis with tool use. According to Snorkel, Qwen3-4B’s reinforcement learning has outperformed Qwen3-235B with a difference in size. 1/60. On their benchmark. SnorkelFinance little model 59.7percentage 51.37percent (Figures from other Snorkel publications). Learning environment - FinQA: RL environment 290 expert questions 22 public companies, made with the rLLM team from UC Berkeley and posted in OpenEnv. According to the authors, one run cost less than $500 on 8xH100 and doubled pass@1.
What is Tool Discipline in Practice? Big generalists are good at reasoning, but poorly disciplined in tools: hallucinate column names, ignore circuit constraints, and generate SQL that returns a meaningless result. This is not treated by a larger model, but by habits - first examine the tables and diagram, validate the data before the next step, retrace and self-correct when mistaken. This is exactly what I learned through RL.
The report summarizes this into an idea that Snorkel calls. task fidelityNot all learning tasks are equally useful. They introduce four acceptance criteria.
- achievability (solveable)
- non-triviality (trivial)
- functional correctness (The solution really works.)
- reliability (reproducible) According to them, additional training on “good” tasks gave +6.2 p.p. to the share of passing tests, and on the "bad" - all +1.1: approximately five times the difference due to data quality alone.
Separately, I note that all this is a story based on the materials of Snorkel - a company that sells data curation, so its interest is direct. Benchmark's own, narrow domain (Financial tool use over SQL), and portability to other agent tasks is an open question.
But if you take their word for it, it seems that a small, well-trained model with a good environment and tool discipline may be cheaper and more reliable than a giant – especially where the agent works with circuits and APIs. For AI4SDLC, this is a direct signal: invest not only in the choice of the model, but also in the quality of tasks and in the guardrails on which the agent walks in the tools.
P.S. I already am. handler Everything I Learned Training Frontier Small Models - Maxime Labonne, Liquid AI
#AI #AI4SDLC #Engineering #Research #Agents #Data #Software