Does AGENTS.md Help the Agent? Live Research Insights

AGENTS.md / Does a context file help the agent? — Research Insights Made Simple

After you add AGENTS.md, the agent runs tests more often and reads more files. Does it solve more tasks?

In the new Research Insights Made Simple episode, live today at 13:15, I’m covering “Evaluating AGENTS.md”. On SWE-bench and CTXbench the authors tested how an instruction file affects the share of tasks solved and the cost of running the agent. Generated files increased the average cost per run by 20–23%, and the authors found no statistically significant gain in the share of tasks solved. I also wrote about the paper earlier.

Today we’ll discuss • How the agent works without a file, with instructions written by the team, and with a file generated by a model. • Why an overview of the repository didn’t help it find the right file faster in these experiments. • Which project rules are worth passing to an agent and how to test their value on your own tasks.

What in your AGENTS.md actually changes the agent’s behavior? Share an example in the comments.

#AI #Agents #Architecture #Data #Evals