AGENTS.md: Does a Context File Help the Agent?

Alexander Polomodov starts with the experimental design. SWE-bench Lite compares runs with and without a file produced by /init. CTXbench adds developer-written instructions from 12 repositories. Tests determine whether the fix succeeds.

Generated files raise costs on both benchmarks. On CTXbench, developer-written files beat generated ones, while the comparison with no file yields no statistically significant gain. Traces show agents using the named tools and running checks. The repository overview does not shorten the path to files in the original patch.

The host uses an export bug to explain a limitation of the experiment. If the issue already specifies the edge cases and constraints, the file contributes less information. He proposes keeping rules that are hard to infer and running the same tasks with and without the file, with separate checks for architectural requirements.

The Q&A covers rebuilding an implementation from requirements and tests, maintenance rules, and agent memory. A review of recurring errors can produce a new instruction, a skill, or an automated check. Timestamps follow the Podster audio.