Skip to content
Primary sourcePodcastHarness longreadPrevious reviewNext reviewDCASHarness ScalingHarness-ZeroCompactionRLLego-RL code
Research Insights Made Simple
Research reviewResearch Insights Made Simple

The harness as a training environment

What goes into the weights? Lego-RL trains a model inside Claude Code, OpenCode and OpenHands

/ Research Insights Made Simple · Agent harnesses 20/24 · Training ground

Slide contents

  1. 1. The harness as a training environment

    What goes into the weights? Lego-RL trains a model inside Claude Code, OpenCode and OpenHands

  2. 2. Four stops: environment, signal, weights, transfer

    Lego-RL leads; four companion papers test its conclusions

  3. 3. AHE tunes the harness, Lego-RL the weights

    Atlas question: what belongs in the model, what in its environment

  4. 4. The harness is part of the environment

    Only the model learns; the harness is left untouched

  5. 5. A gain may not survive harness swaps

    KAT-Coder and its base Qwen3.6 in three harnesses

  6. 6. Fine-tuning ties a model to its harness

    Two bases and two fine-tuned models in four harnesses

  7. 7. Tokens are captured at the API boundary

    A proxy between the harness and the model server

  8. 8. The agent hunts for reward loopholes

    Walls go in the sandbox, not in the prompt

  9. 9. A sandbox crash is not zero reward

    Broken rollouts are excluded from the gradient

  10. 10. Only mid-difficulty tasks give a signal

    Groups of 0/8 and 8/8 give no gradient

  11. 11. Each harness is measured against its base

    Validation peak, one run, T = 0.7

  12. 12. Lego-RL adds 5.8–⁠9.4 points in every harness

    Qwen3.5-35B-A3B before and after, with Qwen3.6 and KAT-Coder

  13. 13. Rollouts, not training, are the bottleneck

    The agent itself takes 91.3% of trial time

  14. 14. The model learns to reread its edits

    Verification grows more than recovery

  15. 15. Context compaction can be trained too

    The same reward trains summaries and actions

  16. 16. Plans help when execution is trained

    Qwen3-Coder-30B-A3B in Claude Code, then transfer

  17. 17. Procedures distill more easily than knowledge

    Distilled Qwen3.5-9B in a simple harness vs its base in h*

  18. 18. Harness diversity beats harness count

    18 of 243 configurations, tested in Claude Code

  19. 19. Diversity costs the base harness almost nothing

    ClawGym-Bench, 30B model, GRPO

  20. 20. Lego-RL's conclusions rest on one run

    And what limits the four companion papers

  21. 21. Habits into weights, boundaries into harness

    Train where the model will work