[2/2] John Schulman on dead ends, scaling RL, and building research institutions (AI column)

Continue. story I want to talk about theses related to the leadership of research teams, as well as highlight the main ideas that I found useful.

7How to build research teams: hands-on vs hands-off Interesting Reflections on the Archetypes of Research Leaders

• Hands-on (Coding, reading code, deep technical feedback) - it is focused on projects to achieve goals, can steer teams with beginners, works in the execution mode of this project.
• Hands-off (Works as a consultant, gives career advice, motivates) Works more in the exploratory research format and the team should have senior people

But the key to adaptability is that research is changing and what worked. 7 Years ago, it might not work now. Early OpenAI was a peacetime (exploratory)new labs (Thinking Machines) - wartime (We need to catch up with SOTA. (state of the art) Build a research culture)

8A Day in the Life of a Researcher Shulman works in coffee shops. (noise + laptop + ideas), then execution: code, review docs/plots/code of colleagues. Using Cursor and Claude Code for coding, GPT-5 Pro for literature/detailing ideas/feedback on writing. Advice from John: Keep a research notebook (More important with LLM for context)

9Slowing progress: 10x researchers 10x ideas Za. 10 Over the years, the number of ML-researchers has increased 10-100x, but "consequential" ideas (scaling laws) They didn't accelerate. Reasons:

• Low results are exhausted
• Upstairs. (More baselines, more experiments) Type shift: used to be risky people, now more traditional (engineering > research taste) But quality is growing: old papers (70–90she) They had little rigor.

Thinking Machines: Philosophy and Tinker Shulman's new lab. The philosophy is not only to catch up with SOTA, but also to pump the research muscle and lay crops. This lab already has a Tinker tool - low-level API for fine tuning (Python scripts without GPU hassle)For advanced ML engineers. The plan is to evolve into a full-stack product for less advanced engineers.

I found it interesting from different angles.

1. For leaders, there are ideas about switching between hands-on and hands-off depending on the phase of the project and the maturity of people.
2. For engineers, the explanation is that AI labs are not magic—there are many dead ends, biases, and that culture is more important than processes.
3. For researchers, there are thoughts about the return of RL (verifiers, multi-agent)The movement towards continuous learning, and the importance of new research, even in the context of laws of scaling (More data and more GPUs are better results)

#Engineering #AI #Metrics #Software #Architecture #RnD #ML