Sachin Katti: When the Human Becomes the AI System's Bottleneck

This lecture contains a thesis that sounds like a victory for infrastructure and almost like a warning for human work. Sachin Katti says OpenAI will have succeeded at compute when the human becomes the bottleneck. Agents will finish their steps quickly enough that we stop waiting and remain in flow.

In this Stanford MS&E435 “Economics of the AI Supercycle” session, Apoorv Agrawal talks with Sachin Katti. At the time of the recording Katti led Industrial Compute at OpenAI, covering compute for training and inference. He is now OpenAI's VP of Compute Strategy & GPT-Infra; previously, he served as Intel's CTO and taught at Stanford. This is not a neutral market overview but the view of a frontier-lab operator.

A strong part of the lecture is the agent as a new workload. A chatbot followed a short path: user → one inference call → answer An agent closes the loop: inference → search or database → tool call → VM or application → observe and assess the result → another inference call

Katti describes execution as a DAG, while the control logic may remain cyclic. Its nodes need different resources: » Accelerators for model inference » CPU-backed runtimes and VMs for tools and tests » Systems with plenty of memory for long context » Networking and orchestration to assemble the route That means model + runtime + accelerators + network + data center + power must be optimised as one system.

This extends two earlier channel themes: Dylan Patel's hardware-software co-design and the tree of agent calls that turns business processes into GPU demand. What is new is OpenAI's operator view: where to place each node and how whole-graph latency affects human flow. From this I would measure not only tokens per second, but time to first useful action, time to a verified artifact, and task cost. Katti's three levers are cheaper tokens, smarter tokens, and fewer tokens per outcome.

There are also very large numbers. Katti predicts that inference will consume more than 80% of compute, including synthetic data and part of post-training. He calls OpenAI's 30 GW target aspirational and roughly equates 1 GW with 500,000 GPUs. These are the speaker's estimates. Compute and revenue tripling together over three years shows correlation, but does not establish causation.

Here the tension appears. For Katti, the human as bottleneck marks success for compute infrastructure: today an agent spends minutes or hours completing a task with tools, the person switches tasks, and later has to reload the context. Faster AI is meant to restore interactivity.

The channel has already examined the same picture from the opposite side. In Your Attention Is the Bottleneck, the person becomes a dispatcher for many loops—assigning tasks, removing blockers, verifying results, and burning out from context switching. In the post about understanding as the new bottleneck, changes are produced faster than a team can update its mental model of the system. Cognitive debt accumulates: the code works, but nobody can responsibly own its evolution.

Making the agent faster does not remove that problem and may increase the arrival rate of decisions for the person. Speed up a production line without strengthening quality control, and work in progress, missed defects, and rework may all grow. The human bottleneck has at least four constraints: attention, understanding, verification, and responsibility. None scale with tokens per second.

Agentic UX should therefore be designed around more than a fast response. Agents need to batch questions, return compact context, expose evidence, tests, and uncertainty, and involve people where mistakes are expensive. The system should measure time to an understood and verified outcome, together with the number of steering points, context switches, and rework cycles.

I would watch 18:05–24:40 for the agentic graph and human flow, then 29:05–33:05 for latency. The lecture leaves a useful question: how many parallel agent loops can a person not merely steer, but actually understand and close responsibly?

#AI #Agents #Engineering #Architecture #Infrastructure #AI4SDLC