McKinsey on AI Agent Economics: Measure Outcomes, Not Tokens (Category #Management)

I read a McKinsey interview with David Tepper, CEO of the AI FinOps platform Pay-i, about the economics of agentic systems. What makes the conversation interesting is that it does not look outward from IT through models, context, evals, and tool calls. It looks inward from the perspective of the CIO, CFO, and process owner: which agents deserve a budget, and how can that be demonstrated?

The central idea is that token price says almost nothing about a system's value. Tokens are a bill, not an outcome. An agent may make hundreds of calls, take a different route through the same task, and consume 30 times more or fewer resources. The useful unit is the cost of a completed and accepted business task.

McKinsey distinguishes three things that the market routinely calls agents: - Workflow — a conventional process with AI at selected steps; - Pipeline — a predetermined sequence of calls; - A genuine agent — a system that chooses its own tools, order of operations, and run time. The architecture bounds costs in the first two cases. With an agent, cost becomes a long-tailed distribution. Autonomy should not be purchased just in case; it has to pay for its own uncertainty.

The agentic-programming studies cited in the article offer striking estimates: up to 1,000 times more tokens than an ordinary coding chat, with about 59% of spend going to verification and corrections (1 and 2). These are not universal business benchmarks, but the mechanism matters: making an output acceptable may be the most expensive part. The author proposes this test for deciding when an agent is worthwhile: P(success) > T(verify) / T(do). If a person needs two hours to perform a task and six minutes to verify an agent's output, the agent needs only about a 5% success rate to reach parity. But that holds only when failure has no external effects. A bad document can be discarded; a false promise to a customer creates recovery costs. Tepper calls verification and rework the agency tax.

The interviewee's business perspective is especially clear here: - Technical metrics — latency, error rate, and cost per call — are not enough. - Before launch, define the KPI the agent is expected to change: request handling time, revenue, error count, or conversion. - ROI is not a KPI. It is a calculation based on the KPI and the scenario's full cost.

The practical plan is therefore organizational: inventory the agents, assign owners and business KPIs, measure the outcome of the entire workflow rather than only the provider bill, and create a shared governance loop spanning finance, IT, and the business sponsor. Once a pilot is over, a KPI can easily become a justification for money already spent.

This perspective is useful but not sufficient. Business metrics do not replace evals, safety, or engineering reliability, just as a perfect architecture does not answer why a company needs the agent in the first place.

My main takeaway is that an agent discussion should begin with the unit of value, acceptance criteria, verification cost, and consequences of failure. Architecture and tokens come next. Framed this way, AI becomes a managed business investment rather than an experiment confined to IT.

#AI #Agents #Management #FinOps #Metrics #Engineering