The Economics of AI Development: A Complete Analysis from Tokens to Accepted Work

Discussions about AI budgets often mix three different prices. The first is the cost of reaching a known quality level. It falls quickly as newer models, hardware, and inference optimization make yesterday's outcome cheaper. The second is the price of the new frontier, long context, high speed, and guaranteed capacity. That layer remains premium. The third is the company's total budget, driven by task volume, automation depth, and the number of production use cases.

Market signals point to demand expansion. Menlo Ventures estimated U.S. enterprise GenAI spend at $37 billion in 2025, up from $11.5 billion a year earlier. This is a venture firm's model based on roughly five hundred enterprise decision-makers—not a census of every invoice. The State of FinOps 2026 says 98% of respondents now manage AI spend, versus 63% a year earlier and 31% two years earlier. McKinsey likewise reports broad adoption but a much narrower group scaling AI across the business.010101These are survey-based models, not a market census: McKinsey's State of AI 2025, for instance, rests on an online poll of 1,993 respondents weighted by each country's share of GDP. I walked through its methodology on my channel.Knizhny kub · McKinsey State of AI 2025 (in Russian)

These observations cannot yield a precise 2029 budget. They can support a management scenario. In this scenario, today's fixed quality becomes 3–10 times cheaper, a typical accepted task becomes 2–5 times cheaper, the new frontier stays around the same order of list price or falls by up to half, while the total budget of a successful AI portfolio grows 2–5 times. These are author ranges with different confidence levels, not a market forecast to the last percentage point.

Optimizing the price per million tokens sees only one multiplier. The actual bill depends on task volume, model calls per task, context per call, call price, seats, platform layers, and reserved capacity. A cheaper call makes new use cases viable, while an agent turns one user action into a branching workflow.

AI spend = tasks × calls per task × tokens per call × token price + seats + platform + fixed capacity

The FinOps Foundation gives a useful range: one agent interaction may generate 5, 10, or 50 model calls. Every call can resend context, append tool results, retry, or open another branch. An aggregate invoice cannot distinguish a valuable complex trace from a looping agent. Cost must be attributable to use_case, task_class, andtrace_id.

Lower unit prices therefore do not guarantee lower total cost. They create room for more demand. The management question is not whether consumption rose, but what outcome each additional call created and where the workflow developed a heavy tail.020202This is the premium capacity layer: per SemiAnalysis a one-year H100 contract rose about 40% in six months, and the shortage is now driven by production inference over agents, not training. I covered that H100 price index on my channel.Knizhny kub · the GPU shortage and H100 price index (in Russian)

A global per-user token cap mixes autocomplete, architectural analysis, and a production agent with authority to change a system. It punishes a useful power user yet offers weak protection from one runaway workflow. For a copilot, the model call often costs less than a few minutes of an engineer's time. For an autonomous agent, the danger lies in branches, retries, growing context, and repeated tool calls.

A minimum production control plane bounds several resources at once:

• —max_cost_usd, steps, retries, and repeated tool calls;
• —tool-output size, model timeout, workflow timeout, and concurrency;
• —tool authority, permitted environments, and human approval requirements;
• —loop detection, kill switches, and safe fallback or handoff;
• —mandatory team, product, environment, use-case, and trace tags.

Raising a limit is itself a management decision. It needs an owner, a reason, and an observable outcome. Otherwise, “temporarily give the agent more budget” quietly becomes a permanent mode without feedback.

The argument over budgeting by person, team, or project is a false choice. Seats naturally belong to a person or function, exploration to a team, production inference to a product or workflow, and shared gateways, evals, and security to a platform. Putting all of them under one cost center hides both value and the reason spend grows.

With limited history, planning P50 and P90 plus an uncertainty reserve is more honest than one point estimate. A 15–25% reserve is a practical starting heuristic, not a FinOps standard. It should shrink as stable workload and quality profiles emerge.

Showback before chargeback

Early hard chargeback encourages teams to hide experiments and debate false precision in shared-platform allocation. For the first one or two quarters, expose cost and outcome by product and team; then add budgets and soft thresholds. Chargeback becomes useful for stable production workloads with a clear owner and unit economics.

Tokens are a convenient billing unit but a weak value unit. A generated patch is not yet an outcome: it may fail tests, require lengthy review, or introduce a regression. The denominator of unit economics should contain only work that passes a common acceptance criterion.030303“Passed the tests” and “accepted” are not the same: a model can post a strong pass rate and still leave code an enterprise team spends ages untangling. I covered the Sonar talk on exactly this on my channel.Knizhny kub · why pass rate is no longer enough (in Russian)

cost per accepted task = (model + tools + retrieval + gateway + compute + human review + rework + expected failure loss) / accepted tasks

“Accepted” must be defined for each task class. For a bug fix, it may be a hidden regression test and a green full suite; for review, a useful finding without noise; for a migration, a correct artifact with no regression; for a production agent, a completed workflow with permitted effects. Price and latency become comparable only after candidates pass one quality threshold.

Run every candidate several times on the same real tasks and count the entire trace: tools, retries, fallback, and human time. A cheaper model at 70% success can beat a more expensive one at 95% only while the review and rework difference remains small. One extra minute of engineer time—or one rare expensive failure—can reverse the choice.

Self-estimates are not outcomes

METR's 2025 RCT across 16 experienced open-source developers and 246 tasks found a 19% slowdown with early-2025 tools, even though participants believed afterward that they had accelerated by roughly 20%. The authors now mark that result as out of date. Their 2026 update shifted toward likely acceleration but encountered strong selection effects and led to a redesign of the next experiment.

The useful conclusion is neither “AI always slows developers down” nor “new models solved productivity.” A narrow sample cannot represent all software work, and tools change quickly. The study demonstrates why perception, activity, and benchmarks cannot replace an internal baseline. DORA offers the more durable framing: AI amplifies the strengths and weaknesses of the delivery system around it.040404On this METR study I invited a Yandex platform lead, and we spent 40 minutes on the methodology, the 16-engineer sample, and how the headlines outran the paper itself. The episode is on my channel.Knizhny kub · Research Insights #17 on METR (in Russian)

An HTTP client or OpenAI-compatible endpoint is usually the easiest layer to replace. Migration fails higher up: prompts were tuned to one model's style, tools expect particular behavior, recovery assumes familiar failures, and the eval set misses silent degradation. Deeper still are conversation state, data, commercial commitment, and team skills.

The useful abstraction is not a lowest-common-denominator API but the business task contract: input, expected artifact, acceptance criteria, and permitted effects. Business code calls review_patch orsummarize_incident; configuration selects the model. State, documents, audit trails, and evals remain under organizational control.

A lowest common denominator also has a cost because it blocks valuable provider capabilities. A practical author heuristic is to make the critical 80% portable and isolate the valuable provider-specific 20% behind adapters, regression evals, and an exit plan. A critical workflow needs a certified secondary route, not merely a compatible request schema.050505The same lock-in, seen from the vendor's side, is a “moat”: it helps to see which moats AI startups build, so you know what portability is actually buying you. I covered YC's “7 moats” breakdown on my channel.Knizhny kub · 7 moats for AI startups (in Russian)

Account-team engagement should not wait for a large invoice. A critical process, sensitive data, SLA requirements, regional constraints, or guaranteed capacity can justify it earlier. Sustained spend of $10–25k per month with one provider, or a credible $100–250k annual forecast, is only an economic heuristic for procurement and legal overhead—not an official threshold published by any provider.

Commit around the base load—P50–P60, for example—and keep peaks on PAYG when provider economics allow it. A contract without usage export, model versions, deprecation windows, and exit rights can offer an attractive discount while making future optimization more expensive.

Users should not have to track price sheets, regional availability, and the latest leaderboard. They need to recognize intent and error cost. A practical interface exposes modes: Auto for most requests, Fast for simple transformations, Deep for architecture and difficult defects, and Sensitive for an approved local or contracted route.

Behind that interface sit three independent loops. The gateway handles authentication, secrets, budgets, retries, logging, and enforcement. The router selects an allowed route by risk, capability, cost, health, and latency. Evals independently verify quality so the router cannot certify its own decisions.060606Measuring review quality is its own hard problem: SWE-PRBench uses real merged PRs and real human review comments as ground truth instead of synthetic ones. I covered that benchmark on my channel.Knizhny kub · SWE-PRBench on review quality (in Russian)

A learned router is a poor starting point. Begin with complete telemetry and task tags, then aliases and static rules, certified fallback and canaries, followed by cascades that escalate on failed tests, invalid schemas, or insufficient retrieval. Learning becomes appropriate only after real outcome labels accumulate.

RouteLLM and FrugalGPT demonstrate the potential of cascades, but their savings percentages do not transfer to another company's task distribution and evals. Routing is not about finding the cheapest model; it is about finding the cheapest route that still passes acceptance.

A local model is not free. The API invoice becomes GPUs or cloud commitment, idle capacity, HA headroom, storage, inference engineering, SRE, security, and model updates. Variable cost may be lower, but fixed cost and underutilization make small or unpredictable workloads expensive.

Q* = fully loaded local fixed cost / (cloud variable cost per accepted task − local variable cost per accepted task)

The formula is meaningful only after a common quality gate. If the local candidate needs more review, rework, or escalation, its variable cost per accepted task may exceed cloud. Local is strongest for data residency, air gaps, stable high-volume classification, extraction, embeddings, reranking, and narrow tasks after distillation.070707If you take local seriously, there's a practical guide to squeezing more from your hardware: why llama.cpp beats ollama, what quantization buys you, and how Dense compares to MoE. I retold that article on my channel.Knizhny kub · getting more out of local LLMs (in Russian)

Hybrid beats infrastructure ideology

In a mature system, local is one route. It handles repeatable flow, preprocessing, and tasks with a strict data path; the complex or uncertain tail escalates to a frontier API under the same acceptance gate. Privacy, latency, and utilization no longer require giving up frontier quality where it genuinely pays off.

The first quarter should not begin with a large model marketplace or a GPU purchase. Make volume, quality, cost, and failure consequences visible across a few high-volume use cases; then add guardrails; only afterward should routing and the commercial layer become more sophisticated.

Days 0–30: inventory and baseline

Find seats, keys, gateways, endpoints, and owners. Select 3–5 high-volume use cases, define the accepted outcome, and capture volume, cost, pass rate, review time, and P50/P90/P99. Without this, negotiation and optimization remain anchored to the invoice rather than the work.

Days 31–60: guardrails and showback

Add trace budgets, loop detection, kill switches, the sandbox/production boundary, product showback, and golden evals. Validate caching and batch against the real workload profile; a low hit rate may not justify another layer of complexity.

Days 61–90: routes and resilience

Certify a secondary provider, add canaries and static modes, run a failover drill, prepare a negotiation package, and launch one local pilot with fully loaded TCO. By the end, every decision should have an owner, a baseline, and a stop criterion.

The operating system cannot collapse into one average cost or composite score. Each task class needs acceptance, cost, distribution tails, human time, technical failures, and failure consequences together. P99 matters as much as P50 because loops, overload, and rare expensive incidents live in the tail.

This scorecard gives engineering, platform, FinOps, risk, and procurement one language. The router receives outcome feedback, teams see review cost, the platform sees heavy traces, and procurement sees the volume worth committing. The objective is no longer “spend fewer tokens.” It is “maximize accepted work at the required quality and risk level.”