Skip to content
Research Insights Made Simple
Research reviewResearch Insights Made Simple

Own GPUs or API: where break-even lies

How many tokens per second must your GPU deliver to beat an API?

/ RIMS · Own inference or API

Slide contents

  1. 1. Own GPUs or API: where break-even lies

    How many tokens per second must your GPU deliver to beat an API?

  2. 2. Utilization pays for own GPUs, not volume

    Tokens per second a pair of H100s needs to match an API

  3. 3. From token price to a decision

    The cluster series gave mechanisms; this deck adds money

  4. 4. API prices span fiftyfold

    Dollars per million tokens: input, output and a 3:1 blend

  5. 5. An H100 hour costs $1.80 to $12.29

    On-demand price per GPU, November 2026

  6. 6. Token price: hourly cost over work

    Four factors, and only one appears on the GPU bill

  7. 7. Lower utilization raises the bar

    Required tokens per second for an H100 pair at $3.95 an hour

  8. 8. Peak-sized capacity idles at night

    The Orbit example and the serving lab's lesson

  9. 9. GPUs are only the first bill line

    What adds to the GPU hour, and what stays with the API

  10. 10. Count the price of an accepted task

    Tokens get twenty times cheaper; review does not

  11. 11. APIs have caching and batching too

    Discounts that move the break-even point

  12. 12. Hundreds of fine-tunes, one base's price

    One 80 GB A100 · Llama-7B · synthetic workload

  13. 13. Local boxes don't pay back in tokens

    A $3,999 Ryzen AI Max+ 395, 128 GB, against API prices

  14. 14. Four questions before buying GPUs

    Answer in order; the first yes decides

  15. 15. The calculation transfers a method, not numbers

    Dated prices, host assumptions and what we did not measure

  16. 16. Measure first, buy second

    Three takeaways and a one-day experiment