Skip to content
#Engineering

[2/2] The History of Google TPU and its Evolution (Category Engineering)

#Engineering #AI #ML #Software #Architecture #Infrastructure #Data

Continue. story TPU from Google 2021 For example, with TPU v4. 4. TPU v4 (2021) - Optical Circuit Switching TPU v4 introduced optical circuit switching to speed up communication between chips, which is critical for working with increasingly complex AI models. The productivity has been 275 TFLOPS per chip, with improved optical interconnects. 5. TPU v5 and v5e (2023) - Cost optimization TPU v5e and v5p focus on cost-effective learning at scale, with improved energy efficiency, dynamic scaling, and sparseness support. 6. TPU v6 Trillium (2024) - Optimizing productivity Trillium, the sixth generation of the TPU, offers an impressive leap forward. 4.7 TPU performance per chip compared to TPU v5e. It also has the following characteristics Double the capacity and bandwidth of High Bandwidth Memory (HBM) Double the bandwidth of interchip connections

  • On. 67Percent more energy efficient than TPU v5e
  • Scaling up to 256 TPU in one pod with low latency 7. TPU v7 Ironwood (2025) - again infernal. Ironwood, introduced in April 2025 A new TPU specifically designed for inferencing (as a TPU v1). Revolutionary characteristics of Ironwood:
  • Scaling up to 9,216 liquid-cooled
  • 42.5 exascale (into 24 More than the most powerful supercomputer El Capitan)
  • 4,614 TFLOPS on a chip with 192 GB HBM memory (into 6 more than the Trillium)
  • 2Multiple energy efficiency compared to Trillium

To summarize, Google’s processors have come a long way. The question is, how do they feel compared to Nvidia? Below is the answer to that question.

  • NVIDIA H100: 3,958 TFLOPS (FP8), 80 GB HBM3, memory bandwidth 3.35 TB
  • NVIDIA H200: 3,958 TFLOPS (FP8), 141 GB HBM3e, memory bandwidth 4.8 TB
  • TPU v6 Trillium: ~2 PFLOPS FP16 for tensor operations
  • TPU v7 Ironwood: 4,614 TFLOPS on the chip, 192 GB HBM, 7.37 TB/s throughput

As you can see from FLOPS, it's normal. If we look at the effectiveness of independent nThe TPU v5e shows in 50-70Percent lower cost per billion tokens for training large models compared to NVIDIA H100 clusters. TPU v5e also consumes significantly less power than H100 for a similar workload. (H100 can consume in ~5 More power than a TPU v5e chip under load). In real problems, the indicators are similar To train GPT-scale models: TPUs are more cost-effective 4-10 compared to the GPU For inferencing: TPU v5e provides in 3 times the capacity per dollar TPU v4 showed performance 1.2-1.7 faster and uses it faster. 1.3-1.9 Less energy than the NVIDIA A100

As a result, TPU has both advantages and disadvantages. (+) Specialization for tensor operations and deep learning (+) High energy efficiency and economic efficiency (+) Google Cloud integration and optimization for TensorFlow/JAX (+) Scalability in the Google Cloud Ecosystem (-) Available only through Google Cloud (-) Less flexibility compared to GPUs for different types of computing (-) Limited development ecosystem compared to CUDA (-) Less memory per chip compared to the latest GPUs (until recently)

To sum it up, it seems that Google is doing well with its lineup of processors for Gen AI/ML tasks and will continue to build on this infrastructure, which gives them a significant competitive advantage in the era of Gen AI applications summer. But for the rest, these processors mean a vendor lock in direct use or a reference point, where to look in the future.

#AI #ML #Software #Engineering #Architecture #Infrastructure #Data