Google discloses Ironwood TPUv7 details
The newest TPU promises a big step up in performance from Trillium (TPUv6). We're awaiting more details. We believe the big jump comes from native FP8 support, and the new chip delivers raw throughput on that type similar to Nvidia Blackwell. Blackwell, however, also supports FP4 at twice the rate, and Nvidia is promoting that type for inference. Google also increased HBM capacity and interface speed, but it's in line with Blackwell and AMD's offerings.
A key Google advantage has been interconnect, and Ironwood continues to excel here. The interchip interconnect (ICI) can link 9,216 Ironwoods, creating a huge, flat computing plane.
The TPU architecture is unusual. Details for v7 have been withheld, but we expect Google continues to employ a few cores, large MAC arrays, and large on-chip memories. Supplementing these are "sparse cores" that handle embeddings and do other supplementary functions.
