Google on Monday, Nov. 9, announced two new AI chip systems, TPU 8tx and TPU 8i Max, bringing its eighth generation of Tensor Processing Units to four systems in a single year for the first time.
Why it matters: Google's TPUs compete with Nvidia's GPUs for AI work. Google said TPUs are now "a rapidly growing business with nearly 100 customers," including frontier AI labs, trading firms, retailers and universities, in its announcement.
The two new systems
- TPU 8tx is built for secure, confidential inference. It keeps data and model weights encrypted and isolated even while being processed, runs on the same silicon as TPU 8t with AMD EPYC host processors, and delivers up to 2x lower latency than TPU 8t, Google said.
- TPU 8i Max puts two TPU 8i chips on one package, doubling compute, on-chip SRAM and high-bandwidth memory bandwidth. It is aimed at agents working across huge context windows and fits twice the silicon in the same rack space.
They join TPU 8t, for training, and TPU 8i, for real-time reasoning, which Google introduced at Cloud Next in April. Google said 8tx targets regulated companies and governments that need to fine-tune models on private data.
Software push
SiliconANGLE framed the launch as part of Google's push into custom silicon for AI agents. Google also said its open-source TorchTPU lets PyTorch code run natively on TPUs, and it is open-sourcing TPU Sync, which it said can cut time to first token by more than 90%. It did not give prices or a shipping date for the new systems in the announcement.
Related: Google open-sources TorchTPU and Nvidia board opens probe of Jensen Huang.