The article asks a question the industry has debated for years: can someone outside Google take an open-weight model, serve it through a familiar inference engine on TPUs, and beat NVIDIA on the economics that matter? It publishes the first third-party TPUv7 Ironwood results on the InferenceX Official Preview, using Qwen3.5 397B in FP8 on the new TorchTPU vLLM stack. In apples-to-apples aggregated FP8 serving, Ironwood delivers up to 50% better performance per dollar than B200 and B300; at 100 tok/s/user it costs about $0.181 per million total tokens versus $0.222 on B200 and $0.276 on B300.
The interesting part is where the advantage comes from. Across most of the raw-throughput curve TPU does not outperform the GPUs; its lower hourly cost is what shifts the performance-per-dollar frontier, and high-concurrency wins come with longer TTFT. The article also covers the kernel and serving-stack work behind the results — DP attention, MoE routing, and reduced GDN padding — and the roadmap items still ahead, such as speculative decoding, disaggregated prefill, KV-cache offloading, and agentic workloads.
Read the full article on SemiAnalysis.