Ironwood is Google’s newest AI accelerator chip

At this week's Cloud Next conference, Google pulled back the curtain on its latest TPU AI accelerator chip, dubbed Ironwood. This seventh-generation marvel marks a significant shift as it's the first TPU specifically tailored for AI model inference. Set to roll out later this year for Google Cloud users, Ironwood will be available in two powerhouse configurations: a 256-chip cluster and a massive 9,216-chip cluster.

Amin Vahdat, Google Cloud VP, shared his excitement in a blog post, saying, "Ironwood is our most powerful, capable, and energy-efficient TPU yet. And it's purpose-built to power thinking, inferential AI models at scale." It's clear that Google is aiming high with this new chip.

The AI accelerator market is heating up, with Nvidia leading the charge, but don't count out the tech giants. Amazon is in the game with its Trainium, Inferentia, and Graviton chips, available through AWS, while Microsoft is pushing forward with Azure instances powered by its Maia 100 AI chip.

According to Google's internal benchmarks, Ironwood packs a punch with 4,614 TFLOPs of peak computing power. Each chip comes equipped with 192GB of dedicated RAM, boasting a bandwidth that's nearing 7.4 Tbps. That's some serious firepower!

Ironwood isn't just about brute force; it's smart, too. It features an advanced core called SparseCore, designed to handle the kind of data you'd see in sophisticated ranking and recommendation systems—like those that suggest what clothes might catch your eye. Google's engineers have also worked to reduce data movement and latency within the chip, which should translate to significant energy savings.

Looking ahead, Google plans to weave Ironwood into its AI Hypercomputer, a modular computing cluster within Google Cloud. This integration is on the horizon, and Vahdat is confident, stating, "Ironwood represents a unique breakthrough in the age of inference, with increased computation power, memory capacity, networking advancements, and reliability."

Updated 10:45 a.m. Pacific: A previous version of this story mistakenly referred to Microsoft’s Cobalt 100 as an AI chip. It's actually a general-purpose chip; the correct AI chip from Microsoft is the Maia 100. We've made the necessary correction.