Exclusive Tour of Amazon's Trainium Lab: Chip Wins Anthropic, OpenAI, Apple

Shortly after Amazon CEO Andy Jassy unveiled AWS’s landmark $50 billion investment partnership with OpenAI, the company offered me a private tour of the chip development lab at the core of the deal, at mostly its own expense.

Industry observers are closely tracking Amazon’s Trainium chip, built at that facility, for its potential to lower the cost of AI inference and possibly challenge Nvidia’s near-monopoly.

Intrigued, I accepted the invitation.

My guides for the day were lab director Kristopher King (pictured on the right below), engineering director Mark Carroll (on the left), and the team’s PR contact who arranged the visit, Doron Aronson (shown with me later in the story).

AWS Chip lab leaders Mark Carroll and Kristopher King.Image Credits:TechCrunch/Julie Bort

AWS has been Anthropic’s primary cloud platform since the AI lab’s early days — a relationship strong enough that it endured when Anthropic later added Microsoft as a cloud partner, and as Amazon deepened its ties with OpenAI.

The OpenAI agreement designates AWS as the exclusive provider of the model maker’s new AI agent builder, Frontier — a tool that could become integral to OpenAI’s business if agents fulfill Silicon Valley’s expectations. Whether that exclusivity holds as announced remains to be seen. The Financial Times reported this week that Microsoft may argue the Amazon deal violates its own agreement with OpenAI, specifically that Redmond is entitled to all of OpenAI’s models and technology.

Why does OpenAI find AWS so attractive? As part of this pact, the cloud giant committed to supplying OpenAI with 2 gigawatts of Trainium computing capacity — a massive pledge, considering that Anthropic and Amazon’s own Bedrock service are already consuming Trainium chips faster than Amazon can produce them.

Amazon’s Trainium3 chip.Image Credits:Amazon

Trainium vs. Nvidia

Beyond offering an alternative to Nvidia’s backlogged, hard-to-find GPUs, Amazon says its new chips running on its specialty Trn3 UltraServers deliver comparable performance at up to 50% lower cost than using traditional cloud servers.

Alongside Trainium3, released in December, the AWS team also built new Neuron switches. Carroll says that combination is transformative.

“What that gives us is something huge,” Carroll said. The switches enable every Trainium3 chip to communicate with every other chip in a mesh setup, reducing latency. “That’s why Trainium3 is breaking all kinds of records,” especially in “price per power,” he added.

When trillions of tokens are processed daily, such improvements add up.

In fact, Amazon’s chip team earned praise from Apple in 2024. In a rare moment of openness for the secretive company, Apple’s director of AI publicly described how Apple used another of the team’s chips — Graviton, a low-power ARM-based server CPU and the first breakout chip the team designed. Apple also lauded Inferentia, a chip built specifically for inference, and gave a nod to Trainium, which was still new at the time.

These chips follow the classic Amazon playbook: see what customers want, then build an in-house alternative that competes on price.

Historically, the challenge for chips has been switching costs. Applications written for Nvidia’s chips must be re-architected to work with others — a time-consuming process that discourages developers from switching.

But the AWS chip team proudly told me that Trainium now supports PyTorch, a popular open-source framework for building AI models. That includes many models hosted on Hugging Face, a vast library where developers share open-source models.

The transition, Carroll said, requires “basically a one-line change, then recompile, and then run on Trainium.” In other words, Amazon is chipping away at Nvidia’s market dominance wherever possible.

AWS also announced a partnership this month with Cerebras Systems, integrating that company’s inference chip on servers running Trainium for what Amazon promises will be superpowered, low-latency AI performance.

But Amazon’s ambitions go beyond the chips themselves. The team also designs the server that houses the chips. Along with networking components, they created “Nitro,” a hardware-software combination that provides virtualization (allowing many software instances to run separately on the same server); new state-of-the-art liquid cooling; and the server sleds (pictured below) that hold the gear.

All of this is aimed at controlling cost and performance.

AWS Austin chip lab tour, sled with components.Image Credits:TechCrunch/Julie Bort

Working 24/7 on the “bring-up”

Amazon’s custom chip-designing unit began when the cloud giant acquired Israeli chip designer Annapurna Labs in January 2015 for about $350 million. So the team has spent over a decade designing chips for AWS. The unit retains its Annapurna roots and name — its logo is everywhere in the office.

This chip lab sits in a shiny, chrome-windowed building in Austin’s upscale Domain district, a walkable area filled with shops and restaurants, sometimes called Austin’s Silicon Valley.

The offices have a classic tech corporate vibe: cubicle desks, gathering spots, and conference rooms. But tucked away at the back of a high floor is the actual lab, with sweeping city views.

The shelving-filled lab, about the size of two large conference rooms, is a noisy industrial space thanks to equipment fans. It looks like a cross between a high school shop class and a Hollywood set for a high-end lab — except engineers wear jeans instead of white lab coats.

AWS Austin Chip Lab.Image Credits:TechCrunch/Julie Bort

AWS Austin chip lab.Image Credits:TechCrunch/Julie Bort

Note that chips aren’t manufactured here, so no white hazmat suits were needed. The Trainium3 is a state-of-the-art 3-nanometer chip produced by TSMC, arguably the leader in 3-nanometer manufacturing, with other chips made by Marvell.

But this is the room where the magic of the “bring-up” happens.

“A silicon bring-up is when you get the chip for the first time, and it’s like a big overnight party. You stay here, like a lock-in,” King explains. After 18 months of work, the chip is activated for the first time to verify it works as designed. The team even filmed part of the Trainium3 bring-up and posted it on YouTube.

Spoiler alert: It’s never problem-free.

For Trainium3, the prototype chip was originally air-cooled, like previous versions. The current chip is now liquid-cooled, which offers energy advantages and was a considerable engineering feat.

During the bring-up, the dimensions for how the chip attached to the air-cooling heat sink were off, so the chip couldn’t be activated.

Unfazed, the team “immediately got a grinder and just started grinding off the metal,” King said. To avoid disrupting the bring-up pizza party atmosphere, they snuck off and did the grinding in a conference room.

Staying up all night and solving problems “is what silicon bring-up is all about,” King said.

The lab even has a welding station, where hardware lab engineer and master welder Isaac Guevara demonstrated welding tiny integrated circuit components under a microscope. This is such difficult work that senior leader Carroll openly admitted he couldn’t do it, prompting laughter from Guevara and the other engineers in the room.

AWS Austin chip lab tour, welding station.Image Credits:TechCrunch/Julie Bort

The lab also contains both custom-made and commercial tools for testing and analyzing chip issues. Here’s signal engineer Arvind Srinivasan demonstrating how the lab tests each tiny component on the chip:

AWS Austin chip lab tour, testing equipment.Image Credits:TechCrunch/Julie Bort

Sleds are the star of the lab

But the star of the lab is an entire row showcasing each generation of the sleds the team designed.

AWS Austin chip lab tour wall of sleds.Image Credits:TechCrunch/Julie Bort

Sleds are the trays that house Trainium AI chips, Graviton CPU chips, and supporting boards and components. Stack them on a rack with the networking component — also custom-designed by this team — and you get the systems at the heart of Anthropic Claude’s success.

Here’s the sled shown off during the AWS re:Invent conference in December:

AWS Austin chip lab tour, Trainium3 sled.Image Credits:TechCrunch/Julie Bort

Proven by Anthropic and OpenAI

I expected my guides to boast about the OpenAI deal during the tour. But they didn’t.

The reticence could relate to the potential legal haze surrounding the deal. But the sense I got was that these boots-on-the-ground engineers — currently designing the next version, Trainium4 — haven’t had much chance to work with OpenAI yet. Their day-to-day has so far focused on Anthropic’s and Amazon’s needs.

Currently, the largest chunk of Trainium2 chips is deployed in Project Rainier — one of the world’s biggest AI compute clusters — which went live in late 2025 with 500,000 chips. It’s used by Anthropic.

But a wall monitor in the main office displayed a quote about how OpenAI will use Trainium. The pride was there, if subtle.

In addition to this lab, the team has its own private data center for quality and testing. A short drive away, it doesn’t run customer workloads, so it’s housed at a co-location facility, not an AWS data center.

Security is tight: strict protocols to enter the building and to access Amazon’s area within.

The data center’s cooling system is so loud that earplugs are mandatory, and the air is thick with the acrid smell of heated metal. It’s not a pleasant place for the average person to hang out.

Here’s me and Aronson at the AWS Austin chip lab data center, protecting our ears next to live servers.Image Credits:TechCrunch / Julie Bort

At this data center, rows and rows of servers are filled with sleds that integrate all of Amazon’s newest custom chips: Graviton CPU, liquid-cooled Trainium3, Amazon Nitro — all humming away. The liquid runs on a closed system, meaning it is reused, which should help reduce environmental impact, the engineers said.

Here’s what a current Trn3 UltraServer looks like: multiple sleds on top and bottom, with Neuron switches in the middle. Hardware development engineer David Martinez-Darrow is seen performing maintenance on a sled:

AWS Austin chip lab tour data center.Image Credits:TechCrunch/Julie Bort

While attention on the team has always been high, scrutiny has really ratcheted up recently.

Amazon CEO Andy Jassy keeps a close eye on this lab, publicly bragging about its products like a proud parent. In December, he said Trainium was already a multibillion-dollar business for AWS and called it one piece of AWS tech he’s most excited about. He also gave the chip a shout-out when announcing the OpenAI agreement.

The team feels the pressure, too. Engineers work 24/7 for three to four weeks around each bring-up event to fix issues so the chips can be mass-produced and deployed in data centers.

“It’s very important that we get as fast as possible to prove that it’s actually going to work,” Carroll said. “So far, we’ve been doing really well.”

*Disclosure: Amazon provided airfare and covered one night at a local hotel. Honoring its Leadership Principle of Frugality, this was a back-of-the-plane middle seat and a modest room. TechCrunch covered other travel costs like Ubers and luggage fees. (Yes, I checked a bag for an overnight trip. I’m high maintenance that way.)