OpenAI's o3 Model Potentially More Expensive to Operate Than Initially Thought

# openai

When OpenAI introduced its o3 "reasoning" AI model back in December, they teamed up with the creators of ARC-AGI, a benchmark designed to push the limits of AI capabilities, to demonstrate what o3 could do. Fast forward a few months, and the results have been tweaked, showing that o3's performance might not be as dazzling as we first thought. Just last week, the Arc Prize Foundation, which oversees ARC-AGI, revised their estimates on the computing costs for o3. Initially, they figured that the top-performing version of o3, dubbed o3 high, would set you back about $3,000 to crack a single ARC-AGI problem. But now, they're thinking it's more like $30,000 per task. That's a hefty jump! This revision sheds light on just how pricey cutting-edge AI models can be, especially in the early stages. OpenAI hasn't put a price tag on o3 yet, nor have they released it to the public. However, the Arc Prize Foundation suggests that the pricing of OpenAI's o1-pro model could give us a rough idea of what to expect. For some context, o1-pro is currently OpenAI's most expensive model on offer. Mike Knoop, a co-founder of the Arc Prize Foundation, shared with TechCrunch, "We believe o1-pro is a closer comparison of true o3 cost ... due to the amount of test-time compute used. But this is still a proxy, and we've kept o3 labeled as preview on our leaderboard to reflect the uncertainty until official pricing is announced." Given the massive computing power o3 high reportedly requires, a high price tag wouldn't come as a shock. According to the Arc Prize Foundation, o3 high gobbled up 172 times more computing resources than o3 low, the least demanding version of o3, just to take on ARC-AGI tasks. There's also been a lot of buzz about OpenAI's potential pricing plans for enterprise clients. Back in early March, The Information reported that OpenAI might be looking to charge up to $20,000 a month for specialized AI "agents," like a software developer agent. Some might argue that even OpenAI's most expensive models are still a bargain compared to what you'd pay a human contractor or employee. But as AI researcher Toby Ord pointed out on X, these models might not be as efficient as we'd hope. For instance, o3 high needed a whopping 1,024 tries at each task in ARC-AGI to hit its best score.

Related article

Comments (46)