Google’s latest AI model report lacks key safety details, experts say

On Thursday, weeks after launching its latest and most advanced AI model, Gemini 2.5 Pro, Google released a technical report detailing the results of its internal safety assessments. However, experts have criticized the report for its lack of detail, making it challenging to fully understand the potential risks associated with the model.

Technical reports are crucial in the AI world, offering insights—even if they're sometimes unflattering—that companies might not usually share publicly. These reports are generally viewed by the AI community as genuine efforts to foster independent research and enhance safety evaluations.

Google's approach to safety reporting differs from some of its competitors. The company only publishes technical reports once a model moves beyond the "experimental" phase. Moreover, Google omits certain "dangerous capability" evaluation results from these reports, saving them for a separate audit.

Despite this, several experts expressed disappointment with the Gemini 2.5 Pro report to TechCrunch, pointing out its scant coverage of Google's proposed Frontier Safety Framework (FSF). Google unveiled the FSF last year, aiming to pinpoint future AI capabilities that might lead to "severe harm."

"This report is very sparse, contains minimal information, and was released weeks after the model was already made public," Peter Wildeford, co-founder of the Institute for AI Policy and Strategy, told TechCrunch. "It's impossible to verify if Google is living up to its public commitments and thus impossible to assess the safety and security of their models."

Thomas Woodside, co-founder of the Secure AI Project, acknowledged the release of the report for Gemini 2.5 Pro but questioned Google's dedication to providing timely supplemental safety evaluations. He noted that Google last published dangerous capability test results in June 2024, for a model announced in February of that year.

Adding to the concerns, Google has not yet released a report for Gemini 2.5 Flash, a smaller, more efficient model announced last week. A spokesperson informed TechCrunch that a report for Flash is "coming soon."

"I hope this is a promise from Google to start publishing more frequent updates," Woodside told TechCrunch. "Those updates should include the results of evaluations for models that haven’t been publicly deployed yet, since those models could also pose serious risks."

While Google was among the first AI labs to propose standardized reports for models, it's not alone in facing criticism for a lack of transparency. Meta released a similarly brief safety evaluation for its new Llama 4 open models, and OpenAI chose not to publish any report for its GPT-4.1 series.

Google's assurances to regulators about maintaining high standards in AI safety testing and reporting add pressure to the situation. Two years ago, Google promised the U.S. government to publish safety reports for all "significant" public AI models "within scope," followed by similar commitments to other countries, pledging "public transparency" around AI products.

Kevin Bankston, a senior adviser on AI governance at the Center for Democracy and Technology, described the trend of sporadic and vague reports as a "race to the bottom" on AI safety.

"Combined with reports that competing labs like OpenAI have reduced their safety testing time before release from months to days, this meager documentation for Google’s top AI model tells a troubling story of a race to the bottom on AI safety and transparency as companies rush their models to market," he told TechCrunch.

Google has stated that, although not detailed in its technical reports, it conducts safety testing and "adversarial red teaming" for models before their release.

Updated 4/22 at 12:58 p.m. Pacific: Modified language around the technical report’s reference to Google’s FSF.