OpenAI’s latest AI models have a new safeguard to prevent biorisks

OpenAI's New Safety Measures for AI Models o3 and o4-mini

OpenAI has introduced a new monitoring system for its advanced AI models, o3 and o4-mini, specifically designed to detect and prevent responses to prompts related to biological and chemical threats. This "safety-focused reasoning monitor" is a response to the enhanced capabilities of these models, which, according to OpenAI, represent a significant step up from their predecessors and could be misused by malicious actors.

The company's internal benchmarks indicate that o3, in particular, has shown a higher proficiency in answering questions about creating certain biological threats. To address this and other potential risks, OpenAI developed this new system, which operates alongside o3 and o4-mini. It's trained to recognize and reject prompts that could lead to harmful advice on biological and chemical risks.

Testing and Results

To gauge the effectiveness of this safety monitor, OpenAI conducted extensive testing. Red teamers spent approximately 1,000 hours identifying "unsafe" biorisk-related conversations generated by o3 and o4-mini. In a simulation of the monitor's "blocking logic," the models successfully declined to respond to risky prompts 98.7% of the time.

However, OpenAI admits that their test did not consider scenarios where users might attempt different prompts after being blocked. As a result, the company plans to continue using human monitoring as part of its safety strategy.

Risk Assessment and Ongoing Monitoring

Despite their advanced capabilities, o3 and o4-mini do not exceed OpenAI's "high risk" threshold for biorisks. Yet, early versions of these models were more adept at answering questions about developing biological weapons compared to o1 and GPT-4. OpenAI is actively monitoring how these models might facilitate the development of chemical and biological threats, as outlined in their updated Preparedness Framework.

OpenAI is increasingly turning to automated systems to manage the risks posed by its models. For instance, a similar reasoning monitor is used to prevent GPT-4o's image generator from producing child sexual abuse material (CSAM).

Concerns and Criticisms

Despite these efforts, some researchers argue that OpenAI may not be prioritizing safety enough. One of OpenAI's red-teaming partners, Metr, noted they had limited time to test o3 for deceptive behavior. Additionally, OpenAI chose not to release a safety report for its recently launched GPT-4.1 model, raising further concerns about the company's commitment to transparency and safety.