Ex-OpenAI Policy Lead Slams Company for Altering AI Safety Narrative

# openai # policy

A former OpenAI policy researcher, Miles Brundage, recently took to social media to call out OpenAI for what he sees as an attempt to "rewrite the history" of its approach to deploying potentially risky AI systems. This week, OpenAI released a document detailing its current stance on AI safety and alignment, the process of ensuring AI systems act in predictable and beneficial ways. In it, OpenAI described the development of AGI, or AI systems capable of any task a human can do, as a "continuous path" that involves "iteratively deploying and learning" from AI technologies. "In a discontinuous world [...] safety lessons come from treating the systems of today with outsized caution relative to their apparent power, [which] is the approach we took for [our AI model] GPT-2," OpenAI stated in the document. "We now view the first AGI as just one point along a series of systems of increasing usefulness [...] In the continuous world, the way to make the next system safe and beneficial is to learn from the current system." However, Brundage argues that the cautious approach taken with GPT-2 was entirely in line with OpenAI's current iterative deployment strategy. "OpenAI's release of GPT-2, which I was involved in, was 100% consistent [with and] foreshadowed OpenAI's current philosophy of iterative deployment," Brundage posted on X. "The model was released incrementally, with lessons shared at each step. Many security experts at the time thanked us for this caution." Brundage, who joined OpenAI as a research scientist in 2018 and later became the company's head of policy research, focused on the responsible deployment of language generation systems like ChatGPT while on OpenAI's "AGI readiness" team. GPT-2, announced by OpenAI in 2019, was a precursor to the AI systems behind ChatGPT. It could answer questions, summarize articles, and generate text that was sometimes indistinguishable from human writing. Though GPT-2 might seem basic now, it was groundbreaking at the time. Due to concerns about potential misuse, OpenAI initially withheld the model's source code, instead allowing select news outlets limited access to a demo. The decision received mixed feedback from the AI community. Some argued that the risks associated with GPT-2 were overstated, and there was no evidence to support OpenAI's concerns about misuse. The AI-focused publication The Gradient even published an open letter urging OpenAI to release the model, citing its technological significance. OpenAI eventually released a partial version of GPT-2 six months after its announcement, followed by the full system several months later. Brundage believes this was the correct approach. "What part of [the GPT-2 release] was motivated by or premised on thinking of AGI as discontinuous? None of it," he stated on X. "What's the evidence this caution was 'disproportionate' ex ante? Ex post, it prob. would have been OK, but that doesn't mean it was responsible to YOLO it [sic] given info at the time." Brundage is concerned that OpenAI's document aims to establish a high burden of proof, where concerns are dismissed as alarmist unless there's overwhelming evidence of imminent dangers. He finds this mindset "very dangerous" for advanced AI systems. "If I were still working at OpenAI, I would be asking why this [document] was written the way it was, and what exactly OpenAI hopes to achieve by poo-pooing caution in such a lop-sided way," Brundage added. OpenAI has faced criticism in the past for prioritizing "shiny products" over safety and rushing releases to outpace competitors. Last year, the company disbanded its AGI readiness team, and several AI safety and policy researchers left for rival firms. The competitive landscape has intensified, with Chinese AI lab DeepSeek's R1 model, which is openly available and matches OpenAI's o1 "reasoning" model on key benchmarks, drawing global attention. OpenAI CEO Sam Altman has acknowledged that DeepSeek has narrowed OpenAI's technological lead, prompting OpenAI to consider accelerating its release schedule. With OpenAI reportedly losing billions annually and projecting losses to triple to $14 billion by 2026, a faster product release cycle could boost short-term profits but potentially compromise long-term safety. Experts like Brundage are questioning whether this trade-off is worthwhile.

Related article

Comments (36)