DeepMind's AGI Safety Paper Fails to Sway Skeptics

# deepmind # agi

On Wednesday, Google DeepMind dropped a hefty 145-page paper diving deep into their approach to AGI safety. AGI, or artificial general intelligence, is the kind of AI that can tackle any task a human can, and it's a hot topic in the AI world. Some folks think it's just a fantasy, while others, like the big shots at Anthropic, believe it's right around the bend and could cause some serious trouble if we don't get our safety measures in check. DeepMind's paper, penned with help from co-founder Shane Legg, reckons AGI might show up by 2030 and could lead to what they call "severe harm." They don't spell it out exactly, but they toss around scary phrases like "existential risks" that could "permanently destroy humanity." "We're betting on seeing an Exceptional AGI before this decade's out," the authors note. "An Exceptional AGI is a system that can match the skills of the top 1% of adults on a bunch of non-physical tasks, including tricky stuff like learning new skills." Right from the start, the paper compares DeepMind's way of handling AGI risk with how Anthropic and OpenAI do it. It says Anthropic isn't as big on "robust training, monitoring, and security," while OpenAI's all about "automating" a type of AI safety research called alignment research. The paper also throws some shade on the idea of superintelligent AI — AI that's better at jobs than any human. (OpenAI recently said they're shifting their focus from AGI to superintelligence.) Without some major new breakthroughs, DeepMind's authors aren't buying that superintelligent systems are coming anytime soon — or maybe ever. But they do think it's possible that our current methods could lead to "recursive AI improvement," where AI does its own AI research to make even smarter AI systems. And that, they warn, could be super dangerous. Overall, the paper suggests we need to develop ways to keep bad actors away from AGI, better understand what AI systems are up to, and make the environments where AI operates more secure. They admit a lot of these ideas are still in their early stages and have "open research problems," but they urge us not to ignore the safety issues that might be coming our way. "AGI could bring amazing benefits or serious harm," the authors point out. "So, to build AGI the right way, it's crucial for the top AI developers to plan ahead and tackle those big risks." Not everyone's on board with the paper's ideas, though. Heidy Khlaaf, the chief AI scientist at the nonprofit AI Now Institute, told TechCrunch she thinks AGI is too fuzzy a concept to be "rigorously evaluated scientifically." Another AI researcher, Matthew Guzdial from the University of Alberta, said he's not convinced recursive AI improvement is doable right now. "Recursive improvement is what the intelligence singularity arguments are based on," Guzdial told TechCrunch, "but we've never seen any evidence that it actually works." Sandra Wachter, who studies tech and regulation at Oxford, points out a more pressing worry: AI reinforcing itself with "inaccurate outputs." "With more and more AI-generated content online and real data getting replaced, models are learning from their own outputs that are full of inaccuracies or hallucinations," she told TechCrunch. "Since chatbots are mostly used for searching and finding the truth, we're always at risk of being fed false information that's presented in a very convincing way." As thorough as it is, DeepMind's paper probably won't end the debates about how likely AGI really is — and which AI safety issues need the most attention right now.

Related article

Comments (45)