Eric Schmidt Opposes AGI Manhattan Project

# openai # China # agi

In a policy paper released on Wednesday, former Google CEO Eric Schmidt, along with Scale AI CEO Alexandr Wang and Center for AI Safety Director Dan Hendrycks, advised against the U.S. launching a Manhattan Project-style initiative to develop AI systems with "superhuman" intelligence, commonly referred to as AGI. Titled "Superintelligence Strategy," the paper warns that a U.S. effort to monopolize superintelligent AI could provoke a strong response from China, possibly in the form of a cyberattack, which might disrupt global relations. The co-authors argue, "A Manhattan Project for AGI assumes that rivals will acquiesce to an enduring imbalance or omnicide rather than move to prevent it. What begins as a push for a superweapon and global control risks prompting hostile countermeasures and escalating tensions, thereby undermining the very stability the strategy purports to secure." This paper, penned by three key figures in the American AI sector, arrives shortly after a U.S. congressional commission suggested a "Manhattan Project-style" initiative to fund AGI development, drawing parallels to the U.S. atomic bomb project of the 1940s. U.S. Secretary of Energy Chris Wright recently declared the U.S. to be at "the start of a new Manhattan Project" on AI, speaking at a supercomputer site with OpenAI co-founder Greg Brockman by his side. The Superintelligence Strategy paper challenges the recent push by several American policy and industry leaders who believe a government-backed AGI program is the best way to keep up with China. Schmidt, Wang, and Hendrycks see the U.S. in a situation akin to a standoff over AGI, similar to the concept of mutually assured destruction. Just as nations avoid monopolizing nuclear weapons to prevent preemptive strikes, the authors suggest the U.S. should be wary of rushing to dominate highly advanced AI systems. While comparing AI to nuclear weapons might seem over the top, global leaders already view AI as a crucial military asset. The Pentagon has noted that AI is accelerating the military's kill chain. The authors introduce the idea of Mutual Assured AI Malfunction (MAIM), where governments might take preemptive action to disable threatening AI projects rather than waiting for adversaries to weaponize AGI. Schmidt, Wang, and Hendrycks recommend that the U.S. shift its focus from "winning the race to superintelligence" to developing methods to deter other countries from creating superintelligent AI. They suggest the government should "expand its arsenal of cyberattacks to disable threatening AI projects" controlled by other nations and restrict adversaries' access to advanced AI chips and open-source models. The paper highlights a split in the AI policy community between the "doomers," who believe catastrophic AI outcomes are inevitable and advocate for slowing AI progress, and the "ostriches," who push for accelerating AI development and hope for the best. The authors propose a third path: a cautious approach to AGI development that emphasizes defensive strategies. This stance is particularly noteworthy from Schmidt, who has previously emphasized the need for the U.S. to aggressively compete with China in AI development. Just months ago, Schmidt wrote an op-ed stating that DeepSeek marked a pivotal moment in the U.S.-China AI race. Despite the Trump administration's determination to advance America's AI development, the co-authors remind us that U.S. decisions on AGI have global implications. As the world observes America's push into AI, Schmidt and his co-authors suggest a more defensive strategy might be the wiser choice.

Related article

Comments (27)