Top AI executives from Anthropic, OpenAI, Meta, and Microsoft urge policymakers to regulate self-improving AI

More than 20 of the most influential minds in artificial intelligence, including Nobel laureate Geoffrey Hinton and Anthropic’s Jack Clark, just published a paper that reads less like academic research and more like a flare gun fired into a crowded room. The paper, titled “What if automating AI R&D triggers an intelligence explosion,” landed on September 28 with a simple thesis: AI systems that can improve themselves might accelerate beyond any human’s ability to pull the brakes. The group includes executives and researchers from Anthropic, OpenAI, Meta, and Microsoft, four companies that are simultaneously building the most powerful AI systems on Earth and now publicly asking governments to keep those systems in check. The intelligence explosion problem At the core of the paper is a concept called recursive self-improvement, or RSI. The idea is straightforward in theory and terrifying in practice: an AI system that can improve its own capabilities could, in principle, initiate a feedback loop where each iteration makes itself smarter, which makes the next iteration even smarter, and so on. The authors argue that this dynamic could undermine human oversight entirely. If an AI s...