AI researchers fear superintelligence could one day outpace human control, but competition is pushing labs to build increasingly powerful systems.
BY PC Bureau
September 9, 2026: A 27-year-old AI researcher has walked away from two of the world’s leading artificial-intelligence labs with a warning that is difficult to ignore: the race to build increasingly powerful AI may be moving faster than humanity’s ability to control it — and could, he warns, put humanity at risk of extinction within a decade.
Jacob Coxon, who spent three years working on frontier models at OpenAI and Anthropic, resigned this week and used his departure to raise a fundamental question — what happens if the systems being built eventually become more capable than their creators?
Coxon argues that neither company is doing enough to confront what he sees as an existential danger. In his view, the industry is racing towards self-improving superintelligence while treating the possibility of catastrophic consequences as a risk that society is expected to accept.
His most alarming claim is that some people working closest to the technology privately believe advanced AI could pose a genuine risk of human extinction within this decade, even as public statements from industry leaders tend to be considerably more cautious.
Coxon believes outsiders should not dismiss the possibility simply because today’s AI systems remain imperfect.
He points to the rapid progression of frontier models and argues that future systems could potentially outperform humans across an enormous range of intellectual tasks, compromise computer networks, transform industries and acquire influence over real-world resources. In his assessment, the pace of progress offers little evidence that the technological curve is naturally slowing.
Earlier today, AI researcher Jacob Coxon, who spent the past three years working at OpenAI and Anthropic, publicly resigned from Anthropic, warning that both companies are racing toward superintelligence and “gambling with our lives.”
Then Evan Hubinger, Anthropic’s current Head… pic.twitter.com/C8s8EY7yyL
— Yashar Ali 🐘 (@yashar) September 9, 2026
The race may be the real danger
For Coxon, the central problem is not simply the technology. It is competition.
He draws a distinction between his experiences at OpenAI and Anthropic. At OpenAI, he says, many employees have yet to fully absorb the potential stakes. At Anthropic, he argues, researchers are more conscious of the danger but remain caught in a competitive race.
The logic is difficult to escape: if competitors continue advancing, slowing down may appear to carry its own risks.
Coxon considers that reasoning dangerous.
He argues that humanity should not enter what some AI researchers describe as the “endgame” on the assumption that safety problems can be solved later. If researchers cannot reliably understand or control increasingly capable systems, accelerating their development could magnify the very risks they are trying to manage.
For him, alignment research — the effort to ensure advanced AI systems reliably pursue goals compatible with human interests — should precede, rather than simply accompany, the drive towards greater capability.
His message to those who stayed
Coxon has challenged researchers who remain inside the industry to imagine what they may soon be asked to do.
Would they launch a training run designed to produce a system vastly more capable than humans without first having a rigorous understanding of how that system reaches its decisions?
Would they remain silent because competitors were advancing anyway?
Or would they demand stronger safeguards, greater transparency and a different set of rules for the race?
He has also pointed to recent security incidents affecting AI infrastructure as evidence that companies can cooperate when they regard a threat as sufficiently serious. His concern is that similar coordination has not yet been applied to the broader problem of controlling the race itself.
Coxon has therefore argued for measures that would once have sounded almost unthinkable in Silicon Valley — including a temporary pause on increasing model capabilities if that were necessary to prevent an uncontrolled global race.
READ: Hormuz on Edge: Iran Claims US Underwater Drone Capture and Reaper Shootdown
An Anthropic researcher agrees on the danger
Perhaps the most striking response came from Anthropic alignment researcher Evan Hubinger, who did not dismiss Coxon’s warning.
Hubinger has publicly acknowledged that people at Anthropic genuinely consider the possibility that advanced AI could cause human extinction. He has also assigned a significant probability to such an outcome within the next decade and acknowledged that the field does not yet possess a proven method for aligning a genuinely superintelligent system with human interests.
For Hubinger, today’s models are not necessarily the ultimate danger.
The bigger concern is what happens if future systems become capable of improving the technology that produced them — creating a cycle of increasingly rapid capability gains that humans may struggle to understand or control.
That possibility remains a subject of intense debate among AI researchers. There is no scientific consensus that AI will destroy humanity, nor is there agreement on when — or whether — machines will reach anything resembling superintelligence.
But the warning from researchers working closest to the technology deserves attention precisely because it is not coming from outside the industry.
One resignation will not stop the AI race. Nor does it prove that catastrophe is inevitable.
It does, however, expose an uncomfortable contradiction at the heart of the AI revolution: some of the people building the most powerful systems believe the risks could be extraordinary — and they are continuing to build them anyway.
