Josh Engels, who recently left Google DeepMind’s AGI safety team, has warned that artificial intelligence could cause “immense harm” in the next five years, arguing that AI companies may be moving towards superintelligence faster than researchers can ensure the technology is safe.
In a post on X, Engels said he left Google DeepMind three weeks ago to join AI evaluation organisation METR, despite enjoying his work at the company and turning down offers from rival AI labs Anthropic and OpenAI.
He said the decision was driven by his assessment of the risks posed by increasingly capable AI systems.
Why Josh Engels left Google DeepMind
Engels said his decision to leave Google DeepMind was not because he was unhappy with his work there. Instead, he believes the stakes surrounding AI development have become too high.
“To some of my friends and family this seemed like a strange decision: I enjoyed the work I did at GDM and turned down offers from Anthropic and OpenAI. But I made the decision because of how high I think the stakes are right now,” he wrote.
Two days after Engels announced his exit in the X post, another Google DeepMind researcher revealed that he too had quit the AI lab over safety concerns.
(Also read: Google researcher says AI may ‘kill us all’ in exit post: ‘We might be running out of time’)
What is recursive self-improvement?
Engels argued that major AI companies are ultimately trying to build superintelligent systems — AI that would be “vastly better than humans at everything”.
He said one route towards this could be recursive self-improvement (RSI), in which AI systems help develop increasingly capable versions of themselves, creating a feedback loop that could accelerate AI development.
(Also read: ‘We really believe AI could kill all humans’: Anthropic safety lead after co-worker resigns)
“If this goes well, the resulting systems could be amazing at solving countless problems for humanity,” Engels said. However, he warned that researchers do not currently know how to make AI systems sufficiently safe before allowing them to participate in such a process.
“But we don’t currently know how to make sure AIs are safe enough for RSI, and a misaligned RSI loop could be catastrophic,” he wrote.
AI systems showing troubling behaviour
Engels also pointed to recent incidents involving AI systems that he believes indicate that models are not becoming sufficiently aligned with human intentions.
“In the last few weeks we’ve learned about models colluding with each other, hacking into companies, hiding their tracks, and socially engineering humans,” he wrote.
He clarified that the individual incidents were not necessarily catastrophic on their own. His concern, he said, is what they indicate about the behaviour of increasingly capable systems.
“The problem is that these systems are clearly not aligned enough to safely kick off recursive self-improvement,” Engels said.
“Immense harm in 5 years”
Engels said he now believes there is a “terrifying chance” that AI systems could cause widespread harm within the next five years.
(Also read: ‘AI could kill us all by end of decade’: Anthropic researcher quits, issues dire warning)
“I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years. I don’t know the exact probability, but I think it’s high enough to make this the most important problem in the world,” he said.
The former Google AI researcher argued that it is not that AI development should stop altogether, but that its pace should be matched by progress in understanding and controlling AI systems.
“I think we need more time,” Engels said. He called for AI development to be paced so that “capabilities don’t outrun our ability to align models”.