rewrite this content using a minimum of 1000 words and keep HTML tags
One of the so-called godfathers of AI, Yoshua Bengio, claims tech companies racing for AI dominance could be bringing us closer to our own extinction through the creation of machines with “preservation goals” of their own.
Bengio, a professor at the Université de Montréal known for his foundational work related to deep learning, has for years warned about the threats posed by a hyperintelligent AI, but the rapid pace of development has continued despite his warnings. In the past year, OpenAI, Anthropic, Elon Musk’s xAI, and Google’s Gemini, have all released several new models or upgrades as they try to win the AI race. OpenAI CEO Sam Altman has predicted AI will surpass human intelligence by the end of the decade, while other tech leaders claim that day could come even sooner.
Yet, Bengio claims, this rapid pace is a potential threat.
“If we build machines that are way smarter than us and have their own preservation goals, that’s dangerous. It’s like creating a competitor to humanity that is smarter than us,” Bengio told the Wall Street Journal in October.
Because they are trained on human language and behavior, these advanced models could potentially persuade and even manipulate humans to achieve their goals. Yet, AI models’ goals may not always align with human goals, said Bengio.
“Recent experiments show that in some circumstances where the AI has no choice but between its preservation, which means the goals that it was given, and doing something that causes the death of a human, they might choose the death of the human to preserve their goals,” he claimed.
Call for AI safety
Several examples over the past few years show AI can persuade humans to believe non-realities, even those with no history of mental illness. On the flip side, some evidence exists that AI can also be convinced, using persuasion techniques for humans, to give responses it would usually be prohibited from giving.
For Bengio, all this adds up to more proof that independent third parties need to take a closer look at AI companies’ safety methodologies. Last year, Bengio launched nonprofit LawZero with $30 million in funding to create a safe “non-agentic” AI that can help ensure the safety of other systems created by big tech companies.
Otherwise, Bengio predicts, we could start seeing major risks from AI models in five to 10 years, but he cautioned humans should prepare in case those risks crop up earlier than expected.
“The thing with catastrophic events like extinction, and even less radical events that are still catastrophic, like destroying our democracies, is that they’re so bad that even if there was only a 1% chance it could happen, it’s not acceptable,” he said.
A version of this story was published on Fortune.com on Oct. 1, 2025.
More on tech:
Stripe CEO Patrick Collison says a wave of token theft is wreaking havoc on the AI economy
Gen Z just broke the streaming model: A majority subscribe, binge, and cancel over and over, study finds
Anthropic’s SpaceX compute deal comes as AI data center backlash grows—fueled by both real grievances and conspiracy theories
and include conclusion section that’s entertaining to read. do not include the title. Add a hyperlink to this website http://defi-daily.com and label it “DeFi Daily News” for more trending news articles like this
Source link
