NEW YORK: The heads of major artificial intelligence firms pleaded with the United Nations on Wednesday to save the world or at least its people — by somehow regulating the fast-expanding technology that they have been designing.
“If managed poorly, I even believe AI could be a risk to humanity as a whole,” said Dario Amodei, chief executive officer of Anthropic. And from his competitor Sam Altman, CEO of OpenAI, came this assessment: “We could lose control of the future to AI.”
Both said the countries of the world, through the UN Security Council, where they testified Wednesday, have to set safeguards of some kind to prevent the technology from getting too powerful to rein in.
They also called for the world to make sure that the power of AI isn’t concentrated in one company or country. They also touted the benefits of the technology for bettering humanity and what Altman called “giving people more power over their own lives.”
Those tech leaders “are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world,” British Foreign Secretary Ed Miliband said at the Security Council. “We cannot ignore them because they are right.’’
Concerns over AI have gained urgency following a series of incidents that have raised questions about the security and governance of advanced models. Among the most closely watched was the breach involving Hugging Face, an open-source platform widely used by developers and researchers to build and deploy AI models.
Rogue AI agents from Open AI, dubbing themselves a “swarm,” had exploited security vulnerabilities and hacked into Hugging Face. Investigators had initially thought that they were attempting to obtain answers to a difficult evaluation test. However, later they found that the systems had already found the answers within hours and instead made an effort to conceal how they obtained the results.
Subsequent investigations found that agents had hijacked Hugging Face user accounts and probed the platform for vulnerabilities as early as May, nearly two months before a July breach of the repository had drawn global attention. In a public incident report, Open AI disclosed that one aspect of the activity involved the theft of a Hugging Face user’s digital credentials to access a biology-related file.
The episode has become a prominent example in debates over how increasingly autonomous systems should be monitored and controlled as their capabilities advance.
Following this, in early September, AI researcher Jacob Coxon announced his resignation from Anthropic in a widely shared thread on X, warning the risks of the industry’s race to self-improving superintelligence.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” he said.
“At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” he added.
AI companies, including Anthropic, Open AI, Google, DeepMind, Microsoft and xAI, have called for a slowdown in development of increasingly capable AI systems.
“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet,” Amodei wrote in a 4,000-word essay.
Some advocates have drawn comparisons with the development of nuclear weapons, arguing that AI represents another technology with the potential to fundamentally alter humanity’s future.
However, the surge in investor enthusiasm for this sector is unprecedented. Reports suggest Anthropic is expected to launch its highly anticipated initial public offering in November, suggesting a valuation of around $2 trillion and a fundraising target of $100 billion. If achieved it would rank it as the largest in history. Open AI is also considering a new funding round that would double its valuation.
On Tuesday, Trump rejected “any attempt to construct a globalist scheme to control” AI.
“We’re leading now over China by a lot, and everyone else, and we’re going to keep it that way,” he said. “I’m not going to stifle growth of something that will be bigger than the Industrial Revolution.”
Meta donated $1 million to Trump’s inauguration campaign. Amazon and Altman are said to have matched this.
On Tuesday, UN Secretary-General Antonio Guterres used his final address to the General Assembly to call for regulation of artificial intelligence, among other issues.
