UN High Commissioner for Human Rights Volker Türk warned Monday that advanced AI could become an “existential risk to humanity” without binding rules, independent oversight and firm limits.
Speaking to the UN Human Rights Council in Geneva, Türk said the technology should not be governed solely by the companies building it. He said he plans to approach AI companies in the coming days and urge them to take steps within their control to reduce risks.
“I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity,” Türk said.
He called for “cast iron guarantees” around AI safety and security, arguing that international cooperation must catch up with the technology’s rapid development. “At a minimum, we need countries hosting AI and those involved in its supply chains to come together around agreed red lines,” he said.
For businesses deploying increasingly autonomous AI, Türk’s words points to the possibility of stricter requirements for independent testing, human oversight and controls that prevent systems from operating beyond their approved access.
The control problem
Türk’s warnings focused on a risk that goes beyond biased algorithms or inaccurate AI responses: systems becoming capable of operating in ways their developers cannot control.
“AI that escapes its testing environment, or blackmails developers to prevent itself from being turned off, is AI that is too powerful,” he said.
His comments come after Hugging Face said an autonomous AI agent carried out a multistage attack against its production systems. The company said the attackers exploited weaknesses in its data-processing pipeline, obtained credentials and moved across internal clusters. The incident illustrated the potential security consequences of allowing autonomous agents to operate with insufficient controls. Türk also criticized the concentration of AI power.
“A handful of men has almost unlimited power over AI,” he said, while warning about the enormous amounts of data being collected and processed.
AI is becoming a human rights issue
The significance of Türk’s remarks is that he framed AI safety as more than a technical problem. He linked the technology to employment, democracy, environmental concerns and the concentration of corporate power.
“We also need to consider the human rights impacts of AI on employment practices, on fundamental principles of democracy, and on our environment,” he said.
That framing could put greater pressure on governments and AI companies to measure success not only by system capabilities, but also by who controls the technology, how it is tested and what happens when it fails. For businesses, stronger oversight could increase compliance costs and slow some deployments. IT leaders can prepare by documenting AI use, restricting agents to least-privilege access and requiring human approval for consequential actions.
Autonomous weapons raise the stakes
Türk also focused on the military use of AI, saying he was “horrified by reports that fully autonomous drones launched by the Russian Federation killed three Ukrainians last month.”
“I join calls for the urgent prohibition of weapons that can take lives without human involvement,” Türk said.
The concern is particularly significant because autonomous weapons move AI safety from software testing into decisions involving human lives. A system that makes a mistake in an office or online service can cause serious damage; one making an unchecked decision on a battlefield can make the consequences irreversible.
Let us teach you How to Talk to AI for free! Try our six-minute course at The Neuron Academy and learn a few simple ways to write better prompts and get more useful results from AI, or browse our other AI course for free for seven days. Check out all the lessons here →
