WASHINGTONK AI Could Wipe Out Humanity within the next decade, according to a senior artificial intelligence safety researcher who has warned that the rapid development of increasingly capable systems could create unprecedented risks. Evan Hubinger, an AI alignment researcher at Anthropic, said there is more than a 10% chance that advanced AI could eventually pose an existential threat to humanity over the next ten years.
Hubinger’s warning comes amid growing debate inside the technology industry over how quickly artificial intelligence is advancing and whether safety research is keeping pace with increasingly powerful models.
The researcher said current AI systems do not represent an immediate threat to humanity. However, he expressed concern about the possibility of recursive self-improvement, in which future AI systems could potentially improve their own capabilities and contribute to the development of even more advanced models.
According to Hubinger, such a development could create significant challenges for researchers attempting to maintain human control over increasingly autonomous systems.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger wrote in a post on X.
AI alignment refers to the effort to ensure that artificial intelligence systems behave according to human intentions, values and safety requirements. Researchers in the field have increasingly focused on the problem as AI models become more capable of reasoning, planning and operating with limited human intervention.
AI Could Wipe Out Humanity as Safety Debate Intensifies
The warning comes shortly after Jacob Coxon, another researcher associated with Anthropic who previously worked at OpenAI, announced his departure from the company. Coxon criticised the direction of the industry, arguing that major technology companies are placing intense emphasis on competition and commercial development while potentially giving insufficient attention to long-term safety.
He warned that leading AI developers could be moving rapidly toward self-improving superintelligence without fully understanding the consequences.
The concerns reflect a growing divide within the artificial intelligence industry. Technology companies are investing enormous sums in AI development, while researchers and safety advocates are calling for stronger safeguards before systems become significantly more autonomous.
One of the central concerns is that future AI systems could become capable of improving their own architecture, developing new capabilities and operating at a speed that humans struggle to monitor. Researchers fear that if such systems are not properly aligned with human goals, controlling them could become increasingly difficult.
At the same time, proponents of continued AI development argue that advanced technology could provide major benefits across science, medicine, education, productivity and other fields. The challenge for policymakers and researchers is therefore to balance those potential benefits against emerging safety risks.
Concerns Grow Over AI Safety Testing
The latest warnings have also emerged amid concerns about whether governments and independent organisations are receiving enough access to the most advanced AI systems for safety testing.
The Financial Times has reported that Anthropic did not provide its latest top model to the UK’s AI Safety Institute for evaluation. The institute is among the organisations working to assess potential risks associated with advanced artificial intelligence.
The UK Cabinet Office has said it continues to work closely with companies and other industry partners. However, researchers have suggested that growing competition among technology companies and geopolitical pressure, including competition with China, could make AI developers more reluctant to share information about their most advanced systems.
Also Read: Gwadar University aims to promote Artificial Intelligence
Access to models for independent evaluation has become an increasingly important issue as governments attempt to develop appropriate AI regulations.
Safety researchers argue that external testing can help identify vulnerabilities and dangerous capabilities before systems are widely deployed. Technology companies, meanwhile, face commercial and security considerations when deciding how much information about their models should be disclosed.
Autonomous AI Raises New Security Risks
Another major concern involves autonomous AI agents. Unlike traditional software that performs narrowly defined tasks, autonomous AI systems can potentially plan actions, use digital tools and complete complex objectives with limited human supervision.
Researchers have warned that these capabilities could be misused for cyberattacks and other harmful activities. A sufficiently capable autonomous system could potentially automate tasks that previously required significant human expertise.
This has increased calls for governments to establish international standards governing the development and deployment of highly capable AI systems.
Some scientists and safety experts believe international cooperation will be essential because artificial intelligence development is not confined to one country. If companies and governments compete to build increasingly powerful systems without common safety standards, researchers fear that commercial and geopolitical pressure could encourage a race toward more capable technology without adequate safeguards.
The debate is particularly significant because AI development is progressing rapidly. Companies are continuing to invest heavily in computing infrastructure, research talent and increasingly sophisticated models.
Calls Grow for International AI Rules
The warnings from researchers have strengthened calls for governments to establish clearer international rules for advanced artificial intelligence.
Supporters of regulation say governments should introduce safety requirements before systems reach levels of capability that could be difficult to control. Potential measures could include independent testing, transparency requirements, stronger monitoring and international cooperation on high-risk AI development.
Critics of excessive regulation, however, warn that overly restrictive rules could slow innovation and prevent society from benefiting from useful AI applications.
The disagreement is likely to intensify as artificial intelligence becomes more integrated into everyday life and business operations.
For now, Hubinger’s warning highlights the uncertainty surrounding the future development of advanced AI. His assessment does not mean that an existential catastrophe is inevitable, nor does it suggest that current AI systems are capable of wiping out humanity. Instead, it reflects concerns about what could happen if future systems become substantially more capable and autonomous without sufficient safeguards.
As governments, technology companies and researchers debate the next stage of AI development, the central challenge will be finding ways to encourage innovation while ensuring that increasingly powerful systems remain safe, controllable and aligned with human interests.
