No one anticipated that it would feel so typical. There isn’t a dramatic moment, robots marching through urban areas, or a football game interrupted by an emergency broadcast. Rather, it appears as follows: a Manchester logistics manager approving a route that her algorithm had already planned, a radiologist verifying a diagnosis that a model had flagged two minutes prior to opening the file, and a journalist questioning whether the first draft she is reading was written by a human. A slow takeover looks like this, and some of the most reputable scientists in the world now think it’s already happening.
Research labs on both sides of the Atlantic have been making a certain kind of noise in recent weeks. Something much more subdued and unsettling than the typical excitement surrounding the release of new products.
Anthropic’s alignment science lead, Evan Hubinger, publicly stated that he thought there was a greater than 10% chance that the technology his company develops would wipe out humankind within the next ten years. On a conspiracy forum, that is not a fringe figure. A senior scientist at one of the world’s most well-funded AI firms is discussing extinction in a matter-of-fact manner because the math seems to point to it, much like someone discussing interest rates.
According to computer scientist Geoffrey Hinton, whose early work on neural networks may have paved the way for modern AI, his own estimate ranges from 10% to 20%. Hinton was awarded the Nobel Prize. He is not prone to hysteria. The fact that those who are most familiar with these systems appear to be the most terrified of them is truly unsettling.
The Hollywood version of the threat, in which machines develop malevolent intent and plot against their creators, is not what is currently being discussed in Parliament and research circles. The more pragmatic worry is more subtle and perhaps more difficult to counter.
Researchers refer to this as “gradual disempowerment,” a scenario in which artificial intelligence (AI) does not drastically subjugate humanity but rather grows so strong, efficient, and economically appealing that human labor, human judgment, and ultimately human relevance simply become… optional. Imagine a factory floor that is cheaper, faster, and cleaner when no one is using it. Imagine that spreading to government offices, law firms, hospitals, and creative studios.
Gradual disempowerment is a civilization-scale issue, according to Jay Chooi, founder of Robocurve, a company that monitors AI-driven robotics capabilities. He is correct to refer to it as such. The idea, which surfaced from scholarly research in early 2025, contends that rather than machines becoming hostile, the threat is that they will become indispensable while people quietly cease honing the skills that initially made them so. It is not necessary to defeat a species that abandons its critical thinking. It simply fades.
Who is issuing the warnings is what distinguishes this moment from earlier waves of technological alarm. In an appearance before UK lawmakers this month, Beatrice Fihn—who was awarded the Nobel Peace Prize for her efforts to end nuclear weapons—made a clear comparison between the threat posed by nuclear weapons and artificial superintelligence. In the same room, former British Defense Secretary Des Browne made the same analogy. These are not careless speakers.
Thus far, the legislative response has been minimal. Alex Sobel, a Labor MP, has proposed a bill that would completely forbid the development of artificial superintelligence. In a letter to the UN and the prime minister, Darren Jones called for a multinational treaty that would provide a framework for managed development rather than a complete prohibition.
Anthropic’s most recent model, Mythos 5.1, has reportedly not been made available to the UK’s AI Security Institute for pre-release safety testing. That fact alone begs the question, “What exactly does oversight mean if the companies creating these systems won’t submit to independent scrutiny?”
Whether any of the suggested regulatory frameworks could significantly slow down a technology that is already ingrained in financial markets, healthcare systems, legal procedures, and military planning is still up for debate. It makes sense that OpenAI’s chief scientist called for international coordination last week. Additionally, historically, when a country’s competitive advantage is at stake, it is the kind of thing that countries agree to in theory but disregard in reality.
As all of this is happening, it seems like the world is having a crucial discussion five minutes later than it ought to. The machines are already writing the initial drafts, running the routes, and identifying any irregularities. The question of whether AI has already taken over the world and whether anyone is truly paying attention may be more pressing than the question of whether it will.
FAQs
1. What do leading AI scientists actually believe about AI risk?
Some estimate a 10–20% chance AI could cause human extinction within a decade.
2. What is “gradual disempowerment”?
It’s when AI slowly replaces human labor, judgment, and creativity until people become irrelevant.
3. Why is Anthropic’s Mythos 5.1 model controversial?
Anthropic refused to submit it to the UK’s AI Safety Institute for safety testing.
4. Is any government taking legislative action against superintelligent AI?
UK MP Alex Sobel has tabled a bill to ban artificial superintelligence development.
5. Does AI need to turn hostile to pose an existential threat?
No — becoming economically indispensable while humans lose critical skills is dangerous enough.
