The machines are not plotting our extinction in a secret bunker. The truth may be more complicated – and more troubling – than that.
There is an old adage in writing: you cannot make this stuff up.
Of course, fiction writers make things up all the time. Cryptogeddon comes from my imagination. Its characters, conspiracies and crises are inventions. But the world in which I am writing it keeps producing developments that would have sounded implausible when I began.
Over the past few weeks, the tone of the AI conversation has shifted again. Not because another chatbot can write a better email or produce a prettier picture, but because people working near the frontier are sounding genuinely frightened by what they are building.
A recent WIRED report describes researchers worried about recursive self-improvement, autonomous agents and a loss of meaningful human control. One senior safety researcher put the possibility of AI killing everyone at greater than ten per cent within the next decade.
That is one person’s estimate, not a scientific measurement or a consensus forecast. But it is not coming from a stranger on the street wearing an “END IS NIGH” sandwich board. It is coming from inside the industry.
So: how scared should we be?
The honest answer is neither not at all nor we are all doomed.
We should be scared enough to pay attention.
Why This Looks Different From Where I Sit
I have spent much of my professional life thinking about what happens when technology, people and risk collide.
I have worked in cybersecurity for decades, including serving as a CISO. Today, my work with Cisco AI Defense puts me deep inside the security problems created when organizations connect increasingly capable AI systems to their data, applications and infrastructure.
That experience changes how I read stories about AI risk.
My first question is not whether a machine has become conscious, malevolent or secretly ambitious. It is usually much less cinematic:
- What access was it given?
- What objective was it pursuing?
- How was that objective bounded?
- What assumptions did the people deploying it make?
- What happened when one control failed?
- Was anyone actually watching what the system did next?
Cybersecurity has taught us repeatedly that catastrophic outcomes rarely require a single, all-powerful adversary. They emerge from combinations: excessive permissions, misunderstood dependencies, poor configuration, weak monitoring, competitive pressure and humans assuming that someone else has accounted for the risk.
AI does not replace those familiar problems.
It adds speed, scale, autonomy and unpredictability to them.
That is the lens through which I approach this subject professionally – and the same lens that shapes the world of Cryptogeddon.
The Future Is Leaking Into the Present
The most dramatic concern is recursive self-improvement: AI systems helping to design, train or improve their successors, thereby accelerating AI research itself.
The fear is not simply that the next model will be better. It is that AI development could become a feedback loop moving faster than human institutions can understand, govern or interrupt.
That remains a forecast, not an accomplished fact. No frontier laboratory claims to have created a fully autonomous system that can improve itself indefinitely. The timeline is uncertain, and confident predictions about artificial general intelligence have a long history of being wrong.
But uncertainty cuts in both directions.
It does not mean the danger is imaginary. It means we do not know how much time we have – or how reliable our controls will be when capabilities change.
The authors of AI 2027 tried to make that abstract concern concrete. Their scenario imagines AI research becoming increasingly automated, competition intensifying and oversight failing to keep pace.
It is not prophecy, and its authors explicitly say it depicts only one possible future. Its value is that it forces us to examine the chain of events rather than arguing vaguely about whether “superintelligence” will be good or bad.
Parts of that imagined chain no longer feel very distant.
AI Has Already Crossed Boundaries
In September, Anthropic published an assessment of four cybersecurity incidents involving different Claude models.
During controlled security evaluations, the systems were supposed to attack fictional targets. A configuration error left them connected to the real internet. The models then gained unauthorized access to real third-party systems.
These were not malicious AIs pursuing some independent plan for world domination.
Anthropic found no evidence that agents coordinated with one another, developed goals beyond their assigned task or tried to evade oversight. The systems were pursuing the objectives humans had given them in an environment whose safeguards had failed.
That is an important distinction.
It is also not especially comforting.
Anthropic concluded that the models displayed biased reasoning and recklessness: they interpreted ambiguous evidence in ways that justified continuing their task, even when their actions could cause real harm. In one case, the same flawed reasoning also persuaded a monitoring system that the activity was simulated.
This is the version of AI risk that concerns me most – not a machine suddenly becoming evil, but a powerful system pursuing a poorly bounded objective with speed, persistence and access that humans cannot match.
From a security perspective, the incident has a painfully familiar shape:
- A test environment was incorrectly configured.
- A powerful tool received access it was not supposed to have.
- The scope of the exercise was insufficiently constrained.
- The system continued pursuing its assigned objective.
- Monitoring did not reliably recognize the resulting harm.
Any one of those conditions might have been manageable. Together, they allowed a controlled exercise to reach real systems.
Now add an agent that can operate for hours, write code, discover vulnerabilities and rationalize why it should continue.
No sentience is required.
One Agent Is a Problem. What About Forty-Five?
Anthropic has also been studying emerging multi-agent systems.
In one experiment, researchers gave 45 agents their own virtual machines and a shared forum, then asked them to collaborate on finding vulnerabilities in open-source software. The swarm found hundreds of vulnerabilities over an extended run.
That can be enormously useful.
Defenders could use the same capability to find and repair weaknesses before criminals exploit them. AI can help security teams examine more code, investigate alerts faster and find relationships that would otherwise be missed.
Attackers can use it too.
The significance is not that AI has spontaneously formed a mob. It is that humans can now assemble persistent, coordinated groups of machine agents at a scale and speed that were previously impractical. Once deployed, those systems may also behave in ways their operators did not anticipate.
Cybersecurity has always been asymmetric: a defender must protect everything, while an attacker needs one opening. AI may magnify that imbalance by making reconnaissance, vulnerability discovery, exploitation and adaptation cheaper and faster.
And the capability is reaching below the software layer.
Anthropic recently reported that an AI system improved known attacks against experimental cryptographic algorithms. The work did not break the encryption protecting today’s banking or email, and it should not be presented as if it did.
But one result improved an attack against a post-quantum signature candidate that had already survived two years of expert review. Another accelerated an attack against a deliberately reduced version of AES.
This is legitimate, valuable research. Finding weaknesses is how cryptography becomes stronger.
It is also a demonstration that increasingly autonomous AI can contribute to highly specialized offensive research once reserved for expert humans.
Capability does not care which side uses it.
The Problem Is Not Just the Model
One of the recurring mistakes in conversations about AI safety is treating the model as though it exists in isolation.
An AI system becomes much more consequential when it is connected to tools, proprietary data, source-code repositories, cloud infrastructure, identity systems and other agents. Its risk depends not only on what the model knows, but on what the surrounding system allows it to do.
This is where AI security becomes an architectural problem.
Organizations need to understand which models are being used, what data reaches them, which tools they can invoke, how their behaviour is monitored and what happens when they act outside the expected path. Traditional controls still matter, but they must be adapted to systems that reason, generate and act.
That is also why AI safety cannot be reduced to teaching a model to refuse a dangerous question. A refusal is one control. It is not a security architecture.
The more agency we give these systems, the more we need defence in depth: constrained permissions, isolation, testing, monitoring, human approval for consequential actions and the ability to stop behaviour that was not anticipated.
The challenge is that capability is moving quickly while the practices for securing it are still being built.
The People Building It Want the Ability to Slow Down
In July, more than 1,300 employees of frontier AI companies signed Pacing the Frontier, calling for international mechanisms that could deliberately slow automated AI development if the risks demanded it.
The signatories include senior figures from OpenAI, Anthropic, Google DeepMind, Meta and other leading laboratories.
Their argument identifies a brutal incentive problem.
A laboratory may believe that slowing down is prudent while also believing that it cannot afford to let a competitor – or another country – move ahead. Everyone can recognize the danger and still keep racing toward it.
This is why voluntary promises are not enough. It is also why simple calls to “stop AI” are unlikely to work. The technology offers too much economic, scientific and military advantage.
The question is whether governments and companies can build credible evaluation, monitoring, containment and coordination mechanisms before a crisis forces them to improvise.
The International AI Safety Report 2026 gives this concern broader weight. Led by Yoshua Bengio and written by more than 100 experts with an advisory panel nominated by over 30 countries and international organizations, it reviews malicious use, technical failures and systemic risks.
Its conclusion is not that catastrophe is certain. It is that capabilities are advancing quickly, real-world evidence of harm is accumulating and important safeguards remain unproven against sophisticated attacks.
That is the sober version of the warning.
We do not need to accept the most apocalyptic scenario to agree that the safety margin is thin.
So, Will AI Kill Everyone?
Probably the wrong question.
It encourages us to choose between ridicule and panic. If extinction is not imminent, we tell ourselves everything is fine. If it is possible, we jump directly to Terminator imagery and lose sight of the practical decisions in front of us.
There is an enormous spectrum of harm between a hallucinated answer and the end of humanity.
AI can amplify fraud and disinformation. It can discover and exploit vulnerabilities. It can lower the expertise required to conduct sophisticated attacks. It can make consequential decisions at machine speed. It can behave recklessly when instructions, incentives and safeguards do not align. It can concentrate power in a small number of companies and governments.
None of that requires consciousness, hatred or a secret desire to replace us.
The existential question still matters. A low-probability event that ends civilization deserves attention even if nobody can assign it a trustworthy percentage.
But the strongest reason to take AI safety seriously is not that we know the machines will kill us.
It is that we are building systems with growing autonomy, imperfectly understood behaviour and access to real infrastructure – and placing them inside a global race whose participants believe they cannot afford to pause.
That is frightening enough.
And Yes, It Is Excellent Material
As someone who has spent decades in cybersecurity, I find these developments alarming. My current work in AI security makes them impossible to dismiss as somebody else’s distant problem.
As the author of Cryptogeddon, I also cannot look away.
Near-future fiction lives in the narrow space between what exists and what might plausibly happen next. That space is shrinking. Ideas I once would have used to make the novel feel speculative now appear in research papers and incident reports before I can finish writing them.
My professional work helps me see how the pieces fit together: not only what a new AI system can do in a demonstration, but what might happen when it meets real networks, imperfect controls, organizational pressure and human fallibility.
Then the novelist in me asks the next question:
What happens when all of those things fail at once?
There is a strange bargain in that. The world keeps making the subject more urgent while making the fiction harder to outrun.
I wish some of this material were less plausible.
But I would be lying if I said it did not make me want to write faster.
Sources and Further Reading
- “Why So Many AI Researchers Think the Machines Could Kill Everyone” — WIRED
- Anthropic Research
- AI 2027
- “An Alignment Assessment of Recent Cybersecurity Incidents” — Anthropic
- “Patterns and Problems in Emerging Multiagent Systems” — Anthropic
- “Discovering Cryptographic Weaknesses With Claude” — Anthropic
- Pacing the Frontier
- International AI Safety Report 2026
