Large-Scale AI Disasters Are No Longer Science Fiction

Large-Scale AI Disasters Are No Longer Science Fiction

In November 2026, the United States experienced a succession of troubling technical failures. Microsoft’s email services suffered a major outage, bank customers lost access to some financial services, Apple’s iCloud and Apple Pay experienced problems, and the National Security Agency’s internal communications network was also affected. The most severe incident occurred on 13 November, when a communications failure prompted the Federal Aviation Administration to halt departures nationwide for about four and a half hours, causing widespread flight cancellations and delays.

These incidents unfolded over just a few days, affecting essential areas including aviation, finance, communications and national security. People began to wonder whether they shared a common cause, or even whether someone had used artificial intelligence to launch a large-scale cyberattack.

On 17 November, The Washington Sun reported that, according to three sources familiar with private discussions in Congress, Wall Street executives and the NSA had told lawmakers that some of the outages were connected to efforts to patch security vulnerabilities discovered by AI. AI models had identified large numbers of serious flaws, prompting organisations to accelerate security updates. The concentrated patching effort, however, exceeded what existing systems could handle and caused service disruptions instead.

This is a startling explanation. People have long worried about AI being used to attack computer systems, but another possibility has now emerged. Even when AI is simply helping people find vulnerabilities and improve cybersecurity, it may indirectly trigger widespread system failures.

At present, however, we cannot conclude that all the incidents over those few days were caused by AI. Investigation material released by the FAA indicates that the aviation failure involved legacy communications infrastructure and abnormal network traffic associated with a new router. There is no evidence that an AI attack caused it. Microsoft and Apple have also offered explanations for their own outages. Meanwhile, Reuters reported that the US Director of National Intelligence linked the NSA’s network problems to patching a PHP security vulnerability previously discovered using AI.

These facts need to be distinguished clearly. The large-scale technical failures happened, and AI is changing how vulnerabilities are discovered and repaired. But the evidence does not yet establish a common AI-related cause behind all the incidents.

Even so, these events reveal a problem that deserves serious attention.

Over recent decades, we have built vast digital infrastructure. Financial transactions, air traffic management, electricity management, healthcare, transport and government communications increasingly depend on complex software systems. These systems have developed over many years and incorporate technologies, equipment and software from different eras. Many older systems still perform critical functions. They can keep running, yet may struggle to withstand frequent and substantial technical changes.

Artificial intelligence is changing this situation.

In the past, finding a serious security vulnerability in complex software might take security specialists weeks or even months of research. Advanced AI models can now analyse large volumes of source code, simulate different attack conditions and identify security problems that were previously difficult to detect. In October 2026, OpenAI announced security research on PHP in which researchers used AI to discover serious vulnerabilities that could allow attackers to execute code remotely. Because PHP is widely used in websites and online services, such flaws can have far-reaching consequences.

Finding these vulnerabilities is clearly a technical advance and an important security achievement. The accompanying problem is that our ability to discover them may be growing faster than existing software maintenance systems can deal with them.

Once a vulnerability is discovered, affected organisations must consider repairing it as soon as possible, since attackers may use AI to find the same flaw. Yet a security patch involves more than replacing a few lines of code. It can affect operating systems, databases, networking equipment, business applications and communications between systems. A patch that works in a test environment may still cause problems in a complex production environment.

If hundreds or thousands of serious vulnerabilities are discovered within a short period, businesses and government agencies face a new dilemma. Patching immediately may cause a system failure, while delaying may leave an opening for attack. The more critical the system, the harder this choice becomes.

This means that, while AI improves our security capabilities, it may also increase the risks inherent in maintenance itself.

A more serious problem arises from the interdependence of modern infrastructure.

Aviation systems depend on communications networks; financial systems depend on data centres and cloud services; hospitals rely on computer networks for diagnosis, treatment and medication management; and commercial activity depends on banking payments and logistics. A failure in one system can propagate through these dependencies.

In July 2024, a faulty software update from cybersecurity company CrowdStrike affected about 8.5 million Windows devices, causing widespread service disruptions in aviation, banking and other industries around the world. AI did not cause that incident, but it demonstrated that an ordinary software error can disrupt modern society across industries and national borders.

By 2026, AI has introduced another variable.

In the past, most system updates, vulnerability analysis and security testing were constrained by the working capacity of human engineers. AI is gradually breaking through that limit. It can continuously analyse large amounts of code, identify more problems in less time and help generate fixes. If AI agents gain broader operational access in future, they may also modify software directly, run tests and even take part in deployment.

This automation can substantially improve efficiency, while also accelerating the pace of system change.

Do human engineers still have enough time to understand every change? Can businesses adequately validate every update? When multiple AI systems maintain different parts of our infrastructure at the same time, does anyone have a sufficient grasp of how their actions affect one another?

Greater AI intelligence will not automatically resolve these questions. An AI system may perform its own local task accurately without understanding what that task means for other systems. Several locally correct actions do not necessarily produce a safe overall result.

This is one of the greatest dangers of complex systems.

An AI responsible for cybersecurity finds a vulnerability and recommends an immediate fix. Another, responsible for software maintenance, installs the update. A third, managing cloud resources, reallocates capacity in response. Each action may satisfy its own task requirements, yet their combined effects may exceed what the system as a whole can withstand.

In such circumstances, a disaster requires neither a malicious AI nor one that suddenly becomes conscious and turns against humanity. Many automated systems operating at speed without adequate coordination may be enough to generate an expanding chain reaction.

This differs considerably from traditional science-fiction portrayals of AI disaster.

People have often imagined AI disasters as stories in which artificial intelligence gains consciousness, develops an independent will and ultimately decides to confront humanity. Such plots readily attract attention, but they focus on whether AI has subjectivity, possesses purposes of its own or will deliberately harm people.

The dangers we face in reality may require none of these conditions.

An AI that follows human instructions entirely and has no independent purposes can still contribute to a disaster. The problem may lie in narrowly defined objectives, dependencies between systems, the speed of automated operations and humanity’s insufficient understanding and control of the overall process.

More concerning still, this risk need not originate in a mistaken technical decision. Actions intended to improve security, repair vulnerabilities or increase efficiency may themselves bring about serious consequences.

Once technical systems become sufficiently large and complex, the soundness of a measure cannot be judged solely by whether it works in one part of the system. We must also consider its effects on the whole and whether other systems can absorb the changes it introduces.

AI is making this existing problem more pronounced.

Human society has developed ways of operating that depend heavily on digital technology. Many essential services no longer have fully independent manual alternatives. When core systems stop working, people may lose timely access to money, transport, communications or healthcare.

We therefore cannot measure AI progress only by the number of problems it solves, the amount of code it generates or the productivity gains it delivers. We must also consider whether society has the capacity to absorb the technical changes involved.

For critical infrastructure, security updates require adequate testing, staged deployment, fault isolation, rollback mechanisms and independent emergency capabilities. In particular, we should not assume that because AI can produce a fix more quickly, the fix can enter production just as quickly. Human engineers’ experience and understanding of complex systems remain indispensable throughout this process.

There is also a deeper social issue.

AI development is driving intense technological and commercial competition. Businesses pursue greater efficiency, lower costs and faster software development and deployment. Governments also hope to use AI to improve public services and national security. In this competitive environment, the faster technical capabilities grow, the greater the pressure on organisations to adopt them quickly.

Yet the reliability of social infrastructure depends on knowledge, experience, institutions and maintenance capabilities accumulated over many years. These conditions cannot simply grow in step with AI computing power and model capabilities.

When the pace of technical change exceeds society’s capacity to absorb and coordinate it, risk continues to accumulate.

The incidents of November 2026 cannot yet be described simply as a single disaster caused by AI. They have, however, shown us the real consequences of widespread technical failure and made the question of how AI can amplify existing system risks concrete and urgent.

Whether AI will eventually become conscious or develop independent purposes remains a philosophical question worth studying. But humanity need not wait for answers before confronting the real dangers AI may cause.

An AI system without consciousness or malice, even one consistently performing reasonable tasks, can contribute through complex technical and social relationships to consequences beyond human control.

The possibility of large-scale AI disasters therefore extends beyond science fiction. The technical conditions, social structures and systemic risks on which they depend are gradually taking shape in the real world.

The question humanity must face is whether, as we become increasingly capable of rapidly changing the technical systems of an entire society, we also have the capacity to understand, coordinate and control those changes.


Discover more from Geoffrey Chen

Subscribe to get the latest posts sent to your email.