While governments today are paranoid about LLMs breaking security, my prediction is that in a few years, they’ll be more worried about LLMs enhancing security.
Today, these models are indeed quite good at compromising legacy systems. I’m actually surprised that we haven’t seen more high-impact LLM-assisted attacks. Most of the recent highly publicized ones have been low-skill attacks on very soft targets. These early sorties have been more GPT2-level than Claude Mythos, but there’s no question that bigger hacks are coming. Ultimately, though, the fight won’t be about attacks but about defenses.
Security Improves
After a (potentially brutal) transition period, AI is going to produce unimaginable security gains, for two reasons. First, these models really are excellent at finding security vulnerabilities, and there are a finite (if huge) number of them in any given codebase. Second, large software companies are waking up to security vulnerabilities in a way we have not seen in decades.
Writing secure software is incredibly difficult. The “position of the interior” problem means that the authors need to be perfect while the attackers just need to find one mistake. There are structural changes that help—writing in memory-safe languages is a big one—but most of the software industry runs on decades-old code that can’t just be rewritten in Rust overnight. Even then, not all vulnerabilities are memory vulnerabilities, and mistakes are inevitable. LLMs, though, can hammer away at code at superhuman speed and superhuman endurance. Their eyes never glaze over, and their persistence is only limited by how many tokens you want to spend (although, in a very amusing parallel to humans, they do sometimes try to give up early unless you push them).
If writing secure software becomes a game of who can spend more tokens, big companies in rich countries are going to have an enormous advantage. On the other hand, if open-weight models catch up fully, then everything will just asymptote towards the “no bugs” line. Nothing is ever perfect, but we will get very close, at least for critical products. Granted, these upgrades will still need to get deployed, so fear not, water-utility–hating script-kiddies: you have several years of fun left.
While the ability of AI to find flaws is critical, the threat of its success is also driving change. We are seeing the largest tech companies take security seriously for the first time in decades. Microsoft probably has the worst reputation in the industry, being somewhat infamous among the giant companies for shipping insecure products. Even they, though, have woken up to the problem, patching 570 flaws in July 2026 alone (for comparison, the typical monthly number is about 100). AI is making it increasingly hard to ignore decades of accumulated security-related tech debt, and that debt will, eventually, get paid down.
Can software be too secure?
So, more secure software! Great! Yeah, well, for the most powerful entities in the history of the world, i.e. nation-state governments, this is a big problem. Governments really want to be able to peer inside the electronic communications of their citizens, but—at least in the generally free democracies—they’ve had trouble forcing tech companies to build in backdoors. There was a huge fight in the ’90s that government lost, followed by an FBI skirmish with Apple in 2015 that was resolved because the FBI decided just to hack the phone instead.
While both parties were incentivized to portray that 2015 hack as something of one-off, it really wasn’t. Almost all phones and computers can be compromised if they’re seized while powered on. The technical reasons could fill a book, but the short of it is that post-boot security is almost always just UI. Your phone and computer aren’t really “locked” in any real sense once you type in your PIN/password the first time after boot. The operating system is just displaying an un-dismissable window on top of the interface that blocks you from seeing anything or interacting with the software (hence why nonsense like this is possible if app developers are sloppy enough). You don’t need to build a star-sized quantum computer to break file encryption; you just need to trick the OS into dismissing the lock screen. Think of it like distracting a security guard so you can slip in through the unlocked door right behind him.
While these attacks are well beyond the reach of the average person, they’re not especially hard to pull off, either. Companies like Cellebrite and Graykey sell commercial products to governments that perform these attacks all day. These tools just hammer away at the OS, trying to exploit various kernel and driver vulnerabilities until they get a hit.
Notably, there are very effective ways to block these sorts of exploits, especially on phones.[1] The USB port is an especially weak spot: any time something is connected, an enormous number of drivers and modules are loaded to determine what the connected device is and how to interact with it; many of them are exploitable. Cutting off the USB port when the screen is locked is an extraordinarily effective security measure, but when Apple shipped this, they included an inexplicable hour-long delay, as if people routinely plug in headphones 45 minutes after using their phone and don’t unlock the screen afterwards. Unsurprisingly, the delay was almost immediately exploited to extend the window indefinitely.
Another bit of low-hanging fruit is automatically rebooting the phone after some period of inactivity so that data are protected via encryption instead of UI. Apple and Google (if Advanced Protection Mode is enabled) actually do this . . . after 72 hours. Considering that the average American unlocks their phone approximately once every five minutes to check TikTok and sleeps for seven hours, this seems kind of unnecessarily long, no? Like, by an order of magnitude?
The unspoken reality is that there has been a bit of a détente between government and big tech. They’re permitted to make their products pretty secure, but not too secure. As soon as a major bug is publicly revealed or discovered, they’ll patch it, but everyone knows there are a million more in the code, standing by ready and willing to report for duty when needed. Some especially important ones are often quietly left alone—the so-called bugdoors—until they are accidentally made public or generate bad PR (it’s hard to prove one way or the other, of course, but good recent bugdoor candidates are Apple’s suspicious test registers and the absolutely wild secret notifications plist).
To the extent that major structural changes that block entire classes of exploits are ever implemented, the terms are quietly negotiated. You can imagine the discussion that led to the 72-hour reboot compromise: “come on guys, if it really belongs to a terrorist, you can get the phone to Fort Meade within three days, right?”
This uneasy truce has held for decades, but it is hard to see how it can continue. The decision about whether to implement a security feature or exactly how many hours a timer will countdown are things that can reasonably be ascribed to judgment, while declining to patch a known flaw is harder to justify, especially when that flaw could be exploited by anyone with access to a frontier (or less) model and not just the NSA. “We just never noticed that” is going to become an increasingly untenable defense.
The Fallout
Like everything with AI, it is hard to predict where we end up. That there will be incredible pressure on software companies to retain vulnerabilities is all but guaranteed. That such pressure will be increasingly less effective for the reasons above also seems to me inevitable.
There seem to be two paths forward: one, government gives up; or two, governments pass formal backdoor legislation that mandates some 21st century version of the Clipper Chip. Option 1 seems fanciful (when has a government ever ceded power?), and option 2 would be an absolute catastrophe both from the perspective of civil rights and cybersecurity. Backdoors inevitably get exploited by bad actors (theory here and here; evidence here). Code that only runs for “good guys” is mythical, and these debates were had and settled decades ago. In the world of adamantine AI-powered security and broken political institutions, though, I suspect that those arguments will carry very little weight.
So, irony of ironies: AI is actually going to destroy computer security. But not like you thought it would!
© 2026 Restive®, Inc.
