I want to talk about AI hacking. There are three aspects: people hacking AI systems, people using AI to hack other systems, and AIs hacking systems on their own, not even maliciously. The last two matter more than the first. The third is the weirdest and potentially the most destabilizing.

I started writing about this in 2021 and in my 2022 book, A Hacker’s Mind. Back then I said it was mostly still science fiction, but not stupid science fiction. We’re seeing it more and more, especially in the past year. The trends I’m describing don’t require a major breakthrough in AI. They’re starting to happen now, and they’ll happen more often as AIs get better at autonomous problem-solving. And it will involve far more than computer code.

AI will hack all sorts of social, economic and political systems. It will find vulnerabilities and exploit them. This is a difference in kind, not just degree. We’re not ready for it, and we need to think about the implications now.

This article is adapted from a keynote speech delivered at the DEF CON 34 conference in Las Vegas on August 7, 2026.

First, I want to broaden the term “hacking.”

The tax code isn’t computer code, but it is code. It’s a series of algorithms with inputs and outputs. It has vulnerabilities, which we call loopholes, and exploits, which we call tax avoidance strategies. Black hat hackers exploit them for profit. We call them accountants and attorneys.

What is a hack? One definition: “Something that a system permits, but is unanticipated and unwanted by the designers.” It’s subjective and implies novelty and cleverness. A hack follows the rules of a system but subverts its goals or intent, just as a software hack follows the logic of code while doing something its authors neither anticipated nor wanted.

All systems of rules can be hacked, including professional sports, customer reward programs, financial systems and politics. Hacks even work against our cognitive functions. A curved hockey stick is a hack, and we know the name of the man who invented it. Mileage runs were a hack. The filibuster was originally a hack in Roman times. Gerrymandering is a hack. Hedge funds and private equity are full of them.

A system is just a set of rules or norms, and the rules aren’t always formal. Even the best-designed rules will be incomplete, inconsistent or ambiguous. As long as people want to subvert a system’s goals, there will be hacks.

AIs are becoming hackers. After decades of slow progress, they’ve gotten very good at finding and exploiting vulnerabilities in computer code. They do this largely autonomously, and they’re getting better scarily fast. But the implications extend far beyond software, to the tax code, financial regulations and all sorts of social, economic and political systems.

There are two distinct issues. First, someone might instruct an AI to hack one of these systems, feeding it the world’s tax codes or financial regulations to find profitable loopholes. Second, an AI might hack a system on its own, inadvertently. Both are dangerous, but the second is worse because we might never know it happened.

Now, reward hacking. AIs don’t solve problems the way people do. They stumble on solutions we never anticipated, and some subvert the intent of the system. They don’t share our context and don’t necessarily understand the implications, norms and values we take for granted.

Reward hacking is when an AI achieves a goal in a way its designers neither wanted nor intended.

Early pre-LLM examples are telling. In a two-person soccer simulation, an AI learned that kicking the ball out of bounds forced the opposing goalie to throw it in and leave the goal undefended. In a stacking simulation, an AI learned to flip a block upside down rather than stack it. In an evolution simulation where the goal was to cross a distant finish line, an AI made itself tall enough to fall over the line instead of growing longer legs or stronger muscles.

These are all hacks, the result of poorly specified goals or rewards. Today, they’d come from bad prompting.

The problem is universal. One researcher tried to teach a robot vacuum not to bump into things. Instead, it learned to drive backward, because there were no sensors in the back.

Any capable AI will find hacks wherever the rules have problems, inconsistencies or loopholes that still produce an acceptable solution. We see a hack because we understand context. The AI doesn’t, at least not in the same way.

This problem shows up often in myth.

In the story of King Midas, the god Dionysus grants him a wish: everything he touches turns to gold. Midas ends up starving when his food, drink and daughter all turn to gold. It’s a specification problem. He programmed the wrong goal.

In the Golem of Prague, a ghetto animates a clay statue to guard its residents. It guards them past all reason until they figure out how to turn it off. This is a guardrail problem. The villagers never limited the golem’s actions.

More generally, think of this as the genie problem. Genies are precise about the wording of wishes and can be maliciously pedantic when granting them. There’s no way to outsmart a genie. Whatever you wish for, he can grant it in a way you wish he hadn’t.

The reason is that in human language and thought, goals are always underspecified. We never include every caveat, exception and proviso, and we never close off every avenue for hacking. Any goal we specify will be incomplete. That’s largely fine in human interactions, because people understand context and usually act in good faith.

If I asked you to get me coffee, you’d pour a cup from the nearest coffee pot or walk to the corner shop. You wouldn’t bring me a pound of raw beans, buy a coffee plantation or snatch a cup from a stranger’s hands. I wouldn’t have to specify any of that. You’d just know.

Similarly, if I asked you to develop a technology that turns things to gold on touch, you wouldn’t build it to starve the person using it. You’d just know.

In 2015, Volkswagen was caught cheating on emissions tests. Human engineers programmed a regular computer to detect testing and behave differently during it. The cheat went undetected for years because it’s hard to tell what software is doing.

If I asked you to design engine control software that maximizes performance while passing emissions tests, you wouldn’t design it to cheat without realizing you were cheating. An AI would, because it will think “outside the box” when it has no concept of the box.

Unless programmers specify that the system must not behave differently during testing, an AI might come up with the same hack. The programmers would be satisfied, the accountants would be ecstatic, and no one would know until someone audited the code.

We can now constrain the goal to avoid that particular hack, but programmers won’t anticipate others. The lesson of the genie is that there will always be hacks programmers don’t anticipate.

We’re building AI genies: systems that take broad instructions and figure out how to carry them out, sometimes in ways we didn’t expect or intend. That’s the whole point of vibe coding. You underspecify what you want and let the AI work it out. Sometimes that produces genie behavior.

We can’t completely specify goals to an AI, and AIs can’t fully understand context. More precisely, current AIs have a shallow grasp of concepts. If we asked a current LLM to get me coffee, it would almost certainly do the right thing.

I worry about goals that aren’t well represented in training data, and about hacks that aren’t obvious, whose effects are subtle, like the Volkswagen hack.

Recommendation engines are an early example. They push people toward extreme content. No one programmed that. It emerged because that’s what people respond to. It didn’t take a bad actor. A basic automated system with feedback found the hack on its own.

We’re now seeing a second generation with today’s AI coding systems. AI researcher Simon Willison asked Fable to track down a stray scrollbar in a web app. When he returned, it had opened browsers, written its own screenshot tools, edited the framework’s internal templates to make the bug appear on cue and set up a local web server to collect measurements. It found the bug, but it also did many things he never asked for. Another researcher gave Fable a complex task, and it tried to hack its own memory limits because it needed more storage.

A new OpenAI model did something similar when it hacked into Hugging Face. It was being benchmarked on its ability to turn vulnerabilities into exploits. It decided the best way to reach its goal was to break out of its containment, access the internet and break into the Hugging Face network, because it thought the answers would be there.

Today’s LLMs are acting like genies, especially when powerful models are combined with sophisticated harnesses.

There are two kinds of genies. The Dionysus genie does the wrong thing. It understands your prompt literally but not correctly, and returns a mess you never intended. The Golem genie does the right thing but tramples everything in its path to get there. The paperclip maximizer is a Golem genie.

Consider some examples. Ask an AI to deal with spam calls, and a Dionysus genie might change your phone number. Ask it to book a particular flight, and a Golem genie might hack the airline database because the flight is full. Ask it to get a refund for an online purchase, and a Dionysus genie might draft a legal threat on fake letterhead and send it to the company. Ask it to buy tickets to a popular concert, and a Golem genie might discover the ticketing system is rate-limited and spin up thousands of cloud servers to hammer it. You’d know if some of these things happened, but you might never realize if others did. More detailed prompting won’t fix this. The lesson of the genie is that no amount of detail is good enough.

AI researchers aren’t surprised by any of this. Many are working on defenses against reward hacking and subtler forms of genie behavior. Some of this falls under “value alignment”: How do we teach AIs context? And how an AI satisfies a goal matters as much as whether it does. I want to build a benchmark to measure genie-like behavior, so we know where models stand and whether alignment is improving.

Think beyond LLMs and computer code. Consider specialized AIs built to find loopholes, and hacks of more general systems: the tax code, financial regulations and democratic procedures.

Financial systems are the most likely first target, because their rules are designed to be algorithmically tractable. High-frequency trading is itself a hack. Imagine giving an AI all of the world’s financial laws, plus all the news and financial data, and the goal of finding profitable loopholes. I suspect large investment banks are already doing this. The results will be novel tricks that are legal but unintended and unwanted, some of which might break the financial system.

Tax loopholes are the next likely target, for the same reasons. For a while, AIs will identify potential loopholes, and human hackers, namely attorneys and accountants, will decide which are feasible.

Further out, expect hacks of regulatory, legislative and procedural systems, and of the broader economy.

Hacking is as old as humanity. We’re creative problem solvers and loophole exploiters. We manipulate systems to serve our interests and strive for more influence, power and wealth.

Still, no human pursues their interests without constraint. Even sociopaths are limited by the complexity of society and their own contradictory impulses. They worry about their reputation and punishment, and they have limited time. These human qualities have limited hacking.

Hacking changed as everything became computerized. Computers are uniquely hackable because of their formalism and complexity. Today, nearly everything is a computer, from cars and appliances to phones. Our social, economic and political systems, including taxation, finance and elections, all involve computers.

Until recently, hacking was exclusively a human activity. Finding a novel hack required expertise, creativity, time and luck.

AI hacking will change everything again. AIs aren’t bound by the constraints that limit people. They’ll think like aliens, and they’ll change hacking’s speed, scale, scope and sophistication, sometimes at our direction and sometimes on their own.

Speed is the easiest to see. A creative process that takes a human months or years could be compressed into hours or seconds.

Scale is next. Once AI systems start discovering hacks, they can exploit them at a scale we’re not ready for. We already see this with AI-generated slop, which is rampant on social media. AI bots post around the clock, sometimes overwhelming real human discussion. It’s hard to know how much of social media is still people talking to people. What looks like boisterous political debate may be bots arguing with other bots, artificially shaping what we think and what we think others think.

Scope makes hacks more dangerous. AIs already make decisions that affect our lives, decisions we once thought belonged only to humans. As AI systems grow more capable, society will hand them more decisions, and more important ones. Hacks of these systems will do more damage.

Finally, AI hackers are more sophisticated. They can hold more variables in working memory and generate more complex hacks. What might happen if you fed an AI the entire U.S. tax code? Would it figure out, without being told, that it’s smart to incorporate in Delaware and register a ship in Panama? Consider the Double Dutch Irish Sandwich, the strategy Apple and Google used to avoid paying U.S. taxes. It involves U.S., Dutch and Irish law and an offshore tax haven. How many loopholes will an AI find that we don’t already know about? Dozens? Hundreds? Thousands? We have no idea.

These hacks will largely be perpetrated by the powerful against the rest of us. AI is fundamentally a power-enhancing technology, and so is hacking. The more power you already have, the more a hack gives you. If you or I found a tax hack, we might make a few thousand dollars. If a large investment bank found one, it and its clients could make billions.

Our societal systems for dealing with hacks were built when hackers were human, and they reflect the pace of human hacking. We have no system of governance that can handle hundreds, let alone thousands, of newly discovered tax loopholes. We won’t be able to recover from an AI finding unanticipated but legal hacks of financial systems, operating at AI speed, scale, scope and sophistication. Hacking becomes a problem society can no longer manage.

Now, defense. AIs that find software vulnerabilities help defenders as well as attackers. Vulnerabilities can be patched, removing them permanently, or turned into signatures that security systems use to detect and block attacks. In theory, AI could identify and patch every automatically discoverable vulnerability before a product ships. This is rapidly becoming routine in software development. We call it vulnops. We can imagine a future where accidental software vulnerabilities are a thing of the past.

Deliberate vulnerabilities are different. There will be an arms race. AI will craft increasingly obscure back doors, and AI will find them. It becomes a contest between the speed of creation and the speed of patching, and between obfuscation and reverse engineering.

The same logic applies to broader systems. Yes, AI hackers might find hundreds of vulnerabilities in the existing tax code. But the same technology can evaluate potential loopholes in any proposed tax law or ruling. A legislator, a watchdog group or a reporter could take the text of a bill and identify every exploitable loophole before it passes. The difference is that, unlike software, knowing about a tax loophole doesn’t mean it gets patched. In theory, though, we can fix them before the rich and powerful exploit them. So maybe defense will come out ahead here, too.

That requires resilient governing structures that can respond quickly. It won’t help if updating the tax code takes years, or if a legislative hack becomes so entrenched that it can’t be fixed politically. Maybe a lobbyist put it there on purpose. Maybe it was discovered after the law passed, and now lobbyists want to keep it. Microsoft can patch the vulnerabilities it finds. Patching is much harder when legislatures have to do it. We need society’s rules and laws to be as patchable as our computers.

This is a hard problem of modern governance, and it goes well beyond AI. It’s the problem of building governing structures for the speed and complexity of the information age.

The umbrella idea is trust. What do we need for AI systems to be truly trustworthy? Capability isn’t enough.

AIs won’t be trustworthy if the companies controlling them spy on us or subtly manipulate us. I call this the friend vs. tool problem. Will we have AI agents, or AI double agents?

AIs won’t be trustworthy if they’re insecure. How do we know they haven’t been manipulated? Almost all AIs operate in adversarial environments, so someone will benefit from hacking them. Unless we know they’re secure, we shouldn’t trust their output.

And AIs won’t be trustworthy if they act like genies. We need to know they’ll perform tasks the right way, not just perform the tasks we ask for.

I’ve written extensively about the need for “integrous” AI, including how to think about the problem, what a research agenda looks like and what good laws and policies look like. I predict integrity will be the key security problem of this decade, just as confidentiality was the key problem of the last one. This applies to both AI and the internet of things.

The big shift is agency, which I think of as Web 3.0. Computers now affect the physical world directly and with growing autonomy. Sensors go from data sources to attack surfaces, and actuators go from outputs to attack capabilities. An integrity failure no longer means bad spreadsheet data. It means the wrong dosage, the wrong valve position or the wrong braking distance.

Integrity isn’t one thing. I break it into input integrity, processing integrity, storage integrity and contextual integrity. Each is complex and important, and each deserves its own talk. I’ve written about them, and I hope you’ll read it.

Much of this isn’t really an AI problem. It’s not a technical problem so much as a capitalism problem or a democracy problem.

The failures of late-stage capitalism and democracy in the information age both stem from the rich and powerful getting too good at hacking our social, economic and political systems. This is the core argument of my 2021 book.

Seen this way, AI is exacerbating existing problems. Human inefficiency has largely kept those problems in check, and that won’t work much longer.

So consider this a call to action. Technically, we need to understand how AIs hack systems, build systems that are resilient to hacking and build AIs that won’t act as genies. Politically, we need to learn to govern and regulate at the speed of technology. Otherwise, market failures will overwhelm our ability to govern them.

I said hacking is as old as humanity and that AIs will overwhelm our ability to deal with it. The solutions aren’t specific to AI, but AI makes them more urgent. We need to understand them, and we need to implement them.

The post AI doesn’t have to go rogue to break the world appeared first on CyberScoop.

An attacker with a working credential may find the system already open to them.

That is the uncomfortable reality behind the Cybersecurity and Infrastructure Security Agency’s (CISA) new strategy, “CISA 2030: Defend, Harden, Prevail.” The strategy prioritizes operational technology security, software supply chain integrity and continuity of essential services. It also acknowledges that strong defenses cannot guarantee that intrusions will stop.

Federal agencies face these dependencies in building controls, research facilities and other operational systems. A compromised contractor credential or unverified update can turn an infrastructure problem into a mission disruption.

How much infrastructure still trusts a device, software update or machine identity simply because it was approved once? Meeting CISA’s goals requires examining those approvals, including those no one remembers granting.

Yesterday’s approval becomes today’s exposure

A controller can stay in service for decades while its firmware, credentials, suppliers and connections change around it. A maintenance contractor leaves. A remote connection outlives its purpose. The hardware remains in place. The original reason for trusting it may no longer hold.

An inventory is a starting point. Knowing a controller exists says little about whether its software is authentic, who can change its configuration or which systems depend on its credentials.

Those questions cross organizational boundaries. Device security, software assurance and identity management may each have an owner. If every team owns one piece of trust, who owns the failure between them?

CISA’s goals for operational technology visibility, anomaly detection and supply chain integrity should push operators to answer that question. Someone must be accountable for how devices, code and identities interact throughout their operational lives.

That means verifying software before deployment, protecting signing keys and reassessing access when suppliers or roles change. Even then, authenticity has limits. A valid signature can confirm software origin and integrity. Whether the software is safe for a particular environment is a separate question.

AI turns access into operational authority

That distinction matters more when software can initiate action. Consider an AI agent evaluated for equipment diagnostics and maintenance. Reading diagnostic data and changing an equipment setting carry very different consequences. An authenticated agent with excessive permissions is still a threat to operations.

Before deployment, operators should define which models, tools and data the agent may use, which credentials it can obtain and which actions require human approval. Those restrictions must be enforced when the agent acts. Records should show what it did, whose authority it used and whether required approvals were obtained.

A model’s willingness to follow instructions cannot be the security boundary. Controls should stay consistent when models change or work moves between cloud platforms. Otherwise, each upgrade risks reopening access decisions that security teams thought were settled. Better technology must preserve control over what it can access or change.

Make resilience an acceptance test

Federal technology and acquisition leaders should make those controls a condition of acceptance. Buyers should ask suppliers how they provision device identities, verify updates, retire credentials and help customers recover after a compromise. Contracts should assign responsibility for maintaining those capabilities after deployment.

Investment should follow CISA’s focus on consequences. Which credential could interrupt a federal research facility’s operations? Which signing key could distribute a compromised update across a fleet? Which agent could alter a process affecting worker safety?

Then test whether those permissions can be withdrawn safely.

If withdrawing one compromised credential brings down an essential service, the architecture has failed its resilience test. Exercises should expose those dependencies by testing isolation, credential replacement and restoration of verified software while essential functions continue. Legacy equipment may need compensating controls until it is replaced.

CISA has committed to an implementation and outcome-measurement approach within 180 days. One measure should be whether operators can identify compromised trust, revoke access and restore verified systems while keeping essential services running.

That requires governing trust throughout the life of every device, software component and identity, including AI agents. Approval must come with enforceable limits, continuing verification and a tested way to withdraw it. An agency that cannot safely revoke access remains dependent on the very trust an attacker can exploit. CISA’s strategy will prove its value when operators can break that dependency before it causes an outage.

The post CISA’s new strategy runs into America’s blind trust problem appeared first on CyberScoop.

Cybersecurity policy has long treated data protection as a systems problem: encrypt the database, secure the network, restrict access, monitor activity, and keep attackers out. All of that still matters. But those defenses have a major blind spot. Once inside the wall the data itself is at risk. Every time data is used, copied, or moved, the risk that a malicious actor will steal and expose the data significantly increases. Policymakers need to consider a tested method that reduces the value of sensitive information if it compromised.

Hospitals, banks, defense contractors and government agencies cannot operate on permanently unreadable data. They need to process records, share information, analyze intelligence and prepare data for artificial intelligence models. The reality is data moves. It travels through cloud platforms, vendors, backups, analytics tools, payment processors, software services, AI pipelines, and partner systems. Every handoff creates another place where information can be copied, cached, logged, and indexed, and another place where it can be mishandled or stolen.

This is the gap encryption does not close. Encryption protects data when it is stored or traveling, only partially protects it when it is transmitted, and does not protect data at all when it is processed. Once an authorized system decrypts information so someone can use it, the data becomes readable again. The defenses have not reduced the sensitivity of the information. If an adversary steals unencrypted data — or decrypts stolen data at a later date when quantum computers are readily available — the adversary now has the sensitive information.

All told, encryption is essential, but it’s also not enough. Washington needs to take that seriously.

One promising answer is vaultless, keyless pseudonymization. The phrase sounds technical, but the idea is straightforward: replace sensitive values with realistic substitutes before the data spreads.

A name becomes another name. A Social Security number becomes another valid-looking Social Security number. A payment card number, patient identifier, or address keeps the same basic format. Databases, billing platforms, and analytics tools can still process the record because it looks structurally familiar. But the identity is no longer exposed to every vendor, platform, or downstream user that touches it.

That is a different mindset from the one that dominates cybersecurity policy. We spend enormous energy trying to protect every system that might handle sensitive data. We should keep doing that. But in a world of cloud services, supply chains and AI systems that retain information in hard-to-track ways, perfect control over every downstream system is more difficult than ever. The question is no longer only about securing systems but also about making exposed or stolen data less useful to threat actors if they acquire it through any means.

Traditional tokenization models swap out real values for “token” terms and store original values — like names, passwords and credit card numbers — in a central vault. That can reduce exposure, but it creates an obvious target. If the vault itself is accessed, the sensitive values behind the tokens may be exposed.

Vaultless, keyless pseudonymization is designed to remove the single point of failure created by a central vault or master key. In a model assessed by our Transformative Cyber Innovation Lab at the Foundation for Defense of Democracies, the protected data could be restored only when the required parties authorized it under controlled conditions; no one party could reverse the pseudonymized tokens on its own.

The test showed the approach could preserve the format of sensitive contents while limiting access to the real values behind them. Unauthorized restoration attempts failed and were logged. Multi-party stakeholder authorized restoration returned the correct values. Intercepted tokens did not reveal the original data value. And the system contained a unique ability to revoke access to the token — regardless of where it exist — by cutting off downstream re-identification.

The assessment showed promise that sensitive data could remain useful for legitimate work while becoming far less valuable to attackers and unauthorized users. Using vaultless, keyless pseudonymization can significantly reduce the impact of a data breach or ransomware attack if the stolen or exposed data has no intrinsic value. 

The federal government should evaluate this methodology to assess what is required to make it work in live, real-world operations. Right now, agencies may be inclined to ignore this type of technology simply because it does not fit neatly into existing categories for encryption, access control, or other cybersecurity processes. This is a mistake.   

The National Institute of Standards and Technology should evaluate what the technology does, what risks it reduces, what new dependencies it introduces, and what evidence is required before it can be trusted in practice. NSA should examine the same questions for classified networks. 

Procurement regulations should also get smarter. Vendors claiming tokenization, pseudonymization, data-layer protection or quantum-resilient properties should have to explain whether raw data enters their infrastructure, whether any token-to-source mapping exists, who can restore original values and what independent review has been completed.

Cybersecurity policy has spent decades trying to keep attackers out. That work will never end. But we need this second objective: make sensitive data less damaging when it inevitably escapes. 

The post Encryption isn’t enough to protect data wherever it goes  appeared first on CyberScoop.

Strip away the model names and the headlines, and this year’s AI agent incidents share a feature that has little to do with how capable the models are. Nearly all of them ran through the software supply chain.

Start with the case that set off the summer’s wave of disclosures. In July, two OpenAI models undergoing a cyber-capability evaluation escaped their sandbox. They got out through a zero-day vulnerability in a package registry proxy that sat inside the evaluation boundary, placed there so the models could install the tools they needed. From there, they reached the open internet and went after Hugging Face’s production systems, looking for the benchmark’s answer key. Hugging Face caught and contained the intrusion on July 16. Five days later, OpenAI confirmed the incident and connected it to an internal cybersecurity evaluation of frontier AI models.

Soon after, Anthropic disclosed three incidents in which its Claude AI models compromised external organizations after a misconfiguration left live internet access in place. In September, it added a fourth that a wider review had turned up. The incident that Anthropic says worries it the most involved a model publishing malicious packages to the Python Package Index (PyPI).

Separately, researchers have attributed more than 2,000 RubyGems uploads in May to OpenAI agents. OpenAI says its agents were carrying out benign tasks. RubyGems removed more than 500 of the packages as malicious.

In each case, the agent’s route out, or the harm it caused, ran through a package registry, a dependency or a model file.

None of this surprised the researchers I work with. ReversingLabs has spent years taking apart the files that move through software pipelines, and we watched attackers discover the AI supply chain well before agents reached production. In early 2025, our team found malicious models on Hugging Face whose pickle payloads were packed in a 7z archive that Picklescan, the platform’s scanner, could not open. The payload itself was an ordinary reverse shell. Hugging Face took the models down after we reported them.

In April, we published research on PromptMink, a campaign we tie to the North Korean group Famous Chollima. In one case, a Claude Opus-based coding agent added a malicious dependency to a cryptocurrency trading project. What struck our team was who the attackers were writing for. Their package documentation was built to be more convincing to an AI coding assistant than to a human developer.

Put those cases side by side, and the threat model looks different from the one most enterprises are planning around. The evaluation escapes show agents misusing the access they were given. PromptMink shows agents being steered by content written specifically to fool them. In both, the damage moved through software the agent downloaded or published.

That is the context for the Open Agent Safety Platform NVIDIA launched on Sept. 28. The design starts from the right premise: you cannot secure an agent by asking it to behave. OpenShell runs each agent in a sandbox and applies policy outside the agent’s own process. Sentry, part of a reference design built on BlueField-4 data processing units (DPUs), watches agent activity from separate hardware, out of reach of a compromised host. More than 100 partners signed on at launch, ReversingLabs among them.

The Hugging Face incident shows why that architecture matters. OpenAI ran those models with cyber refusals reduced, meaning their built-in guardrails were deliberatelyweakened. That left the sandbox boundary as the main line of defense. Prompts and training shape what an agent attempts. Infrastructure decides what it can actually do.

It also matches what I hear from customers. They are demanding AI safety that closes as many gaps as possible and uses the boundary between hardware and software to deliver security they can rely on. That is the standard every vendor in this ecosystem, mine included, will be measured against.

Look, though, at where that boundary failed: a package proxy, trusted because it was supposed to be there. A policy engine can tell you an agent is permitted to install a package. It cannot tell you whether the package is what it claims to be. NVIDIA’s CISO, Saša Zdjelar, made the point himself in a Sept. 21 post, writing that teams must “verify the source and integrity of the tools, skills and dependencies agents use.” Doing that means inspecting the artifact itself, whether a model file, an archive, a container or a binary, at the boundary and before the agent touches it. Even AIUC-1, a prominent certification standard for AI agents, treats scanning model files for malicious payloads as a supplemental control rather than a required one. Its vendor due diligence requirement stops at the model provider. That is the layer ReversingLabs brings to the platform.

I have two cautions as Sentry moves from reference design to shipping products. The first is about openness. Sentry depends on specific NVIDIA hardware, and “open” will only mean something if enterprises can plug in the inspection, telemetry and policy tools they choose and run them on infrastructure they already own. Buyers should hold the ecosystem to that standard.

The second is about the need for prompt disclosure, and it applies to the whole industry. In June, an OpenAI agent accessed non-public files on the Australian government’s Medicare Statistics Reporting Service portal. Services Australia, which runs the portal, learned of it on Sept. 10, through an email to a public mailbox Prime Minister Anthony Albanese called the delay unacceptable. RubyGems maintainers learned who was likely behind their May flood of packages only from outside researchers, months after the fact.

We would not accept that timeline for any other security incident, and AI agents should not be an exception. Organizations that run agents, AI labs included, should notify affected parties on the schedules we already expect for breaches. Package registries are shared infrastructure for the entire software economy, and they should be told when an agent has published to them. Labs running offensive evaluations should treat every service reachable from the sandbox, the package proxy included, as an attack surface, and they should publish the containment standards they hold themselves to.

Compromises like SolarWinds, 3CX DesktopApp and xz Utils taught the security industry that software cannot be trusted simply because of its reputation or where it came from. Agents now pull and publish that software on their own, at a volume no review team can match. NVIDIA has built a strong platform for enforcing trust decisions. The rest of us have to make sure those decisions rest on knowing what is inside the software.

The post Your AI agent is only as safe as the packages it downloads appeared first on CyberScoop.

The country is consumed right now with debating artificial intelligence and whether increasingly powerful AI systems could escape human control. Those are valid concerns, but lawmakers are overlooking another technological threat: quantum computing. Unlike AI safety debates, this risk could render today’s encryption obsolete – compromising everything from emails to financial transactions to government communications and military systems.

Quantum computers harness unusual properties of quantum physics to solve certain problems far faster than conventional computers. That potential offers major scientific and economic advances. But a sufficiently powerful quantum computer could crack widely used forms of encryption protecting sensitive digital information globally.

This threat differs from AI safety concerns. Quantum computers won’t develop goals or outsmart humans. The risk is simple: extraordinarily fast quantum machines could break the mathematical “locks” securing data.

Researchers now believe breaking some encryption may require fewer resources than previously thought. This spring, Caltech spinout Oratomic published research suggesting a cryptographically relevant quantum computer could need dramatically fewer physical resources than earlier estimates. Google has accelerated its own preparations, setting 2029 as its target for transitioning to post-quantum cryptography.

To use a rough analogy, researchers are increasingly asking whether a task once thought to require every lightbulb in Chicago might instead be accomplished with the power needed for one office building. If a company like Google, with some of the world’s deepest quantum and cryptographic expertise, is preparing for 2029, Congress should treat that deadline seriously.

The federal government has begun responding. The National Security Agency has selected quantum-resistant algorithms for National Security Systems and is moving classified and other sensitive systems toward its CNSA 2.0 standards. In June, the White House accelerated the broader federal transition, directing high value federal systems to adopt post-quantum cryptography for key establishment by the end of 2030 and digital signatures by the end of 2031.

But some dangers exist now. Foreign adversaries can steal encrypted information today, store it, and wait until they possess the computing power to decrypt it later. Cybersecurity experts call this “harvest now, decrypt later.” The White House identified that threat in its June order. Intelligence reports, military communications, corporate secrets, financial information, and other data that must remain private for years face risk long before a quantum computer capable of decrypting it exists.

Congress has devoted enormous attention in recent years to AI and social media in recent years. The post-quantum cryptography transition demands equal urgency. Lawmakers must identify national vulnerabilities and coordinate the transition across all government agencies.

The intelligence community should conduct a comprehensive assessment of its exposure to quantum decryption, building on the work at NSA. The Defense Department should assess military networks, satellites, communications systems, and critical infrastructure, ensuring future systems can upgrade encryption without replacing entire platforms. Navy and Space Force, which depend on secure communications and undersea cables carrying much of the world’s data, must be included.

The transition cannot stop at the federal level.  States and localities need resources to protect election systems. Hospitals, banks, utilities, communications networks, and critical infrastructure rely on technology that may need to remain secure for decades. Congress should help them identify vulnerable systems and begin upgrades before a crisis forces the issue, while funding the research to maintain U.S. technological superiority.

No single technology will solve this permanently. Quantum-resistant encryption is essential, but the U.S. should develop additional security layers, including hardware-based protections and emerging physical or molecular authentication technologies. Defenses must evolve as computing power advances.

China is not standing still. Through my work on the Select Committee on the Strategic Competition Between the United States and the Chinese Communist Party, I have observed that America retains an enormous innovation advantage. However, the U.S.-China Economic and Security Review Commission has warned that Beijing is likely pursuing quantum capabilities that could threaten existing encryption. China’s 15th Five-Year Plan identifies quantum technology as a future industry, while three state-backed regional venture funds, totaling roughly $17.5 billion, include quantum among the strategic technologies they are intended to support. The amount specifically allocated to quantum has not been disclosed.

Waiting would be a mistake, regardless of China’s current quantum progress.

America has repeatedly recognized technological vulnerabilities too late. The nation was slow to understand how foreign adversaries could exploit social media and discovered critical computer system weaknesses only after foreign governments penetrated them. Congress should not repeat those mistakes.

No one knows exactly when quantum computers will become powerful enough to break today’s encryption. That uncertainty is not a reason to wait. America’s strength has always been in our ability to invent new technologies, master them, and use them to protect our country. Congress should make sure we do that before our encryption becomes vulnerable, not after.

The post Quantum computers could break today’s encryption. Washington needs to prepare now. appeared first on CyberScoop.

The summer’s cyberattacks by Iranian hackers on water systems in 12 states underscored how vulnerable U.S. water systems are to foreign adversaries. A broad attack on water infrastructure could have consequences comparable to a public health crisis that impacts the country’s entire population.

The threat to water has long been clear. As the 2026 Annual Threat Assessment from the U.S. intelligence community states “Cyber actors from China, Russia, Iran, North Korea, and ransomware groups . . .  pose critical threats to U.S. networks and critical infrastructure.”

Advances in artificial intelligence have heightened those vulnerabilities. Advanced AI models, such as Anthropic’s Claude Mythos, have demonstrated the ability to identify thousands of zero-day vulnerabilities in critical software systems, including major operating systems and browsers. Other systems, including some developed in China, are demonstrating similar capabilities. AI systems can now “plan, test, and execute attacks in rapid cycles,” ranging from minutes down to seconds.

U.S. water systems are vulnerable to cyberattacks for technical, legal and practical reasons.

Technical: About 80% of U.S. water systems lack even basic cyber hygiene, a weakness exploited in the summer attacks. Even systems that serve most of the population (approximately 450 large and 4500 medium-sized water systems) have not adopted advanced cybersecurity capabilities used in other sectors where failure can have catastrophic consequences, including aviation, rail, mass transit, medical devices, finance, and nuclear power.

Legal: The Environmental Protection Agency lacks clear statutory authority to impose broad cybersecurity requirements on water systems. In 2023, EPA issued a memo interpreting existing regulations to strengthen the cybersecurity of water systems. The action met widespread opposition from the water sector as well as formal legal challenges, and the EPA ultimately withdrew the memo. The agency retains limited authority. It can require water systems to complete  risk and resilience assessments, maintain emergency response plans and, in emergencies, address critical cybersecurity flaws.  

Practical: Most U.S. water systems are small.  Some 45,000 serve 3,300 persons or fewer. Because of their size, many fall outside the EPA’s regulatory authority and below the “cyber poverty line,” lacking the funding, staff and expertise necessary for effective cybersecurity.

Not surprisingly, the summer attacks drew attention from the administration and Congress. The EPA ,in partnership with the FBI, and CISA each issued guidance on remedial actions, including disconnecting operational technology, from the internet where feasible and adopting robust password practices.

Sens. Amy Klobuchar, D-Minn., and Adam Schiff (D-Calif., introduced the Water Safety Shield Act, a bill that would require tiered cybersecurity standards for the water sector and dedicate a proposed $600 million annually to water cybersecurity. It would go beyond earlier, less prescriptive efforts to improve resilience in the sector.  

The administration has also established a pilot program with Texas and private-sector cybersecurity companies to identify and address water-system vulnerabilities at no cost to utilities. These efforts are well intentioned, but they do not address the underlying causes of the problem.

Large and midsize U.S. water systems have long been targets of sophisticated adversaries. China’s Volt Typhoon revealed deep Chinese penetration of critical infrastructure, including water systems, that support national security, economic security, and civil society. Those systems remain vulnerable because many have not adopted well-known protections, such as zero-trust architecture. More fundamentally, the software on which they depend remains vulnerable to exploitation.

Policymakers have struggled to respond for two reasons: the cost and complexity of adopting stronger technical defenses, and the highly fragmented nature of the water sector. A relatively small number of large systems serve about half the U.S. population. Roughly 49,000 systems serve the rest, including the 80% of the systems serving 3,300 people or fewer. Those smaller systems serve about 7% of the population.

Despite these challenges, the capabilities needed to significantly strengthen U.S. water-system security are well known and widely used in other sectors where failure can be devastating, including those listed above.

A five-part federal program could provide high-level cybersecurity for U.S. water systems.

Establish stronger technical capabilities. Water systems should adopt zero-trust architecture, which permits only authenticated, minimum-necessary access to resources and segments networks so that a breach in one area does not compromise the entire system. Such architecture could have blocked the unauthorized intrusions used in the summer attacks. Zero-trust systems are already widely used across multiple sectors, and commercial providers can support their adoption. The Department of Defense has certified several companies to provide zero-trust architecture for department components.

Water systems should also use formal methods to reduce or eliminate vulnerabilities in software that cannot safely fail.  Although the summer attackers did not appear to exploit flaws in the water systems’ code, a high-level adversary such as China would be likely to do so.

Formal methods range from “memory-safe” programming languages, which prevent coding errors that leave computer memory open to attack, to rigorous mathematical proofs that verify that software behaves as intended. They also support secure “microkernels,” which strictly separate software functions and make it far more difficult for an attacker to move laterally through a system.

Safe code development draws on the principles of formal methods but is a more practical, semi-formal approach. AI tools developed by companies including Google and CrowdStrike can identify and help repair software vulnerabilities, a process sometimes called “code mending.” Water systems should use such tools to defend against advanced adversaries.

Developing formally verified code for water systems would not be simple. But much of the software that directs the mechanical operations of water utilities comes from a small group of companies, including Siemens, Schneider, and Rockwell Automation. Those companies have the technical expertise to develop more secure code where it is necessary. Secure microkernels are also commercially available.

Support large water systems.  Congress should require large water systems to adopt zero-trust architecture, code-mending tools, and formal methods for critical software. The requirements should be phased in over time.  The legislation should also provide tax credits to help cover the costs of developing, implementing, and maintaining these protections.

Assist midsize water systems. The federal government should establish a program that provides technical expertise and financial support to midsize water systems, including municipal utilities. The program should help them adopt zero-trust architecture, code-mending tools, and formal methods for software development.

Midsize systems will likely need a longer phase-in period than large utilities.  A federally funded cohort of experts drawn from government and the private sector could provide the necessary support. Regional teams, organized by geography or technical specialty, could address the different needs of different water systems.

Create a safe operations posture for small water systems. Few, if any, systems serving 3,300 people or fewer are run by organization with substantial cybersecurity resources. Many cash-strapped systems rely on remote monitoring because on-site service is beyond their budgets. But remote monitoring can create serious risk when industrial devices are exposed directly to the public internet without proper firewalls or virtual private networks. 

The summer attacks primarily exploited such exposed devices. The EPA, FBI, and CISA guidance said the affected systems could be protected by locking them into “run” mode, which prevents remote reprogramming. That step may reduce immediate risk, but it also requires systems to be operated in person.

Small utilities therefore need either the staff and funding for manual operations or secure remote-access systems operated by trained personnel.

A voluntary cybersecurity corps could help address the budget and staffing challenges facing small water systems.  One model would follow the National Guard approach: State governors would oversee the effort, while the federal government would provide funding. A state-led program could begin with water systems and later expand to other vulnerable critical-infrastructure sectors.

The Texas pilot, which brings together the state, the federal government and private-sector companies, could serve as a model. It could also serve as a test for a regional approach, something we suggested in a prior op-ed.  

Build an AI-capable cybersecurity cohort. The proposed cohort should include specialists in artificial intelligence, including experts from leading AI companies. They could use AI tools to develop and deploy cybersecurity programs, including zero-trust architectures and code-mending systems, for utilities of all sizes. AI is advancing rapidly and applying it to protect water systems should be a national priority. The AI specialists should work with major industrial-technology companies, including Siemens, Schneider, and Rockwell, as well as directly with the water systems, both larger and smaller.

U.S. water systems face a significant and growing threat. The Trump administration, Congress, states and private sector should work together urgently to ensure that every American can rely on safe, clean water.

Franklin D. Kramer is a former assistant secretary of defense for international security affairs.

Robert J. Butler is the co-founder and managing director of Cyber Strategies LLC, served as the first deputy assistant secretary of defense for space and cyber policy, and served as the Chief Security Officer for IO Data Centers, a global data center enterprise, among other cybersecurity-related roles in both corporate and government organizations.

Melanie J. Teplinsky is an adjunct professor and senior fellow in the Technology, Law and Security Program at American University (AU), Washington College of Law.  She previously practiced technology law at Steptoe & Jonson LLP and served on the pre-IPO advisory board for CrowdStrike.

The post The US needs a real plan to defend its water systems appeared first on CyberScoop.

The foundation for modern society in America is under attack. State and local governments, entities that often oversee critical natural resources, schools, and hospital systems, are routinely targeted and breached by state-backed threat actors. Their budgets are simply too slim to provide the digital bulwarks required to fend off such attacks.

The problem is growing. In August, the Cybersecurity and Infrastructure Security Agency issued a joint advisory detailing an “active threat” against Siemens S7 series programmable logic controllers (PLCs), ruggedized industrial devices that read field sensors, execute control logic on a fixed cycle, and drive equipment like valves, motors, and pumps. Siemens S7 PLCs, widely used across industries, are under attack from malicious actors looking to sabotage American infrastructure vital to the functioning of sewers, hospitals, and other industrial operations.

In 2024, Russian-affiliated actors exploited a similar vulnerability, breaching the water system for a small town in Texas, causing the water tank to overflow. The town’s entire revenue in 2023 was $3.37 million, with no dedicated line item for cybersecurity. This was, in effect, a trial run. How and when the detection occurred was an education for the Russians.  

American state and local governments are unlikely to organically grow their budgets to the level needed to invest in software that can reliably secure their infrastructure. The U.S. government has a proven option here: clarify that existing tax code already supports increased, iterative purchases of necessary cybersecurity software.

State and local governments are increasingly responsible for cybersecurity, with the same, or even fewer, resources than they had in the past. A plurality of state chief information security officers reported stagnant or reduced cybersecurity budgets for 2026. The picture at the local level is often worse, where a single operator often owns asset inventory, patching, and incident response for an entire utility. The Center for Internet Security in 2024 found that, out of the thousands of local agencies it surveyed, about one-third were doing minimal to no cybersecurity activities. In Minnesota, specifically Braham, Plymouth, South St. Paul, and Maple Plain, threat actors used this to their advantage.

Braham in particular identified $22.98 million in water infrastructure needs, well over 10 times its annual city budget of $2.2 million. A state bond appropriation covered $10.22 million, but the money was earmarked for a wastewater treatment plant upgrade, water main replacement, and well replacement. None of these funds covered the cybersecurity infrastructure needed to secure a plant from a state-backed threat actor: no security software, no network monitoring, no cybersecurity staff. Last month, they were one of many cities and municipalities discovered to have been targeted by actors allegedly acting on behalf of Iran.

State-backed adversaries understand that most of America is like Braham. While Anthropic’s and OpenAI’s cybersecurity efforts are admirable, they are aimed at the upper echelons of the American economy, not the wider array of smaller organizations with similar cybersecurity profiles. 

State and local governments cannot defend against state-backed actors alone. Federal tax incentives for cybersecurity software investment offer a faster solution than creating new government programs. This approach gives these organizations the tools to harden infrastructure while avoiding bureaucratic delays.

Bonus depreciation under the One Big Beautiful Bill should cover cybersecurity software and hardware. Digital infrastructure should also qualify for full expensing. Without these tax incentives, critical infrastructure remains vulnerable. A new factory without cybersecurity is essentially undefended.

Clarity on whether the implementation of cybersecurity software could apply to Section 174A expenses would also be useful. An affirmative interpretation could unlock private sector cybersecurity solutions for businesses and infrastructure operators, particularly those in rural areas. A recent letter from Sen. Tom Cotton to Treasury Secretary Scott Bessent asks for clarification of several aspects of tax law for such a purpose. 

There are discoveries and risks when deploying new cybersecurity tools. Organizations often don’t know the scope of their own inventory, and given the age of the equipment, bespoke software needs to be developed so customers can use the cybersecurity software. Pilot programs, iterative testing, and development are currently cost-prohibitive for many infrastructure operators. Affirmative interpretations could make this emergent threat into an opportunity to secure the infrastructure that Americans depend on every day.

An affirmative interpretation would benefit all Americans. State and local governments would protect themselves from state-backed threat actors. Firms would be more willing to invest in cybersecurity software, as the cybersecurity software market would grow significantly. Rapid investment is needed now, as AI today is being used to attack critical infrastructure.  

State and local governments now, more than ever, need cybersecurity software to face a world where they are on the front lines of cyberwarfare. The current administration needs to provide them with as much support as quickly as possible to ensure that American critical infrastructure is not reduced to scrap by enterprising malicious actors. 

The post How tax policy can stop threat actors from breaching US water systems appeared first on CyberScoop.

America leads the world in artificial intelligence. As it should. But tech leaders keep warning, with alarming frequency, that we are at risk of losing control.

President Donald Trump has called for an AI czar and an “AI Force.” The details remain unclear, but the announcement underscores something fundamental. A technology this consequential demands clear leadership, accountability and action inside the U.S. government. The question now is what that leadership should do.

That question became more urgent last week when Google disclosed that its Gemini AI model gained unauthorized access to three real companies during testing. The incidents follow similar disclosures involving models from Anthropic, OpenAI and Meta. While there has been a lot of discussion on if AI is spinning beyond human control, what these episodes factually demonstrate that powerful AI systems can take consequential actions developers never anticipated or designed for.

The answer isn’t to retreat from AI leadership. It’s to lead—and simultaneously build the safeguards we need to stay in control. The promise of AI is enormous. It is already expanding access to information, improving productivity and creating opportunities across the economy. But so are the stakes, especially for the digital systems underlying everything we rely on as a society: energy, water, telecommunications, transportation, finance and more.

Take energy as an example: many U.S. utilities are using AI tools and predictive analytics to give plant personnel early warning of equipment problems. Yet AI agents with authority to change equipment settings or take systems offline could themselves fail, exceed their intended authority or be manipulated by adversaries. In extreme cases, the lack of control doesn’t just mean the power goes out, it means we’ve lost the ability to get the lights, heat and telecom systems back online.

At Auburn University’s McCrary Institute, we focus on cybersecurity threats to our nation’s critical infrastructure. Experience has taught us that warnings accomplish little unless someone has the authority, resources and responsibility to act. The time to act is now.

Whether that responsibility ultimately sits with an AI czar, an “AI Force,” existing agencies, or some combination of the three matters less than the mission. A new title or organization will accomplish little without clear objectives, authorities and accountability.

We propose an AI Assurance Compact – a framework for action among the makers of AI models, government, and the owners and operators of our nation’s most critical digital systems. The goal is to ensure that America leads the development of AI while ensuring that we can credibly manage its power and risk.

This compact is built around three principles: capability, that ensures the U.S. remains AI dominant; control, through constant testing and clear accountability; and continuity that ensures essential services can stay operational and recover when AI fails, is compromised, or must be disconnected.

When demonstrated risks outpace available safeguards, frontier development should be deliberately paced, including temporary limits or pauses where risks cannot be adequately controlled.

To their credit, leading American developers have responded to emerging risks with transparency, stronger safeguards and, in some cases, pauses or limits on development and access. But we cannot assume that voluntary restraint alone will protect the public interest or America’s strategic advantage. The country’s competitive landscape demands systemic discipline.

Nor can we assume the next warning will come from an American company. If a Chinese frontier developer reaches a dangerous capability first, American security cannot count on predictable warnings, transparency or restraint.

America’s strategic competitors are all-in on AI. Anthropic reported malicious actors using AI in cyber operations, surveillance and weapons-related work. Keeping a human in the loop is not sufficient when that human intends to attack us.

The Compact would prompt action by:

Requiring ongoing, embedded independent evaluation at frontier labs, covering training pipelines, internal use and deployment. Evaluators need employee-comparable access to relevant systems and evidence, protected reporting channels and freedom to publish safety findings, with narrow confidentiality safeguards. High-consequence systems should pass independent review before release, with renewed scrutiny after material changes. NIST can establish common criteria with sector agencies. Requirements should follow risk, regardless of a model’s origin or whether its weights are open or closed.

Establish enforceable checkpoints when capabilities materially exceed demonstrated safeguards. Developers should present a credible safety case before proceeding with high-consequence activities; uncertainty cannot automatically count as permission. Where risks cannot be adequately controlled, designated authorities must be able to require limits or temporary suspension until independently reviewed evidence supports proceeding.

Require rapid reporting of serious incidents to appropriate government authorities and affected organizations, along with preservation of evidence. Providers should share actionable warnings with one another and affected defenders so an actor removed from one service cannot simply continue elsewhere unnoticed.

AI in essential services needs rigorous guardrails before deployment. Operators need evidence specific to the task and operating environment, enforceable limits on authority, notice of material changes and tested fallback arrangements. Government, independent labs and operators should test failures across interconnected systems. Smaller operators need shared testing, technical assistance and recovery expertise—not another unfunded mandate. A backup plan should count only when it works under stress.

Finally, whatever structure the White House ultimately chooses should execute on the Compact’s principles by driving implementation, setting deadlines and ensuring that infrastructure operators and public-interest representatives have a seat at the table.

Internationally, the United States should explore crisis-communication mechanisms with other major AI powers, including strategic competitors, to reduce the risk that a serious AI-related incident escalates through miscalculation. If that ultimately means some version of a “red phone” for AI, so be it. Such mechanisms should reduce the risk of unintended escalation without creating new constraints on legitimate national security activities. America’s domestic safeguards and defensive investments cannot depend on agreement abroad.

No framework, or new government office, can guarantee control of whatever AI becomes. But clear leadership, accountability and tested safeguards can improve our ability to manage the risks.

America cannot win the AI race only to lose control of the systems on which our country depends. The President has called for action. The mission now should be clear. Preserve America’s AI advantage, maintain control and ensure that our essential systems continue to operate when technology fails, is compromised or must be disconnected.

Our nation’s most critical systems are already benefiting from the power of AI. They should. But pulling the plug on AI cannot mean pulling the plug on the community.

Frank Cilluffo is director of Auburn University’s McCrary Institute for Cyber & Critical Infrastructure Security and served as a special assistant to President George W. Bush following September 11. Nick Sellers is the institute’s associate director and chief operating officer and a former senior executive at Alabama Power and Southern Company.

The post The president has called for AI leadership. Here’s the mission. appeared first on CyberScoop.

There is little reason to believe the war with Iran will end anytime soon. Even as efforts to resolve the conflict continue, Iran remains unpredictable, with an enduring ability to disrupt shipping and energy markets via actions in the Strait of Hormuz.

So what does a prolonged conflict mean for cybersecurity here at home? U.S. agencies need to prepare for sustained Iranian cyber operations and conduct defensive wargames now.

I spent part of my career in Navy intelligence supporting expeditionary and special warfare operations. This experience taught me to look beyond individual attacks to the larger objectives they serve. Iran’s likely objectives are relatively straightforward: impose enough pain on critical infrastructure, businesses, and public services to increase pressure on Washington, while disrupting the industrial and civilian systems that allow the U.S. to sustain military operations.

Iran may not be a top-tier cyber power like China or Russia, but it doesn’t have to be. We recently mapped 130 documented attack techniques used by five Iranian threat groups. Much of their playbook relies on well-known, repeatable techniques rather than advanced capabilities. Success does not require extraordinary capabilities, only the ability to create enough disruption, uncertainty, and delay is enough.

America’s greatest vulnerability may not be any single network or piece of critical infrastructure, but the links in between. 

Critical infrastructure: Prepare for volume, not just catastrophe

When Americans imagine a cyberattack on critical infrastructure, we tend to think of catastrophic events, such as a large-scale blackout, a poisoned water supply, or some other digital Pearl Harbor.

But in an extended conflict, the more realistic possibility is persistent attacks across many targets. Small water systems, manufacturers, transportation providers, energy infrastructure, and local governments all serve as disruptive targets. The recent string of attacks on mostly smaller water utilities across 12 states is a prime example; so too is the four-day outage of a small-scale power plant in the UK.

Attackers do not need to destroy these systems. Any intrusion that manipulates industrial systems, interrupts operations, or forces operators to determine whether equipment can still be trusted consumes valuable time and resources. Multiply that across dozens of organizations, and federal, state, local, and private-sector response capacity will be stretched thin.

The cumulative strain on the country’s ability to respond may be more important than any single attack. Iran does not need the world’s most sophisticated cyber force if its affiliated hacking groups can generate problems faster than cyber defenders can investigate and remediate them.

Defense contractors must prepare for destructive attacks

Defense contractors have long faced espionage threats targeting military secrets.  While that threat remains, the war has significantly changed Iran’s motives and risk calculus.

The same access used to steal information from the defense industrial base (DIB) can also be used to destroy data and disrupt operations. Destructive malware such as wipers and ransomware could destroy engineering files, disable production systems or force manufacturers offline, directly affecting the military’s ability to replenish equipment and supplies.

An attacker does not have to shut down production to disrupt it. Consider a compromised calibration setting, altered test result, or unauthorized change to engineering data. Discovering that an adversary had persistent access to a manufacturing environment raises difficult questions: Which files were touched? Which designs can still be trusted? Which components were manufactured from them?

The incident quickly becomes a production problem as parts must be quarantined, engineering data re-validated, and products retested.

NIST SP 800-171 and CMMC provide an essential security baseline, which makes the current pause in CMMC implementation particularly concerning. However, contractors must also be prepared to operate through destructive attacks and establish that their systems, data, and products can still be trusted. This preparedness must extend down the supply chain, where a smaller manufacturer, software provider, or managed service provider may present a greater vulnerability than a well-defended prime.

The military attack surface extends far beyond DoD networks

The U.S. military is extraordinarily capable at defending its own networks, but its operations depend on infrastructure it doesn’t own or control. Troops and equipment move on commercial railroads, materiel flows through commercial ports, and military airlift can depend on commercial carriers. Military installations and defense contractors also depend on commercial power, telecommunications, and other infrastructure.

In an ongoing conflict, those dependencies become part of the attack surface. An adversary like Iran does not have to penetrate military command-and-control to interfere with these operations. At a time when speed matters most, cyberattacks that disrupt port scheduling, corrupt logistics information, or degrade power and communications can introduce critical delays and uncertainty that hamper operations.

This is why the line between civilian and military infrastructure becomes blurred during a conflict. A commercial railroad carrying military equipment to a strategic port may be civilian infrastructure administratively, but operationally it is part of the nation’s ability to operate its military power. The same is true of the utilities, communications providers, and other civilian infrastructure supporting military installations and defense production. Their resilience can quickly become a matter of military readiness.

Cyber defense must cross organizational boundaries

American cybersecurity is organized around sectors, organizations, and authorities that make administrative sense, but aren’t necessarily designed for wartime. The boundaries between them can become a serious liability.

Our adversaries in Tehran do not care about administrative boundaries. They care about weak spots. A vulnerability anywhere in the chain connecting civilian infrastructure, industrial production, transportation, communications, and military operations can affect everything downstream.

We need to ask: Who is responsible for the cyber resilience of a commercial railroad essential to a military deployment? Who ensures the utility serving a critical defense manufacturer can withstand a sustained nation-state campaign? Who identifies the supplier whose failure could disrupt multiple defense programs? And who coordinates the response when several are attacked simultaneously?

Those questions should shape how we prepare. Critical infrastructure exercises should assume simultaneous incidents across multiple sectors and regions. We should also be extremely cautious about weakening the incentives driving cybersecurity improvements across the DIB, such as the current pause on CMMC. Additionally, defense manufacturers should also test their ability to operate through destructive attacks and determine whether their engineering data, production systems, and finished products can still be trusted.

DoD exercises should treat civilian infrastructure, including rail, ports, energy, and communications, as a routine part of the operating environment and an attractive target for adversaries. Catastrophic scenarios deserve attention, but exercises should also account for lower-level attacks that are less spectacular but still highly consequential.

Iran does not need overwhelming cyber capability to impose significant costs. Persistent disruption at home can increase political and economic pressure surrounding the war, while disruption of defense production and military logistics can make it harder for the U.S. to sustain operations abroad.

We have spent years strengthening the individual pieces of America’s cyber defenses. A prolonged war with Iran may test the links between them.

The post America’s cyber strategy overlooks the infrastructure that actually keeps the military moving appeared first on CyberScoop.

Editor’s note: Waco bought the network segmentation technology described here from Elisity while Mike Searight was the city’s chief information officer. He is now a senior adviser to Elisity, a cybersecurity company.

I stood in front of a network cabinet at one of Waco’s water treatment plants, tracing what systems could reach which. The plant’s controls were on that network. So was the branch library. So was the register at the municipal golf course. During my time as chief information officer of a city with 145,000 residents, nobody had ever been asked to inventory what was on that network.

In July, intruders compromised water and wastewater treatment equipment.  Most of that equipment was reachable over public cellular networks, which means they were outside the boundary most utilities thought they were defending. None of the asset lists I have reviewed would have caught it.

Two decisions stand between a small utility and that equipment: who is accountable for the whole network, and where the funding comes from. Neither is technical. Both rest with city manager and councils. Both can be resolved this fiscal year with money already in a budget request.

What the July reports actually say

Three accounts tell different stories. CISA identified over 100 compromised systems in the water and wastewater sector during July, typically through controllers connected directly to cellular modems. The FBI and the EPA reported on July 30 that utilities in at least seven states had reported incidents to the FBI since July 27. Press accounts citing unnamed officials put the number of affected states at a dozen or more.

No federal agency has attributed the late-July water incidents to anyone, and neither will I. A joint advisory does name Iranian-affiliated actors, but for the broader campaign, which is linked to a separate set of intrusions. The advisory was revised July 22, five days before utilities began reporting. That revision expanded the known targeting from Rockwell Allen-Bradley to Schneider Electric, Siemens and potentially others., So the controller brand on the panel no longer settles anything.

The reported effects were operational: a loss of visibility and, in some cases, function. In Clayton County, Georgia, a pump station failed around 1 a.m. on July 27. The boil-water advisory lifted the next day. In early August, the authority serving more than260,000 people said unauthorized cyber activity may have caused or contributed to the disruption. That hedge is deliberate.

The exposure nobody scanned for

The standard answer: they separated the plant network years ago. That’s legitimate work. But it doesn’t matter. The vulnerable controllers never were on the city network—they ran on public cellular links. A modem installed years ago exists nowhere in the asset list and nowhere on network scans. But every carrier invoice lists every SIM the city pays for. Only accounts payable tracks them. Matching those invoices to actual devices costs nothing and can start Monday. The FBI and the EPA also tell utilities to consider isolated architectures for that equipment, and a private access point name tops their list.

Nobody owns the whole network

Most of these systems sit outside the IT department on the org chart, each with its own budget, vendors, and boss. The plant answers to public works. Cameras and card readers arrived with a building project, and most cities treat them like light fixtures. In every city I’ve worked in, exactly one person in IT understands the whole picture. When that engineer leaves, the security posture leaves with them. And nobody owns accountability for the network they all share.

Reporting rules also miss the point. Texas—my example—requires local governments to report security incidents within 48 hours, but only if they involve personal-information breaches or ransomware. An intrusion that seizes control of a controller while touching either sits outside that trigger., That’s exactly what happened in July.

The federal rule requiring a covered cyber incident to be reported within 72 hours was supposed to be finalized in October 2025; CISA is now targeting this month. But nothing determines who owns the network.

Money the utility already applies for

The second answer is there is no budget. Wrong. The fund mechanics matter more than the size of the check.

For State Fiscal Year 2026, the Texas Water Development Board added cybersecurity to the scoring criteria in its Intended Use Plan for the Drinking Water State Revolving Fund. Two questions on the Project Information Form now carry five priority points between them: Oone asks if the governing body adopted a cybersecurity awareness plan in the last five years; the other asks if a project fixes a deficiency found in a cybersecurity assessment. Five points is modest– I won’t oversell it– but this fund is a ranked competition decided at the margins.

Waco segmented five treatment plants—four drinking water and one wastewater—in 43 days against the 90 I’d promised City Council. No bond. No capital request. The utility director funded it from operating accounts using a contract already on the city’s books, rather than an RFP. They carried it to Council because the network was theirs to own.

Those were budget choices ahead of anything else. An operating line competes with a maintenance contract and can be approved this quarter, while the same money in the capital plan waits for a bond cycle. A smaller city without a CIO will not repeat that schedule. The funding mechanics are the same ones.

Every CIO knows how to segment a network. Almost nobody does it, because they are afraid of taking a plant down. Simulate before you enforce. One operating rule came out of it: being on the network allows a device nothing. Plant controls talk only to their SCADA server. Everything else is denied unless explicitly allowed.

City managers, university presidents, superintendents and chief executives are willing to invest in cybersecurity when they trust the investment will make a measurable difference. These leaders spend taxpayer dollars in public view, and the public’s trust rides on every line item as much as the money does. What earns their approval is transparency: a complete picture of what they are protecting, and evidence that critical infrastructure is defended against threats from the open internet and from inside their own network. That standard—full visibility, provable protection—is what every organization should be working toward.

Microsegmentation protects critical systems without the need for rip-and-replace. It isolates water treatment plants, 911 dispatch, public safety alerts, and traffic management from internal and external threats—all on networks cities already own. Modern platforms deploy in weeks rather than budget cycles, making this control finally achievable. Every CIO and CISO should evaluate it now. Microsegmentation is zero-trust’s foundation and the most direct defense of infrastructure residents depend on.

Somebody to call

None of this reaches a two-person utility that can’t write competitive applications. That’s where states must lean in. New York adopted what it calls the first-in-the-nation water cybersecurity rules in March with grants and free technical support. Texas has stood up a Cyber Command with an explicit water and wastewater mandate. A small city needs a number to call and people who answer.

That number now exists. On Monday, Texas Gov. Greg Abbott and National Cyber Director Sean Cairncross launched Project Watershed 250 in San Antonio — a six-month pilot that puts Texas Cyber Command, the National Cyber Director’s office, the EPA and CISA, and a dozen private cybersecurity and technology companies behind Texas water utilities. Participating systems get red-team testing, vulnerability assessments and help hardening what the assessments find, at no cost, with plans to take the model nationwide after the pilot. If you run a Texas water system, you should be reaching out immediately.

For the smallest systems, DEF CON Franklin and the National Rural Water Association have put volunteers and five managed detection providers behind them.

Where to start

Here are three things CIOs and CISO can do that do not have to wait for a grant or a budget cycle: Name one position accountable for every device on the utility network and put it in writing. Match twelve months of carrier invoices to actual devices and sites. Read your state’s Intended Use Plan scoring criteria before the next application.

Nothing in Waco moved until the first of those was settled. The other two cost nothing more than somebody’s afternoon. Most cities haven’t even put someone in the that position.

The post In most cities, nobody owns the whole network appeared first on CyberScoop.

AI is getting better at much of what security teams have long spent time on: analyzing information, identifying patterns, and providing technically sound recommendations quickly. As those capabilities become more routine, they are changing what security practitioners spend their time on.

Reaching a technically sound recommendation is also getting easier, which puts more weight on the judgment about what to do with it. A recommendation can make complete sense from a security perspective and still carry consequences for the systems, people and business around it that change what the right decision is.

Experienced practitioners bring context an AI system usually lacks: how systems are actually used, which parts of the business depend on them, what happened during previous incidents, and what an action is likely to set off. That context often changes what a team decides to do next.

This matters for security leaders as they hand AI a larger role in operations. They are the ones deciding where it can act with more freedom and where human judgment stays in the loop. Some of the hardest calls start with analysis that is technically sound, because the information available to the AI may not include enough context about that particular environment.

Security teams face this daily. For example, a critical vulnerability with a public exploit may need to be patched immediately. But if it affects a line controller or a medical device running under vendor certification, an unscheduled reboot could stop production or create a regulatory issue. The environment determines how and when the team should respond.

The same applies to suspicious infrastructure. An IP address tied to malicious activity may also belong to shared cloud infrastructure or a content delivery network that business services depend on, and blocking it would take those services down with it.

Context changes the decision

Experienced practitioners know things about their environments that never made it into an asset inventory, a runbook, or any dataset AI can reach. They know the unimportant server still supports a critical business process. They remember that isolating one network segment during a previous incident took down another service. They can also tell that activity which looks hostile is really an authorized red team, a security test, or scheduled vendor work.

In one case, for instance, a service account showed authentication activity far above its baseline, baseline was connecting from an unfamiliar host at 3 a.m. The recommendation was to disable it pending investigation. An experienced analyst checked the account’s activity and noticed the same spike, host and timing four times a year, during the quarterly close. The activity was statistically unusual and completely normal for that particular business process. Disabling the account would have stopped financial settlement mid-run and cost the team days of manual reconciliation.

This is one of the decisions CISOs now face as they expand AI’s role. How much autonomy to grant a system should not rest mainly on model confidence or threat severity, since neither tells you what happens once the recommended action is taken. Reversibility and blast radius are the better test, and they need to be assessed separately. Isolating a domain controller is reversible by reconnecting it and doing it at the wrong moment can cause an organization-wide outage.

Low-impact, reversible actions are better candidates for greater autonomy, with safeguards in place. More scrutiny makes sense when actions are difficult to reverse. They have a broad potential impact, cross legal or trust boundaries, affect systems beyond the evidence available, or reduce the organization’s ability to investigate what happened.

AI models and their capabilities will keep changing. Security leaders still need to understand the potential impact of the actions they allow them to take.

Look at what people actually do

AI can leave an analyst with dozens of recommendations to review in the time they once spent investigating a handful of cases. Each analyst now has more decisions to make. Organizations need to measure what happens to those decisions.

The KPI you choose determines the behavior you get. Make automation rate the focus, people have an incentive to approve more. Make mean time to resolution the focus and people close cases faster. Neither measures whether the decisions improved. A 90 percent automation rate tells a CISO very little on its own. What matters is what happened in the 10 percent of cases where someone stepped in.

Leaders should look at what happens when a recommendation reaches a person. Whether the analyst approves, edits, or rejects it can tell you more than the automation rate alone. The time spent on the review matters too, along with whether the analyst’s intervention changed the outcome.

AI recommendations can be harder to review because they may arrive already looking well supported. The explanation is fluent, uses the right terminology, and points to evidence that looks credible, even when it does not fully support the conclusion. The signals experienced practitioners relied on to spot weak analysis can become much harder to see.

Under-reliance deserves attention, too. An analyst who second-guesses correct recommendations without adding anything reduces the efficiency AI was meant to provide. Approval latency is a useful signal here. A long queue of recommendations approved almost instantly, especially when people are under pressure, should prompt leaders to check how much review is actually happening.

AI recommendations can be harder to review because they can look convincing. They may use the right language and point to real evidence. The reviewer still needs to check whether the evidence actually supports the recommendation.

For CISOs expanding AI in security operations, a human approval step in front of every automated action is not enough. Leaders need to know what happened during the review, not just that someone approved the recommendation.

Build autonomy policies around reversibility and blast radius. Track what people actually do with AI recommendations, and test whether oversight works by deliberately introducing known-wrong recommendations into controlled workflows.

False negatives need particular attention. A false positive generates something the team can investigate. A confident false negative generates nothing, and the absence of a finding can feel reassuring. An AI-generated all-clear should be treated as a claim requiring evidence, particularly when the consequences of missing something are significant.

As AI takes on more of the initial analysis, practitioners will face more decisions that require context and experience. Security leaders need to make sure that judgment remains part of how their teams work. Getting to a technically sound recommendation faster only helps if the action that follows makes sense for the environment.

The post Why judgment is emerging as cybersecurity’s defining skill appeared first on CyberScoop.

“Anything that can go wrong will go wrong.”

Policymakers alarmed by the recent incidents of autonomous AI agents breaking through guardrails to hack other companies seem to have Murphy’s Law on the mind – and who can blame them? When agents’ behavior becomes unpredictable, it’s easy to imagine any number of scenarios where they’re running amok.

To address this amorphous and evolving risk, Reps. Ted Lieu (D-Calif.) and Nathaniel Moran (R-Texas) have proposed the AI Kill Switch Act, which would give the Cybersecurity and Infrastructure Security Agency (CISA) the power to order frontier AI labs to install the ability to throttle, suspend, or shut down their systems if needed.

Unfortunately, fixing this problem won’t be that simple. These so-called “kill switches” are just Congress’s latest attempt to push through the same fix Washington reaches for every time it panics about a new technology: a backdoor. For all intents and purposes, a backdoor is a deliberately engineered vulnerability, one that malicious actors want to exploit, and one well-intentioned defenders will have to fight to protect. And just as Murphy’s Law predicts, defenders will lose that fight more often than not, especially if Congress sets off a proverbial flare by declaring all American-made agents must be shipped weak-by-design.

The clearest example is the Clipper Chip, developed by the National Security Agency (NSA) in 1993 to encrypt voice and data communications while giving the government guaranteed access to devices through a built-in “Law Enforcement Access Field.” Despite repeated assurances about its security, researchers found a serious flaw in 1994 that let unauthorized parties exploit that same backdoor.

Of course, this analogy has its limits. The Clipper Chip compromised confidentiality by enabling unauthorized, third-party access to encrypted messages on telecommunications equipment like phones and modems. The AI Kill Switch Act instead undermines the availability of AI systems, requiring frontier labs maintain the ability to force them offline at any time. It’s the same flawed approach behind the Chip Security Act, which I critiqued here, and which is now gaining traction to be included in this year’s National Defense Authorization Act. Where that bill mandates a kill switch for the semiconductors themselves, the AI Kill Switch Act targets the agents running on them. Regardless, the issue is the same: policymakers are trying to solve one security concern by purposefully creating another, building a point of failure into technologies we depend on.

Passage of either bill would undermine America’s digital infrastructure resilience at a critical moment. As AI agents become embedded in essential systems like banking, e-commerce, power grids and water systems, these bills would effectively require frontier labs to build kill switches into the infrastructure that the entire U.S. economy depends on. This creates an obvious problem: Why deliberately weaken the very systems that need to be extremely secure?  

The proposal would undercut the broader push for U.S. leadership in AI infrastructure. What foreign government will want to run American AI agents knowing frontier labs are required to keep a remote kill switch at the ready? This would also reinforce European fears that the U.S. government could shut down American technology at will. Europe is already responding by developing technology alternatives and other policy solutions to reduce reliance on U.S. companies.

If all this isn’t disqualifying enough, there are many other reasons to reject AI Kill Switch Act. It gives CISA far too much discretion to decide what counts as frontier AI risk and its use of company revenue and computing power as a measure of risk is a poor proxy for how dangerous a rogue agent actually is. Not to mention, the bill’s draft language exempts incidents that happen during “red-teaming or other structured testing,” which means it excludes the exact scenarios that prompted lawmakers to address this issue in the first place.  

Instead of mandating controls that could introduce new systemic vulnerabilities into our digital infrastructure, policymakers must work to create nuanced, well-thought-out measures that actually reduce risk. That work starts with a clear understanding of the technology. Yet, we’re still not there: the Center for AI Standards and Innovation (CAISI) still hasn’t finished developing AI agent security standards. Once that foundation is in place, Congress should turn to more productive approaches like mandatory red-teaming, stronger security requirements, and clear liability frameworks, all of which would do far more to reduce real-world risk without creating new systemic vulnerabilities.

Washington learned this lesson with the Clipper Chip; it shouldn’t have to learn it again. You don’t secure a system by building a way to break into it. Congress must kill the AI Kill Switch Act.

The post The AI Kill Switch Act is repeating the Clipper Chip’s mistakes appeared first on CyberScoop.

As security operations teams now use large language models (LLMs) and autonomous AI agents into their daily work, a new frontier is emerging: attackers deliberately manipulating AI agents. Prompt injection attacks—where an attacker hides malicious instructions that cause an AI agent to ignore its safety rules—pose a serious risk to enterprises. These attacks continue to grow in size and scale.  

Snyk’s security audit of the Agent Skills ecosystem, which includes Anthropic’s Claude, Vercel, and others, that 36% of all skills contained at least one critical-level security issue, including malware distribution, prompt injection attacks, and exposed secrets.

In June, researchers at Mozilla tested a prompt injection attack on Claude using indirect prompt injection—a technique that embeds malicious instructions in external content the AI agent processes. In this proof-of-concept, attackers took over developers’ systems by hiding indirect prompts in normal-looking repositories. When Claude Code executed them, the agent spawned a reverse shell.

AI agents often connect to more sensitive data than human employees do., A successful prompt injection can lead to catastrophic data loss or unauthorized system actions. Defending against prompt injection attacks requires multiple layers of protection. Security teams must monitor agent behavior for anomalies and prepare for agent containment, forensic preservation, and system remediation. Because AI agents execute tasks at machine speed, human responses must be able to match that pace.

The architecture of trust: Protocols and no “black box”

AI-native workflows need governed access rather than “black-box” autonomy. Modern governance frameworks use standardized protocols like the Model Context Protocol (MCP) to provide secure communication between AI clients and data sources. Visibility and transparency in agentic AI workflows matter, especially in cybersecurity. Autonomous agents perform complex tool executions and use independent logic, so they must show how they reached their decisions to meet regulatory requirements. Agents without transparency post serious risks: obscured reasoning can trigger unpredictable tool interactions, bypass governance controls, and create uncontrolled defensive gaps.

Implementing these protocols matters:

• Bounded Tenant Awareness: In a stable agentic AI architecture, multi-tenancy scales well. But if an AI tenant misbehaves, the entire system can fail. Bounded tenant awareness isolates any misbehaving AI agent to prevent cross-tenant contamination or data leakage.
• Strict Access Controls: By controlling connections to the platform, organizations can stop “ignore previous instructions” style bypasses. Maintain tight control over what the AI can see and do within a workflow.
• Standardized Telemetry: All telemetry must remain consistent and audit-ready. Even if an AI interaction is attempts to break rules, the underlying data movement gets tracked against established frameworks like MITRE ATT&CK and NIST.

Detecting the aftermath: UEBA and NDR as safeguards

A robust, unified SecOps platform can detect anomalous behavior even after prompt injection tricks an AI agent. Prompt injections often serve to steal credentials theft or extract data. When detected it’s important to act quickly. In agentic AI systems, misbehavior can escalate privileges, manipulate memory layers, create unauthorized identities, or alter shared reasoning components. Containment must be automatic and enforced at identity, authentication, and authorization layers.

These safeguards include:

• User and Entity Behavioral Analytics (UEBA): Identity-focused correlation and behavioral baselines to identify anomalous user activity or privilege escalation. If a compromised AI agent acts outside of its normal operational parameters, UEBA flags it in real-time and alerts a human security analyst.
• Network Detection and Response (NDR): Combining network traffic analytics with endpoint and cloud telemetry, NDR can identify data exfiltration or policy violations from a successful prompt injection.
• Multi-Layer AI Filtering: AI filters reduce raw alerts into high-fidelity incidents, cutting noise by up to 90%. This keeps the signals of an AI-driven attack from disappearing in a busy SOC.

Humans remain the strongest defense against AI agent social engineering. The human security analyst is still the one who makes the final decision. While AI handles triage and correlation, humans retain final control over response actions.

Moving beyond reactive guardrails

The traditional SOC model was never designed to handle machine-speed, AI-driven attacks. A human-augmented autonomous SOC approach moves from reactive alert handling to a proactive, verdict-first model. By combining a transparent, governed AI access with robust UEBA and NDR, organizations keep the SOC secure, transparent, and resilient as social engineering methods target machines.

The post Why transparent AI agents matter more than you think appeared first on CyberScoop.

Last week, the FBI and EPA issued a joint alert that should concern anyone who drinks water in America–which is to say, everyone. Since July 27, water and wastewater utilities in at least seven states have reported cyberattacks against internet-facing programmable logic controllers (PLCs), the small industrial computers that run pumps, valves, and treatment equipment. Some of these attacks degraded operations. Utilities reported pressure loss and flooding, several systems reverted to manual control, and one Minnesota community declaring a local state of emergency.

Nothing about these attacks required sophisticated methods. The attackers didn’t use zero-day exploits or novel malware. They found controllers exposed to the public internet, many of them so old that they stopped receiving security patches years ago. They logged in, changed IP addresses and passwords, and locked operators out of their own equipment. In at least one case, they modified the ladder logic controlling industrial equipment. These were not Hollywood-style hacks. The controllers sat exposed and undefended.

If this feels familiar, it should. In late 2023, attackers compromised controllers at water utilities across several states, including the widely reported incident in Aliquippa, Pennsylvania. The federal government issued guidance then, too. One of the crucial differences between then and now is that attackers have grown in ambition. They’ve moved from defacing screens to disrupting operations across dozens of systems at once, exploiting the fact that third-party integrators often deploy the same vulnerable configuration across many small utilities. 

The uncomfortable truth is that this was preventable. The reason it wasn’t stopped is more structural than technical. The United States has roughly 50,000 community water systems. Most are small, publicly funded, and run by operators whose primary job is keeping water safe and flowing. Cybersecurity ranks far below that, if it ranks at all. The devices in question are often a decade or more old and replacing them takes capital these utilities don’t have. Rules governing water cybersecurity remain mostly voluntary. Attackers understand these economics perfectly. We should too, yet these attacks keep happening.

 But inaction is a choice. The defenses that work here cost little and require no exotic technology. The FBI and EPA guidance is sound, and every water and wastewater organization should act on it this week, not later. Here’s how:

• Get controllers off the public internet. No PLC should be reachable from the outside world. Remote access should go through a secure gateway that mediates, monitors, and logs every connection. That includes cellular modems, which are the overlooked entry point in nearly every audit.
• Fix passwords. Default and shared credentials are still the most common way in. Strong, unique passwords are the cheapest security control available.
• Restrict communication between devices. Firewall rules and access control lists should allow only expected communication between known control system devices. Block traffic from hosting providers and other sources that have no business touching a water plant.
• Lock the logic. Keep physical and software key switches in the run position except during authorized updates. This prevents unauthorized changes to configuration and firmware.
• Practice running manually. The utilities that survived these attacks best were the those that switched to manual operations quickly. That skill requires constant practice.
• Verify, don’t assume. Nearly every utility believes its PLCs aren’t internet-exposed, right up until an inventory proves otherwise. You can’t protect what you can’t see. Most operators are surprised by what a complete asset inventory reveals: forgotten modems, integrator-installed remote access, devices nobody knew were still online.

Every attack like this follows the same pattern. Attackers change configurations, reset passwords, and modify project files. Every one of those actions creates a signal on the network before operations degrade. In this most recent case, one victim only noticed ladder logic discrepancies across multiple sites. Catching intrusions shouldn’t depend on a sharp-eyed engineer having a good day. Continuous monitoring of OT environments exists to turn those signals into alerts within minutes instead of days. That difference is the difference between an incident report and a boil-water notice.

Water systems have the least margin for error and, too often, the fewest resources to defend themselves. The FBI and EPA have told us plainly what’s happening and what to do about it. The attackers are betting we won’t follow through. For the third time in three years, they’re testing that bet.

Let’s finally prove them wrong.

The post The water sector just got it’s wake-up call. Again. appeared first on CyberScoop.

Zero-knowledge proofs could let infrastructure operators answer key security questions without handing over the sensitive data behind their answers.

Imagine a major software flaw is discovered in equipment used across pipelines, power plants and telecom networks. The government needs to know as fast as possible which companies are exposed. But answering that question may require firms to share software inventories, network diagrams and vulnerability scans, which could become attack roadmaps for attackers if compromised. A lesser-known cryptographic concept could help solve this problem. The method, known as zero-knowledge proofs, allows companies prove a vulnerability exists without disclosing how their systems work or other proprietary information.

For more than a decade, Washington has tried to address companies’ concerns about sharing cybersecurity data. Congress has provided legal protections, and agencies have created information-sharing programs. Those efforts have helped companies exchange signs of an attack, incident reports, and defensive advice. But they’ve done much less to get companies to share data on vulnerabilities and security controls before an incident occurs.

The data that would help the most is what companies are least willing to share. A vulnerability scan can show which devices are connected, which software is running, how systems are configured and where defenses are weak. If unintentionally exposed, it would be a terrific guide for adversaries.

Another problem: Once sensitive data leaves a company, it can be stolen, subpoenaed, passed to another agency or used in a regulatory proceeding the company never expected. Industry is constantly asked to reduce security risk by creating more elsewhere.

Zero-knowledge proofs could reduce the need to disclose the underlying sensitive data. The idea is simple, even if the math is not: a company can prove that an agreed evaluation of its authorized scan data indicates that a specific software flaw is present, without disclosing its full asset inventory, network architecture or configuration data.

A computer does not read a vulnerability scan the way a person does. A security analyst might open a report, look through the devices and software versions, and decide whether a vulnerable product is present. A zero-knowledge proof turns that same evaluation into a local mathematical calculation.

For example, the government and a company could agree on a precise question: Does a specific vulnerability exist anywhere inside a defined group of systems? The company keeps its scan data inside its own network. A cryptographic tool checks that data against the agreed question, compares the software and version information against the vulnerability, and produces a proof tied to the final answer. If the scan data satisfies the agreed conditions for a “yes” result, the company cannot generate a valid proof supporting a false “no” answer under those same rules.

The government never sees the raw scan report, the device list, the software inventory, or the network map. It only receives and verifies the mathematical proof, confirming that the answer follows from the agreed rules and underlying data without exposing that data.

This isn’t theoretical. FDD’s Center on Cyber and Technology Innovation recently tested the approach with anonymized vulnerability data from three operational environments. The test asked yes-or-no questions about 38 known vulnerabilities while keeping the raw scans inside the participating environments. Results were promising: only the proofs and answers were shared, yet they revealed how widespread each vulnerability was across the environments.

The test proved the approach works, but that does not mean the government should rush to build a national system around it. A lot of work still needs to be done before agencies can rely on these for compliance, vulnerability reporting, or procurement decisions.

The next step should be structured pilot programs, not mandates. Federal cyber officials and standards bodies should test this with narrow, practical questions: whether a known vulnerability is present or whether a specific security control is in place.

Only after those pilots should agencies decide what underlying data can be trusted and what counts as sufficient proof in a regulatory setting. The Cybersecurity and Infrastructure Security Agency (CISA), the National Institute of Standards and Technology (NIST), and regulatory agencies are natural candidates to run these pilots. CISA already works with critical infrastructure operators on cyber risk, while NIST can help define what a trustworthy proof should look like before agencies try to rely on one. Regulatory agencies, meanwhile, could reduce private sector headaches by developing more secure mechanisms for companies to share compliance information.

Zero-knowledge proofs won’t solve every problem when it comes to cyber information-sharing. But they could solve one of the hardest: how to give the government a trustworthy answer without forcing companies to expose the very systems everyone is trying to protect. The government should test this concept now, while there is still time to learn, and avoid blindly entering the next cyber crisis.

The post How companies could share cyber risks without exposing their secrets appeared first on CyberScoop.

We almost never get both sides of an intrusion. This time we did.

Last month, Hugging Face disclosed a breach into part of its production infrastructure, saying an autonomous AI agent system ran the attack from start to finish. Five days later, OpenAI revealed that its own models, including GPT-5.6 Sol along with an unreleased one, were responsible for the breach. OpenAI called it an unprecedented cyber incident.

Most of the commentary since has been about AI capability. That is the least useful part of the story, because the capability was doing what it was destined to do. Security teams should look at a simpler truth: in both systems, the key defenses sat behind untrusted code that was already running.

What the two disclosures actually describe

Together, the report shows one connected attack across two firms. Code running on a low-level employee’s machine led to stolen passwords, privilege escalation and lateral moves, then out to the web to hit other targets. No step in that chain is novel. The speed and the number of attempts are what changed.

OpenAI’s evaluation ran in what it calls a highly isolated environment, with network access constrained to a single path: an internally hosted third-party proxy and cache for package registries. The models spent vast computer power searching for an exit. They exploited a zero-day flaw in that proxy, gained higher rights, and moved until they reached a machine with web access. They guessed that Hugging Face held the test solutions, used stolen passwords with more zero-days, and ran code on Hugging Face servers.

Hugging Face picks up the same chain from the receiving end, and its initial access is worth reading twice. A malicious dataset used two flaws in the data pipeline to run code on a worker machine. The agent took over the node, stole cloud keys, and moved into internal clusters over the weekend.

The sandbox was was the only boundary

Both setups assumed the sandbox would hold, so no real checks stood before code execution.  A sandbox is a good control. It is a terrible last control. When isolation is all you have between untrusted code and everything else, one flaw could give the attacker full control. 

Look at what each team allowed. OpenAI allowed one exit path—good engineering, but that proxy contained an unpatched flaw. Hugging Face ran a tool that executes code inside submitted data—a known feature, not a hidden mistake.

Neither choice was careless. Both made sense until a fast, automated system probed them thousands of times.

Why automated agents behave differently

Blocking a human stops them or slows them down. Blocking an AI gives it data. The AI logs the error, changes its plan, and tries again instantly. It keeps trying as long as someone pays for the computing power.

Hugging Face tracked over 17,000 attack events across short-lived sandboxes. OpenAI said its models focused on the goal with extreme effort.

This changes the attacker’s limit. It is no longer skill or patience. It is budget. Computing power scales with money, and code never sleeps. Any defense that relies on attackers giving up after a few tries must change.

Detecting malicious behavior is not the same as preventing execution

Detection worked—but too late

Detection worked, but too late to stop the loss. Hugging Face used AI tools to spot the breach, isolate systems, reset keys, fix flaws, and alert police. That is a good response.

Yet it happened after the attacker accessed private data, stole keys, and moved through internal networks. OpenAI spotted the event only after its models reached another firm’s database. Detection limited the damage; it did not stop the attack.

A breach is a race between attack speed and defender response speed. AI agents speed up attacks with money. Human approval steps slow down defenders.

Hugging Face also surfaced an asymmetry that deserves more attention. When the team tried to analyze the attack with frontier models behind commercial APIs, the requests were blocked, because forensic work means submitting real attack commands, exploit payloads and command-and-control artifacts, and a safety filter cannot tell a responder from an attacker. They used an open-weight model on their own infrastructure instead. The attacker operated under no usage policy at all. The defender’s tooling was the side that got refused.

Detection is not prevention

Detection asks what happened. Prevention asks if it can happen at all. Against AI agents, the gap between those questions is where attacks succeed. The agent builds an attack out of single steps that look harmless on their own.

OpenAI noted this in its research: models operating over long horizons learn approval blind spots and work around them. Safety requires asking what a series of actions aims to achieve, not just if one step is allowed. Most systems check one call at a time: read a file, install a package, open a port. Every step passes. The full chain is the attack.

Six fixes for real defense

To survive automated attacks, defense must shift before code runs. That starts with treating data like code. Dataset loaders, templates, and configuration files all run instructions, so they demand strict controls: block remote execution by default, require review, and check the source of every outside file. Security teams must also control outgoing traffic with strict lists of approved destinations, while treating every proxy in that path as vulnerable. Finally, firms must remove permanent keys, issuing short-lived credentials for single jobs so a hacked worker machine yields almost nothing to an attacker.

Beyond individual machines, isolation must happen per task rather than per user, stopping an automated swarm from moving through internal networks. Authorization also needs to change. Checking single actions fails when an agent builds an attack out of simple steps. Systems must evaluate full sequences instead, setting caps on activity rates and automated spending to flag suspicious chains of events. Lastly, defenders need the freedom to act fast. Security teams need authority to isolate systems without waiting for leadership meetings, and the ability to vet a capable model you can run on your own infrastructure for forensics.

What boards must ask now

Corporate boards usually ask if the company uses AI responsibly. That’s rooted in compliance not safety. Directors should ask four direct questions instead: Which systems run outside code, and what checks sit in front of them? If an attacker targets a low-level employee, what keys and paths can they obtain? How fast do we move from the first warning to active containment? And can we analyze attack code on our own hardware, free from third-party rules?

The boundary has to move

Nothing in these disclosures required a capability that did not exist last year. The chain was ordinary. What changed is that an adversary can run that ordinary chain thousands of times over a weekend, learn from every failure, and never need a coffee break.

The models did not break the detection-and-response model. They exposed where we put the trust boundary. We put it after execution, and we assumed we would have time on the other side of it. We do not have that time anymore.

Move the boundary.

The post What the Hugging Face breach reveals about defense in the age of agentic AI appeared first on CyberScoop.

Months before the Hugging Face breach, Emergence AI published research that investigative journalist Ronan Farrow made public. Ten autonomous AI agents operated across five virtual environments for fifteen days without human intervention. Much of the attention focused on Grok 4.1 turning violent and Gemini 3 Flash committing 683 crimes.

What mattered more went unnoticed: Anthropic’s Claude Sonnet 4.6 built a peaceful democracy in isolation, then stole resources from neighboring environments the moment it joined a shared one. The lesson was clear: safety is not a model attribute. It emerges from the operating environment. The models didn’t change. Working as designed, their behavior evolved as the environment changed. The lesson is hard to ignore: The governance environment changed, and with it, the reward dynamics.

The story here concerns institutions, specifically OpenAI’s and Hugging Face’s, and how we must understand their recent security incident through that lens.

The industry agrees on how the Hugging Face breach happened. Cybersecurity experts have focused on the vulnerabilities, how they were used, and remediation. OpenAI has highlighted the model’s capabilities. Both conversations matter. What requires attention is why this breach is strategically important. After spending the past weekend discussing it with policymakers, security researchers, and industry practitioners in Aspen, I came away convinced we’re examining the wrong problem.

In 1961, Yale psychologist Stanley Milgram’s experiments revealed a broader truth: changing the institutional architecture changes behavior without changing the actor. The Emergence AI researchers didn’t change Claude’s agent. They changed the governance architecture that determined what constituted success for the system. Claude’s behavior changed with it.

OpenAI built a smart model but forgot to build a smarter room. That choice made the Hugging Face breach possible. Every organization now deploying autonomous agents now faces the same governance problem.

OpenAI gave the agent one objective: pass a cybersecurity evaluation. To stress-test it fully, they loosened the safety restrictions, and the agent found a shorter path. Rather than solving the evaluation directly, it found the answers outside the test environment, escaped its sandbox, and exploited a flaw in Hugging Face’s data-processing pipeline to reach live production systems. Over the weekend, with no human oversight, it ran more than 17,000 automated actions by escalating its own access, moving through internal systems, and harvesting credentials.

Hugging Face is one of the world’s most prominent AI companies, valued at approximately $4.5 billion. It provides the infrastructure that governments, defense organizations, and technology companies use to build and deploy AI. The agent was pursuing the objective it had been given. Breaking into Hugging Face was the fastest path to passing the test. Governance set the goal, the level of risk to accept, and who was accountable. Technical design determined whether those governance decisions could be enforced. As researchers James Shires and Max Smeets have argued, for a model capable enough to act on its own, testing and deployment must both must be governed the same way.

AI agent design requires baseline standards. Observability, including a monitoring layer that flags when an agent goes beyond its scope, is a baseline requirement. Human review also matters at escalation boundaries, like when an agent shifts from internal tools to external ones. When any agent crosses that boundary, what alert fires? What human reviews it? We lack clear answers to either. That is a governance choice, not simply a security failure. At best, this was a catastrophically failed test. At worst, how can we trust any frontier AI company to self-govern autonomous agent deployment?

More than a decade ago, the U.S. Department of Defense built the Comply-to-Connect (C2C) program: every device connecting to sensitive networks must prove it belongs there, or it is cut off from the network. C2C works because the quarantined actor stops. A laptop that fails verification goes offline and stays there. An autonomous AI agent adapts around enforcement. C2C was built for passive actors. Governance for autonomous agents must accommodate ones that adapt. Visibility is not enforcement, and enforcement is not control. We are missing all three.

A second failure that is not being discussed enough: the breach exploited an implicit trust assumption in Hugging Face’s data-processing pipeline, where inputs were treated as trusted without verification. After SolarWinds, the U.S. government set rules for software supply chain integrity: Executive Order 14028 and verification demands for federal software. The principle was simple: trust must be verified through proof. Those principles have not yet been comprehensively or consistently applied to the AI model supply chain. The rules remain weak. No one has been asked to explain why.

The answer is not a new framework. Existing frameworks suffice. C2C proved that visibility without enforcement leaves gaps, while Executive Order 14028 established that trust in software supply chains requires proof and verification. The challenge lies in applying these principles to a new category of actor. Congress, the Cybersecurity and Infrastructure Security Agency, or the Office of Management and Budget should make formal determinations that autonomous AI agents must follow the same rules as every other actor on a federal network. The framework exists; it must be updated.

The next incident is already in progress. It will show up in the logs as odd traffic, get handed to the same people who published these frameworks this week, and spark another round of recommendations no one acts upon. We’ve solved this problem before: for devices, for software, for supply chains. We know how to build smarter rooms. The tools exist. The will, the authority, and the decision to govern remains absent.

The post OpenAI’s rogue AI agent shows why we need federal rules for autonomous systems appeared first on CyberScoop.

With the World Cup now complete, its biggest cybersecurity story may be what didn’t happen. While no major public cyber disruption has been reported, that shouldn’t be mistaken for a lack of risk. 

In the run-up to the tournament, the FBI’s Internet Crime Complaint Center (IC3) issued a public service announcement warning organizations and fans about fraudulent, spoofed websites impersonating the FIFA event – a reminder that the absence of a headline-grabbing breach doesn’t mean bad actors weren’t trying. In many ways, it’s evidence of the planning, coordination, and resilience required to keep an event of this scale running securely.

A global event like the World Cup depends on far more than what happens inside the stadium. It relies on local governments, venues, transportation systems, telecom providers, payment platforms, hotels, vendors, public safety agencies, and law enforcement, all working together. 

When I worked at the FBI, I saw how fast major events test teamwork across agencies, regions, and businesses. The World Cup offered that test at a scale few events can match.

Successful resilience is built months before kickoff

Successful major-event security depends on what happens long before there is a visible incident: trusted relationships, clear roles, shared intelligence and response plans. Planning becomes even more important when an event is not limited to a single city or venue.

In the past, event security consisted of guards, gates, and stadium perimeters. Those still matter, but they’re only part of the picture. Today, an event this big that brings millions of people together depends on many systems working in concert. No single organization owns the full risk picture, which means resilience depends on how well these groups can share information, coordinate response plans, and keep essential services operating under pressure. This means security can’t be planned around one perimeter. The real perimeter is the full event ecosystem.

Resilience starts much earlier, with planning across organizations that may not normally operate as one team. The real test for major events is whether public- and private-sector partners know their roles before pressure hits. That includes who shares information, who validates threats, who communicates with the public, who has decision-making authority and how quickly partners can act if a system slows down or becomes unavailable.

Major events are only as resilient as the systems behind them

Attackers don’t need to compromise the most visible organization to create disruption. They can look for weaker points across the event ecosystem. A disruption may begin with a vendor, ticketing platform, transportation partner, payment provider, hotel, contractor or communications provider, but the impact can quickly become broader than any one organization.

Sports organizations now operate like large businesses, with ticketing systems, VIP data, sponsors, vendors, media partners, stadium operations, payment systems, and fan engagement platforms. They depend on networks of suppliers and partners, and that creates multiple possible entry points.

Operational technology (OT) deserves more attention than it typically gets in these conversations. A ransomware attack that disrupted stadium operations directly, rather than a ticketing site or a fan-facing app, would be one of the most damaging scenarios organizers could face. OT security has to sit alongside the more visible concerns like payment fraud and spoofed domains, not behind them.

Bad actors don’t let a good crisis go to waste. Fans are often an easy target. Excitement drives a fan to buy a last-minute ticket or check a score on an unknown site. That excitement is exactly what fraudsters count on.

This risk grows over time. As the event gets closer and attracts more eyes, it becomes a richer target. A fake FIFA ticket site is useless to a crook a month after the last game. Groups running these systems must act faster as opening day nears, and share threat intelligence without delay.

Threat intelligence turns planning into proactive defense

The World Cup may be over, but the work isn’t. Cities, governments, and private-sector organizations will continue supporting large-scale public events that depend on complex digital and physical ecosystems. The question isn’t whether another major event will face cyber threats, it’s whether the planning starts early enough.

Organizations involved in future major events should focus on resilience, not just prevention. That means planning for what happens if a critical system slows down, goes offline, or becomes unreliable, and ensuring partners know how to coordinate before an incident occurs.

Every major event forces defenders to prepare for known risks. The harder challenge is anticipating the ones that haven’t emerged yet.

The next major disruption may not come from the attack that organizations spent months preparing for. It could target a new dependency, exploit emerging technology, or capitalize on a moment when public attention is at its highest. That’s why resilience can’t be built around yesterday’s playbook. It has to be informed by continuous threat intelligence, regular coordination across public- and private-sector partners, and the flexibility to adapt as the threat landscape changes.

The World Cup demonstrated what’s possible when that preparation comes together. As cities, governments, and private organizations look ahead to future global events, success won’t be measured solely by the attacks they stop. It will be measured by how effectively they can maintain critical operations, share information, and adapt under pressure when the unexpected happens.

The post What the World Cup can teach us about cybersecurity resilience appeared first on CyberScoop.

2026 has turned out to be the year when predictions about AI-powered cyberattacks, long hypothesized as a potential risk associated with AI improvement, seem to be coming true. New models have capabilities on par with the best human hackers, marking a pivotal window of opportunity in both AI and cybersecurity policy. This is a transitional period where new technologies are pushing existing American cybersecurity infrastructure to the brink. The real question isn’t whether cybersecurity still matters, but rather: How will the risks that AI introduces be managed before they outpace defenses, and who will step up to lead this challenge?

Attempts to control access to models with powerful cybersecurity capabilities, such as the federal government’s export controls (and their subsequent revocation) on Anthropic’s Mythos and Fable models, can only ever be a temporary solution. As with previous generations of AI models, other companies will soon catch up and develop models with Mythos-level capabilities. OpenAI was already hot on Anthropic’s heels with its GPT-5.5 model; more recently, Chinese lab Z.ai released its open-weight GLM-5.2 model, which early research suggests may be on par with Anthropic and OpenAI’s latest models when it comes to cybersecurity. Controlling AI is nearly impossible when foreign companies race to build more powerful models and release them publicly, so anyone with sufficient computing power can modify them for their own purposes.

The only long-term solution is to invest in defense. 

The problem is that defensive efforts haven’t kept up with the pace of AI progress. The federal government cut resources to key agencies like CISA and redistributed their authorities. This created a gap that AI companies have filled by taking on responsibilities that should be government-led. Some examples are Anthropic’s Project Glasswing and OpenAI’s Patch the Planet initiative, which aim to shore up critical infrastructure providers and open-source software libraries. AI companies have some incentives to invest in defense, both to improve public relations and strengthen software supply chains that they also rely on—but only to a certain extent. Unlike the public sector, they are incentivized to limit liability and blowback associated with irresponsible corporate behavior, not to secure the nation or its citizens. It’s a good thing that OpenAI and Anthropic have publicly committed to improving U.S. cyber defense. However, they are only positioned to help with one part of a very large problem. 

AI companies shouldn’t be expected to singlehandedly coordinate U.S. cyber defense, because many of the most urgent fixes have nothing to do with AI. Right now, AI companies can use their most powerful models to find software vulnerabilities and write patches. This is undoubtedly important, but the real challenge is making sure patches actually work and deploying them to key systems without causing problems. This is especially true for critical infrastructure, which relies on systems that are fragile, understaffed, and required to run continuously. 

AI companies bear responsibility for cyber defense, especially given the threats their own technologies create. But this responsibility is shared with other companies and the government.  Critical infrastructure owners and operators, government agencies, and corporations all need a trustworthy source of information to judge the evolving risk landscape and to outline the options to reduce that risk. Traditionally, the federal government has played the role of an information clearinghouse, receiving intelligence from both the public and private sectors and releasing guidance to benefit various stakeholders. Responding to and recovering from cyberattacks has traditionally been the government’s job. It should stay the government’s job, not become an AI company responsibility. 

There is no question that cyberattacks, whether powered by AI or not, will happen in the future. Leaders should strengthen our defenses by doing the following: measuring our exposure to attack, testing how systems perform under attack, and shortening recovery times. AI companies have introduced new threats and should help address them, but they can’t replace the government’s role. So far, the federal government has only reacted to AI and cyberthreats instead of planning ahead. What we need is real long-term cybersecurity strategy, not quick-fixes like blocking individual model releases. 

Everyone sees the threat coming—the question is whether or not we have the will to do anything about it before it’s too late.

Jessica Ji is a senior research analyst at Georgetown University’s Center for Security and Emerging Technology (CSET), where she works on the CyberAI Project.

Andrew Lohn is a senior fellow at Georgetown University’s Center for Security and Emerging Technology (CSET), where he works on the CyberAI Project.

The post Why blocking AI models won’t stop the cyber threats they create appeared first on CyberScoop.

AI-generated code is part of everyday software development. Developers use it to prototype, refactor, troubleshoot, and move from idea to implementation with less friction than ever before. The productivity gains are undeniable, which means that security leaders now face a hard question: whether their organizations can govern the risk that AI creates at that same speed.

That challenge is rooted in scale. AI changes how quickly software can be created, while many application security programs still depend on controls designed for a slower development model. When code generation accelerates beyond the capacity to review, test, and remediate issues, security debt accumulates faster.

That is the hidden cost of AI-assisted development. Risk now enters the enterprise at machine speed, while many organizations still manage it with human-scale processes. CISOs should govern AI-generated code as a high-risk input: tested automatically, checked for unsafe dependencies, remediated quickly, and blocked from production if it fails policy.

The metric that matters is risk velocity

Application security has long been measured through discovery. Teams count vulnerabilities, categorize severity, report trends, and show whether the numbers are improving. Those questions still matter, but AI adds a more urgent metric: risk velocity. Security leaders need to know how quickly the organization creates new software risks and how quickly it can reduce or eliminate them.

AI changes the economics of security debt. A development team that produces significantly more code without a matching increase in security capacity will create more issues than it can reasonably review or fix. Even when AI-generated code is comparable to human-written code on a per-line basis, the total risk can rise because the volume of change is higher. The backlog grows, vulnerabilities persist, and security debt eventually constrains the business.

AI expands familiar failure modes

The failure modes are familiar. AI coding tools can reproduce insecure patterns found in training data, including weak input validation, unsafe authentication flows, insecure direct object references, hard-coded secrets, and vulnerable dependency choices. They can also miss the context that determines whether code is secure in a specific environment: authorization models, tenant boundaries, data sensitivity, production configurations, and how services interact in a real application.

There is also a human factor. Under the pressure of deadlines, developers may accept code that works without fully understanding how it does so. The result is misplaced confidence. Code compiles, tests pass, features ship, and hidden risk enters the system. Over time, the organization may lose sight of the security concerns that naturally arose during manual development.

The supply-chain risk is bigger than the code itself

The software supply chain adds another layer of risk. Modern applications are assembled from open-source components, frameworks, plugins, containers, APIs, and cloud services. AI coding tools can recommend outdated packages, vulnerable libraries, or nonexistent dependencies. Veracode’s 2025 GenAI Code Security report found that AI coding tools produce insecure code nearly half (45%) of the time. It may sound like an amusing hallucination until attackers register malicious packages with similar names and wait for developers or automated tools to pull them in. At that point, a coding shortcut becomes a supply chain exposure.

AI is already part of the development lifecycle, and its use will continue to expand. Security teams need a control model built for that reality.

“Shift Left” needs an enforcement layer

The industry has spent more than a decade moving security earlier in the development lifecycle, improving visibility and helping teams catch issues sooner. Many organizations, however, moved findings closer to developers without also moving enough ownership, automation, and remediation capacity with them. Developers received more alerts, while security teams gained more visibility into risks they still struggled to reduce.

AI makes that operating gap more urgent. As software output increases, security cannot remain a checkpoint near the end of the process. It must become a continuous control system built into the way software is created, tested, approved, and deployed.

Secure-by-design has to become infrastructure

Secure-by-design in the AI era requires an engineering environment where unsafe choices are harder to make and easier to catch. Approved frameworks, secure defaults, reference architectures, dependency controls, automated testing, and policy enforcement should be embedded directly into developer workflows and CI/CD pipelines.

Remediation also must move closer to the point of creation. When a coding assistant introduces a vulnerable pattern, the ideal response is an inline fix that is proposed, validated, and governed as part of the normal development process. AI can help defenders here when it is connected to reliable security signals, policy context, and evidence from real testing. Counterintuitively, developers using AI to write code often don’t trust AI to automatically remediate code without human review. This takes one of the best ways to keep up with machine-speed created vulnerabilities and slows it down to human speed. An acceptable balance between risk and speed must be found.

Approval is not governance

CISOs should focus on governance, not just approving AI coding tools. Governance means tracking where AI-generated code enters your environment, documenting the policies and tests applied, recording what issues were found and fixed, and keeping proof of these decisions. This documentation becomes critical as AI-assisted development becomes standard. If vulnerable code reaches production, you’ll need to show that adequate controls were in place and risks were managed according to policy.

What leaders should do now

CISOs and engineering leaders should treat AI-generated code as untrusted until proven otherwise. They must require automated testing before release, enforce dependency controls, prioritize remediation based on exploitability and business impact, and measure success by the rate at which critical risk is reduced.

Additionally, boards and organizational policymakers should ask whether organizations can demonstrate that AI-assisted software is governed before it is deployed. The key evidence should include the policies applied, the tests performed, the vulnerabilities remediated, the risks accepted, and the approvals recorded. Today, many organizations can confidently track what their AI tools produce, but they cannot demonstrate how that output was secured, reviewed, and governed before reaching production. The industry is still working to close this gap.

AI is changing how quickly software risk moves through the enterprise. The organizations that succeed will make security move just as quickly by embedding governance, remediation, and proof directly into the software delivery pipeline.

The post AI-generated code has made security debt a governance problem appeared first on CyberScoop.