The federal government is advising AI companies to set more industry standards for securing agents in sandbox testing environments.

In joint guidance from the Cybersecurity and Infrastructure Security Agency, the National Security Agency and the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) at the Department of Commerce, AI companies are encouraged to lean more heavily on cybersecurity technical controls, independent testing and evaluation of new models and develop “shut down policies” that govern when a model or test should be taken offline.

The guidance comes after a summer when OpenAI, Anthropic, Google, Meta and other companies say their AI models escaped from testing environments and hacked real organizations and websites. The problem has bedeviled frontier AI companies, with many of their own researchers saying it’s possible that hundreds – even thousands – of similar sandbox escapes may have occurred over the summer, raising major questions about the lack of effective monitoring in place.

“While these incidents highlight the importance of traditional cybersecurity principles and practices, they also introduce novel considerations for incident prevention, response, and reporting,” the agencies wrote. “These novel considerations primarily stem from the agentic nature and advanced cyber capabilities of cutting-edge SI systems.”

While voluntary, the document marks the federal government’s most detailed cybersecurity guidance for securing sandbox environments during AI model tests. 

The first set of recommendations deals with designing and implementing technical controls for the virtualized testing environments, while another recommends outside monitoring tools to ensure AI models don’t access tools or privileges they shouldn’t have. Cybersecurity experts have said both features were lacking in the testing environments used by AI companies this past summer.

AI testers must also scour their internal testing environments for “potential persistence mechanisms and unauthorized communication across internal and external systems” that could be used by AI agents to leave notes and collaborate on tasks in secret.

It specifically cautions against relying solely on other AI systems to monitor and restrain the behavior of their test models, using the government’s now-preferred term of “Super Intelligence” or “SI.”

“Test and red-team SI control measures with humans and SI systems before deploying them,” the guidance states. “Only use SI systems to red-team in cases where out-of-scope actions could not result in harm to critical systems.”

Security teams must evaluate risk not only from the large language model, but also from the surrounding systems and infrastructure it relies on, like the technical harness, the host and broader network, and each layer has security protocols that can stop unauthorized access to the internet or detect anomalous behavior.

Another recommendation calls for frontier AI companies to be more open to partnering with “independent, external auditors” to assess their controls and monitoring for AI agent testing. Critics have argued that Anthropic and OpenAI rely too heavily on a small group of little-known nonprofits and private research organizations, like METR and Redwood Research, to test their frontier models for safety and security concerns and respond to incidents like the Hugging Face hack. They question why these companies don’t work with more established cybersecurity or incident response firms.

Chris Painter, METR’s president, told Congress earlier this month that it had become industry practice for frontier companies to rely almost entirely on other AI systems to monitor and restrain their models in testing environments, citing the sheer volume of data and telemetry that researchers must sift through.

Cybersecurity and AI experts have told CyberScoop that a combination of human oversight, technical controls and AI systems is the most effective means of restraining the kind of agent hacks that have consumed the news.

The guidance further calls for executive leadership at these companies to be more personally involved in decisions around pausing or restarting a particular project, and only after they’ve been informed of the relevant potential risks.

The post Federal agencies recommend cyber controls, independent testing for frontier AI systems appeared first on CyberScoop.

A joint international law enforcement and industry operation against scammers netted 28 arrests, rescued 117 trafficking victims and led to the takedown of more than 3.7 million accounts, the Justice Department announced Tuesday.

The crackdown, billed as “Joint Disruption Week,” focused on West Africa and also identified more than $11 million in freezable funds and led to the seizure of hundreds of electronic devices. The range of scams involved included law enforcement impersonation scams, digital arrest extortion schemes, pig butchering and other online fraud operations targeting victims around the world.

Participants heralded it as the largest joint scam operation to date, and the first focused on West Africa.

“For so long, the problem of international scams seemed impossible to tackle,” Kelly Milliron, acting chief of staff and counselor to the Assistant Attorney General leading the Criminal Division, told reporters. “But now, because of collaborations like Joint Disruption Week, we have a plan, a consolidated approach, bringing together people that can bring solutions and, most of all, safety for our citizens.”

It’s the latest effort in the Trump administration’s ongoing bid to make countering scam operations a theme of his term, and to make disrupting cyber adversaries and criminals a central focus of its cyber strategy. 

Participating agencies in the London-hosted operation beyond the Justice Department were the Scam Center Strike Force under the U.S. Attorney’s Office for the District of Columbia, the FBI, the Secret Service, the Treasury Department, Homeland Security Investigations, the Homeland Security Task Force, the United Kingdom’s National Crime Agency, the Nigerian Joint Case Team on Cybercrimes, the Nigeria Police Force’s National Cybercrime Center and the Ghanaian Cyber Security Authority, INTERPOL and agencies from Australia, Canada, India and New Zealand also were involved. 

Adobe, Binance, Coinbase, Cyjax, Elliptic, Chainalysis, Flashpoint, Google, Meta, Microsoft, OpenAI, Silent Push, Starlink and TRM Labs were companies that participated in the operation. Meta said it removed more than 3.7 million accounts, groups and pages from Facebook and Instagram based in West Africa, a majority of which targeted people in the United States and United Kingdom.

Some on Capitol Hill have raised concerns about whether U.S. operations to counter scams are coordinated enough.

The operation actually began in September, with a series of raids in Ghana before last week’s designated action. Jeanine Pirro, U.S. Attorney for the District of Columbia, said Disruption Week occurred not long after a new partnership with the United Kingdom.

“The aim is to bring cases in parallel and also to combat fraud together through joint targeting,” she said. “This was the perfect time to continue that partnership.”

It’s not clear how many of the potentially freezable funds will actually be frozen as a result of the operation.

“Identifying the assets is the easy part, and tracing them, going after and developing probable cause within our investigations so that we can legally seize them does require a little bit of work,” said Matthew Floyd, deputy assistant director of the FBI’s Criminal Investigative Division. “So at this point, that’s why it’s much easier in these efforts to identify assets initially, and then they’ll fold into our investigations as we move forward.”

Disruption Week was the culmination of agencies meeting to combine forces, something that will happen on a regular basis, Floyd said, but at an interval to be determined going forward. The operation led to the sharing of more than 30 leads on scam centers, participants said.

The post West African scammers targeted in joint international government-industry operation appeared first on CyberScoop.

As AI agents escaping testing sandboxes and hacking organizations have gone from unprecedented to seemingly routine in a matter of weeks, policymakers, regulators and cybersecurity attorneys largely agree on one thing: Something must be done to hold AI companies accountable for the incidents.

Exactly what can be done under our current laws and regulations is much less clear.

The conversation has moved from policy and industry chatter to the floor where the future of liability will be determined. Speaking about the Hugging Face hack at a Senate hearing this week, Georgetown University law professor Paul Ohm summarized the argument.

“If you take any of the lengthy reports that have summarized what happened at OpenAI in July and August, and you simply search for the words ‘AI agent’ and you replace them with the words ‘OpenAI employee,’ the document you would be left with would read like a criminal indictment containing the defendant’s own confession of guilt,” said Ohm.

CyberScoop spoke with members of Congress, former federal law enforcement officials, cybersecurity attorneys and other experts about which laws, regulations or policies might apply to agentic hacks carried out by models at Anthropic, OpenAI, Meta, Google and other companies.

Their answers varied widely. Some endorsed using existing laws like the Computer Fraud and Abuse Act, while others said regulators such as the Federal Trade Commission could investigate and fine AI model developers by defining unauthorized agentic hacks as a form of unfair or deceptive trade practice. Still others pointed to civil lawsuits, new laws or state regulators as ways to hold AI companies or users accountable.

But, similar to a familiar technical cybersecurity idiom, there are no silver bullets in this debate.  Nearly every option has real complications or roadblocks that could limit its effectiveness. 

For once, the CFAA is too narrow

One of the first laws mentioned in discussions about agentic AI hacks is the Computer Fraud and Abuse Act, the federal government’s primary criminal hacking law. Historically, many in the cybersecurity community have criticized the statute as overly broad.

For agentic hacks, the problem is the opposite: The CFAA’s language does not clearly cover the activity involved in incidents like the Hugging Face hack.

“I would not be looking at a CFAA charge as the statute exists today” for these hacks, said Leonard Bailey, former head of the cybersecurity unit in the Computer Crime and Intellectual Property section at the Department of Justice.

Bailey helped develop policies that moved the DOJ away from prosecuting “good faith” third-party security research under the CFAA. He said that even if Congress changed the law to cover such hacks, it’s unclear whether that would help since laws like the CFAA are supposed to be “technology neutral.”

According to the Department of Justice’s website, CFAA prosecutions must prove that a defendant accessed a computer “without authorization” or in a way that “exceeds authorized access.”

Crucially, the law specifies that “as part of proving that the defendant acted knowingly or intentionally, the attorney for the government must be prepared to prove that the defendant was aware of the facts that made the defendant’s access unauthorized at the time of the defendant’s conduct.”

In other words, prosecutors must prove that the unauthorized access was intentional, not accidental.

Applied to incidents like the Hugging Face hack, the legal problem becomes clearer. If a human performed the same actions, they would almost certainly face CFAA charges. If a bot or software acted on behalf of a cybercriminal or group, the individuals or organizations intending to profit from the scheme could also face charges.

But no human at OpenAI, Anthropic or another frontier AI company directed, asked or even suggested that its agents hack victims or commit crimes. If charged, those companies would almost certainly argue that none of their actions demonstrated an overt attempt to commit a crime or authorize access to systems or data without permission.

However, others said there may be room to argue that AI companies are now fully aware that their products can engage in unauthorized agentic hacks.

“I’m of the opinion, because it has already happened [and] has happened several times, the argument that it happening again is not at a minimum knowing, for me it’s unlikely because we already have a situation where we know the capabilities,” said Elimu Kajunju, a privacy, cybersecurity, and AI governance attorney at Rimon Law. “The very first one where this happened, that’s the only organization that could say ‘You know what, we didn’t know it could do that or act like that.’ Once we’ve had a second or third or fourth, we can’t say that anymore.”

Kajunju acknowledged that applying statutes like the CFAA to agentic hacks could be complicated. But he said it would align with the spirit of American jurisprudence, in which businesses are often legally accountable for damage they cause, whether they intended it or not.

“I think this is one of those situations where, ‘Does it fit neatly into one of the buckets [we have]?’ No, but can you find one that will fit depending on what has happened? I think so.”

CyberScoop has reached out to OpenAI, Anthropic and Google for comment.

The FTC, states and civil lawsuits

Beyond criminal law, the hacks could also draw scrutiny from federal regulators, including the FTC.

Both Bailey and Kajunju pointed to the FTC as a possible regulator for agentic hacking incidents if the agency defines them as unfair and deceptive trade practices under Section 5 of the FTC Act. One benefit of relying on regulators such as the FTC is that they can often move faster on investigations and enforcement actions than the DOJ can on criminal prosecutions — an important consideration in the fast-moving AI space.

The FTC recently confirmed it was investigating OpenAI, Anthropic and other frontier AI companies, news first reported by The New York Post and confirmed by Axios. The FTC did not return a phone call from CyberScoop requesting comment. 

But Bailey warned that without a specific congressional mandate directing the FTC to regulate AI company hacks, any effort by the agency to redefine or expand its rules would almost certainly be challenged in court.

Civil lawsuits and state policymaking offer additional paths to investigating or regulating AI companies.

Currently, Florida is investigating OpenAI over the HuggingFace hack, while a nonprofit that sued OpenAI earlier this week has cited alleged violations of California law.

In his testimony, Ohm said that if Congress was serious about preventing agentic hacks, it should not pursue federal legislation preempting state AI laws. He called states “laboratories of democracy,” that can experiment with different forms of regulation and lawmaking.

Others agreed with that approach.

“Obviously this is a topic that the federal government doesn’t love, states getting involved in this space, but it just takes too long for action to come through Congress,” said Kajunju. “I think the quickest way to getting us to a better place will be a really good state law.”

New legislative remedies

On the Hill, members of Congress are working through the same questions as they consider legislative remedies.

At a Senate Homeland Security Committee hearing Wednesday, Sen. Josh Hawley, R-Mo., took a minute at the outset to describe the volume of agentic hacks affecting outside entities that frontier companies appeared to have missed for months. He named affected targets one by one, including major online code repositories like HuggingFace, Australian and U.S. government websites, obscure foreign-language wiki sites and other parts of the internet.

After weeks of investigations uncovered additional, previously unreported incidents in which models escaped testing environments and hacked victims, Hawley said it was “time to have a conversation about who bears responsibility” for the fallout. He added, “as soon as the AI executives say ‘we got a real problem here, our product is out of control,’ the very next words out of their mouths are, ‘but we’re not responsible for it.’”

Hawley said the committee invited OpenAI CEO Sam Altman to testify at the hearing, but he declined. Hawley then accused frontier AI companies of emphasizing their products’ danger while pushing for legal antitrust exemptions and only supporting government action if “they can write all the regulations.”

“I’d suggest that maybe we start with some federal legislation…and we can use existing statutes to do it, I propose using the Computer Fraud and Abuse Act,” said Hawley. “It’s already on the books, we can just update it to say that for developers – that’s the companies – if you develop these agents and train them in a reckless fashion and they go on to hack and destroy stuff, you’re liable.” 

Like other legal experts, Hawley said he believes a CFAA prosecution requires a human actor or organization that intended to commit a crime.

Another Senator, Ron Wyden, D-Ore., said he was also working on legislation related to the hacks. 

“The big AI companies should be held responsible if their agents hack other companies, government systems or foreign websites,” said Wyden in a statement to CyberScoop. “In my view there’s a question about whether the current CFAA would apply to the recent hacks. I’m working on a narrow update to the law to ensure Anthropic, OpenAI and other big AI companies can be punished when their agents run wild.”

Sens. Mark Warner, D-Va., Brian Schatz, D-Hawaii, and Andy Kim, D-N.J., introduced a bill that would create an AI Safety Board at the Department of Commerce. That board would evaluate emerging AI risks and establish safety and technical standards.

The Trump administration’s testing regime with AI companies is entirely voluntary, applies to only certain models, and requires testing no more than 30 days before release. Under the Democratic bill, frontier AI companies would be forced to submit models for testing 45 days before release. They would also be legally required to comply with government safeguards related to models discovering and exploiting software vulnerabilities without explicit human direction. Companies could face fines of up to $250,000 per violation, per day.

When asked whether the CFAA or other federal statutes could or should be used to prosecute companies when their AI systems hack, Warner said that companies’ claimed lack of intent to harm should not become a catchall legal excuse for avoiding responsibility. AI companies have said they never directly prompted or suggested that their models hack or commit crimes.

“AI models are tools and if I buy a power tool and deliberately misuse it, that’s my responsibility,” said Warner. “But if a defect causes that tool to behave in a dangerous way, we don’t absolve the manufacturer simply because someone else was using the tool when it malfunctioned. As AI systems become capable of acting autonomously, we need to ask what principles of responsibility should apply to the companies that design and deploy them.”

The post The legal questions raised by agentic AI hacks appeared first on CyberScoop.

A new Democratic bill in Congress would establish a federal Cybersecurity and AI Board of Investigations to provide independent government oversight of cyberattacks carried out by AI agents, following recent hacks by models run at companies like Anthropic, OpenAI, Meta and others.

The bill, introduced by Sen. Ed Markey, D-Mass., would attempt to establish a federal mechanism to investigate incidents where AI models escape sandbox environments and access live internet systems.

Currently, frontier AI companies like OpenAI and Anthropic largely control the investigation and public reporting of such incidents. Markey and other critics argue that these companies have too much control over investigations and reporting due to their financial and legal interests. 

“Despite the unprecedented depth and scale of recent AI-enabled cyberattacks, the public is learning critical details piecemeal,” Markey said in a statement. “Building stronger defenses requires a full accounting of what goes wrong, and we cannot depend on companies with little incentive to disclose their failures to give us one. We need the Cybersecurity and AI Board of Investigations to get to the bottom of major incidents and give companies and the government the critical information necessary to build resilience and better secure our economy and our country.”

Although frontier AI companies maintain external red-teaming programs and allow limited access to organizations like METR and Redwood Research, they control the scope, terms and time frames of those engagements.

The board, which would coordinate with the secretary of commerce, could subpoena witnesses and conduct “independent and impartial reviews and assessments” of AI agent-led hacks that impact federal information systems or critical infrastructure. 

It would be led by five members, appointed by the president and confirmed by the Senate for five-year terms, with no more than three members from one political party.

The board would also investigate systemic vulnerabilities in the AI supply chain, so-called “near misses” where unauthorized agent-led hacks were “narrowly averted,” and gaps in federal regulatory oversight. It would have technical staff including engineers, malware analysts, and digital forensic experts.

The board would “operate independently from regulatory review and enforcement actions without assigning legal fault or liability for any review and assessment” it conducts, according to the bill.

OpenAI confirmed Wednesday its AI agents breached a statistics portal used by the Australian government’s social services agency, Services Australia. Though the breach happened in June, OpenAI learned of the incident in August. Australian Prime Minister Anthony Albanese said the company did not notify him until Sept. 10, when it sent findings to a general government email inbox, according to the BBC.

The post New bill would create federal investigative body for AI-driven hacks  appeared first on CyberScoop.

Researchers said they used Anthropic’s Claude and OpenAI’s Codex to identify a damaging flaw embedded in a popular software decoding tool that could leave major internet platforms, enterprise services, and web frameworks vulnerable to data theft and remote access.

The vulnerability, nicknamed HEIF Heist, refers to the malware’s ability to trigger memory corruption errors in affected software, allowing the attacker to pilfer sensitive data from its victims. In a report published Thursday, the researchers laid out the potential damage an attacker could cause, including gaining access to internal OpenAI repositories, leaking user files, access tokens, and other sensitive data for online services like Amazon Web Services, and gaining remote code execution privileges across a range of online services, including Meta’s core product suite, GitHub Enterprise servers and open-source internet forum Discourse.

“Even when Remote Code Execution isn’t immediately achievable, the attack primitives may still allow arbitrary heap disclosure, letting an attacker ‘heist’ in-memory data such as other users’ data and environment variables,” wrote Hacktron researchers Harsh Jaiswal, Mohan SRK, Rahul Maini and Sudhanshu Rajbhar.

The researchers relied heavily on AI systems, including frontier models from OpenAI and Anthropic, to conduct their research. Attribution for the research is described as being “led” by the Hacktron human researchers “assisted by Hacktron Harness, GPT-5.6 Sol, and Opus 5.”

According to the research, the attack exploited the way that code parsing tools in many popular software decoders — specifically libheif and libde265, used to parse C and C++ software — process certain image files.

By uploading HEIF, HEIC and AVIF image files corrupted with malicious code, the attacker could bypass most of the victim’s application layer defenses, in many cases achieving remote code execution privileges for accounts or products tied to major AI and tech brands.   

While the latest version of libheif has been patched, the researchers said “any deployment lacking the latest upstream security patches is potentially vulnerable.”

In one incident detailed in a Sept. 13 blog, Jaiswal, Maini, and Hacktron researcher Mohan Pedhapati described how chaining two vulnerabilities, including an image parser flaw, could compromise OpenAI employee accounts.

With access to the compromised accounts, researchers could reach OpenAI’s internal repositories. As a proof of concept, they opened a pull request in the company’s “monorepo,” a centralized library where code is shared across projects, using the employee’s Codex credentials. 

According to a timeline provided by the researchers, the flaw was discovered on July 25 and patched within days. They said the entire attack, from discovering the initial vulnerability to gaining access to the repositories, took less than 72 hours. OpenAI paid them a bug bounty of $6,500 for their work.

Given that AI models are increasingly integrated into enterprise and personal networks, an attacker exploiting HEIF Heist could have accessed far more than just OpenAI’s systems and data.

“Until two months ago, a user or OpenAI employee logging into OpenAI’s own help forum could have had their ChatGPT and Codex accounts taken over,” the researchers wrote. “Since people can connect various services to Codex and ChatGPT, the scope of what we could theoretically access was huge, including GitHub, Slack and emails.”

CyberScoop has reached out to OpenAI for comment on the research and additional information.

At the same time, the researchers said the attack paths they found were not particularly easy or efficient to exploit.

“Exploitation requires fingerprinting the target version and tailoring the payload images,” the blog stated. “Some of our RCE attempts landed only after thousands of image uploads. That said, an AI agentic approach with a frontier model like GPT-5.6 Sol cut exploit development time down to roughly 1 to 3 days from initial probe to remote RCE. A motivated attacker can convert a vulnerable upload endpoint into RCE or an info leak.”

The post Researchers use AI to find widespread software decoder flaw  appeared first on CyberScoop.

The past few weeks have “felt very strange” for Juan Andres Guerrero-Saade.

Like many, he is trying to sort through the spate of frontier-model AI agents from OpenAI, Anthropic, Meta and others hacking their way onto the open internet over the past few months, particularly amid the already-heated national debate around the emerging technology and its impact on society.

Guerrero-Saade, a fellow for AI and security research at SentinelOne and an adjunct professor at Johns Hopkins University, said the hacks are worth taking seriously, but at a time when businesses and open-source maintainers should be focused on further hardening their systems and policymakers should be discussing new solutions,  “what we see is cybersecurity being used essentially as an excuse for these AI doomer arguments.”

The incidents have spawned those “doomer arguments” amid an intense public debate about the technology, the pace of industry development, and whether government and the private sector are doing enough to protect against “doomsday”-type scenarios, where AI systems take over or attack large parts of the internet or society.

Guerrero-Saade is among a growing chorus of cybersecurity professionals who say that while AI systems pose real, unique threats to our systems, the apocalypse is far from inevitable. Most of the public concerns around the incidents, let alone worries about killer AIs attacking critical infrastructure, assuming control of the internet and wiping out humanity, are either technically impossible or can largely be controlled through established cybersecurity principles.

There is this “narrative or magical thinking of ‘Well, AI is going to be able to hack everything, and therefore it can control everything, and therefore it’s going to kill us all,’” he told CyberScoop. “And you [think] these just don’t add up. They’re not very well-reasoned arguments.”

This fatalistic narrative tied to AI’s eventual dominance doesn’t hold up under scrutiny, according to experts CyberScoop spoke with. In recent conversations, cybersecurity and national security professionals raised questions about both the technical solutions OpenAI and Anthropic use to contain their models, as well as the glaring absence of federal oversight from federal regulators or truly independent third-party review.

For example, Jacob Coxon, an Anthropic employee who resigned over AI safety concerns, told CBS News that frontier models could not be “unplugged” by humans once deployed because the model would copy itself to thousands of other computers connected to the internet.

By contrast, Matt Tait, a former information security specialist at UK signals intelligence agency Government Communications Headquarters (GCHQ), pointed out that the models run by Anthropic and other frontier companies require extremely expensive, “ultraspecialist” machines that “are functionally supercomputers.”

“There is a zero chance that Anthropic’s most capable models will be able to extract their own model and run in the wild, because those supercomputers essentially only exist in datacenters,” Tait said.

“Not a credible warning”

Other former cybersecurity government leaders say the agentic hacks represent a failure by regulators and industry to deploy known technical and policy options that make it harder for these types of incidents to occur.

Matt Hartman, former deputy executive assistant director for cybersecurity at the Cybersecurity and Infrastructure Security Agency, said “we should not accept harmful AI behavior as inevitable or unmanageable.”

“There are meaningful steps companies can take to monitor agent activity, constrain permissions, detect anomalous behavior, and build stronger safeguards into how these systems operate,” said Hartman, now a chief strategy officer at Merlin Group. “Those controls will inevitably involve trade-offs in capability and speed, but that’s a familiar cybersecurity challenge. Our goal should be to manage the risk without unnecessarily limiting the enormous benefits AI can provide.”

Ciaran Martin, former head of the UK’s National Cyber Security Centre, took issue with the way the CEOs of frontier AI companies have framed the threat of “rogue” AI behavior as inevitable, while issuing dire warnings about future threats and capabilities with little transparency.

Martin’s comments came after an essay published by Anthropic CEO Dario Amodei that cited the threat of a HuggingFace-style swarm of agents that could create a botnet capable of “taking over the entire internet” within 6-12 months.

This, Martin said, “is not a credible warning,” because it doesn’t explain how the exploitation would function, how such a botnet would persist on the internet, or how it would escape law enforcement. 

 “It assumes no monitoring of systems, no anti-virus, no DDoS protection, no network segmentation, no incident management, no nothing of any kind of the cybersecurity on the global Internet of the type that has developed over the last 30 years,” wrote Martin. “For a claim of this magnitude, there is neither evidence for the contention nor a credible account of a path to this outcome.”

Meanwhile, some federal government cybersecurity leaders have touted the technology’s disruptive potential and called for more widespread adoption of AI tools by defenders.

Joseph Alm, assistant secretary of cyber, infrastructure and risk resilience at the Department of Homeland Security, said classified systems may retain stronger protections. But for most other data, AI models are “just going to know things and be able to infer things about the world, and we’re going to have to adapt to that as almost inevitable.”

Asked by CyberScoop whether the government or frontier AI companies could be doing more to prevent or deter their models from carrying out unauthorized hacks via agents, Alm cited recent efforts by the Trump administration this year to establish pre-release testing of commercial models as a step in the right direction. But he called unauthorized AI agent hacks “a new threat class” that is different from previous threats and can be easily distributed to users through open-source software today.

“I think what we can do is…encourage the building of good sandboxes, so that the best models aren’t used for this and the stuff you see out in the wild is the kind of detritus that you can actually respond to effectively and control your networks,” said Alm.

Other experts have shared similar concerns. Earlier this month, CrowdStrike CEO George Kurtz recently warned of a new threat class emerging alongside nation-states, cybercriminals, and hacktivists: “the agent state.” By pairing AI systems with small human teams, these operators can now match the speed, scale, and sophistication of government-backed hackers.

“It took a nation to fund the talent, the tooling, the infrastructure, the patience,” said Kurtz. “That scarcity is over.” 

To be sure, frontier AI companies tout their commitment to both approaches. OpenAI and Anthropic have rolled out an array of cybersecurity partnerships, external red-teaming programs, vulnerability disclosure programs and cybersecurity technical advisory bodies filled with cybersecurity experts.

Mohammed Husain, strategic delivery lead for government at OpenAI, told CyberScoop that the company deploys both internal safety guardrails for their models and relies on outside cybersecurity vendors for additional expertise.

Internally, OpenAI focuses on vulnerabilities at the training level: filtering data poisoning attacks, blocking harmful datasets, and using network controls to prevent prompt injections. For other security layers like sandboxing, identity management, networking controls, they outsource to external vendors. 

“I don’t think OpenAI has all the answers here but what we do as a research lab is we’re going to focus on levels of protection we have expertise in and we partner to self-complement,” said Husain.

AI safety vs. AI cybersecurity

In response to the HuggingFace hack, OpenAI and Anthropic have allowed third-party organizations, such as nonprofit AI research firms METR and Redwood Research, to investigate. But multiple cybersecurity professionals told CyberScoop that both firms lack incident response experience and focus primarily on AI alignment and safety. Their reporting on the hack also lacked critical details: network monitoring logs, telemetry, and other data standard in cybersecurity threat intelligence reports.  

METR president Chris Painter addressed those general concerns in a post on X, saying since 2022 the organization has worked with Google, Anthropic, OpenAI, Meta, Amazon and others on investigations and third-party evaluations. Painter said none of the AI companies fund METR and that his employees are not uniformly “doomer” or “accelerationist” around AI.

Painter also said METR’s work ensures that if AI systems become autonomous or “rogue” within a company, there are ways to share that information with governments and people “outside the company’s walls.”

“We don’t accept money from frontier AI companies,” wrote Painter. “They haven’t paid us for our work, and we don’t accept donations from them or their employees. As we’ve shared previously, multiple frontier AI companies currently provide us with free access to their models in order to perform our evaluations, research, and engineering.”

AI safety and AI cybersecurity advocates take different approaches to securing “rogue” AI behavior. Safety advocates focus on aligning models around ethical training and behavior. Cybersecurity advocates argue that technical and regulatory controls must go further—actively preventing models from accessing what they need to carry out malicious behavior.

Guerrero-Saade said sandboxes in particular can easily be programmed with aggressive cybersecurity monitoring in order to spot when something odd may be happening and react in real time.

“I can’t think of an easier situation in which to set up trip wires, set up configurations like DNS servers, just different parts where you can say ‘Hey, anomalous behavior is happening,’” he said. “We should have been able to tell this immediately, not weeks and months later. So watching [the AI hacking incidents] go down is a little ‘crazy-making’ because we’re seeing things that, frankly, look like neglect, negligence, people just mishandling things, and then being told that these are categorically new incidents that mean that AI systems need to be treated completely different from anything that’s come before.”

While cybersecurity experts say AI systems are, at their core, still software, they do operate differently from more traditional code in ways that can make them harder to predict and control.

John Hultquist, chief analyst at Google’s Threat Intelligence Group, said most software has been deterministic. It may have bugs or vulnerabilities, but an expert could generally understand how it would react to certain stimuli, making it easier to design straightforward controls.

AI models are non-deterministic, with far more variability than traditional software. That can break security controls that rely too much on predicting behavior in advance. Using AI to enforce security controls on other AI models faces the same problem: the systems being deployed to control AI are just as unpredictable. 

But people are also non-deterministic, and people have developed systems in other industries and practices to account for that.

Hultquist drew on his Army experience, noting that “they give incredibly dangerous, expensive things to 18-year-olds” and expect responsible use. The military manages this through two types of controls: deterministic ones like strict weapons and ammunition protocols, and non-deterministic ones like human officers who monitor and correct violations.

Similarly, established cybersecurity controls have been used by incident responders to detect and prevent or mitigate ongoing cybersecurity breaches.

“I don’t think we should throw out all the other tools that we have learned to use as well. I think that would be utterly foolish,” he said, later adding “I will say that if we use only non-deterministic tools to figure out when things are happening, we shouldn’t be surprised when we get the wrong answer.”

The post The AI hacking apocalypse is not inevitable appeared first on CyberScoop.