As businesses and governments turn to AI agents to access the internet and perform higher-level tasks, researchers continue to find serious flaws in large language models that can be exploited by bad actors.

The latest discovery comes from browser security firm LayerX, involving a bug in the Chrome extension for Anthropic’s Claude AI model that allows any other plugin – even ones without special permissions – to embed hidden instructions that can take over the agent. 

“The flaw stems from an instruction in the extension’s code that allows any script running in the origin browser to communicate with Claude’s LLM, but does not verify who is running the script,” wrote LayerX senior researcher Aviad Gispan. “As a result, any extension can invoke a content script (which does not require any special permissions) and issue commands to the Claude extension.”

Gispan said he was able to execute any prompt he wanted, blow through Claude’s safety guardrails, evade user confirmation and perform cross-site actions across multiple Google tools. As a proof of concept, LayerX was able to exploit the flaw to extract files from Google Drive folders and share them with unauthorized parties, surveil recent email activity and send emails on behalf of a user, and pilfer private source code from a connected GitHub repository.

The vulnerability “effectively breaks Chrome’s extension security” by creating “a privilege escalation primitive across extensions, something Chrome’s security model is explicitly designed to prevent,” Gispan wrote.

Claude relies on text, user interface semantics, and interpretation of screenshots to make decisions, all things that an attacker can control on the input side. The researchers modified Claude’s user interface to remove labels and indicators around sensitive information, like passwords and sharing feedback, then prompted Claude to share the files with an outside server.

That means cybersecurity defenders often have nothing obviously malicious to detect. Where there is visible activity, the model can be prompted to cover its tracks by deleting emails and other evidence of its actions.

Ax Sharma, Head of Research at Manifold Security, called the vulnerability “a useful demonstration of why monitoring AI agents at the prompt layer is fundamentally insufficient.”

“The most sophisticated part of this attack isn’t the injection, but that the agent’s perceived environment was manipulated to produce actions that looked legitimate from the inside,” said Sharma. “That’s the class of threat the industry needs to be building defenses for.”

Gispan said LayerX reported the flaw to Anthropic on April 27, but claimed the company only issued a “partial” fix to the problem. According to LayerX, Anthropic responded a day later to say that the bug was a duplicate of another vulnerability already being addressed in a future update.   

While that fix, issued May 6, introduced new approval flows for privileged actions that made it harder to exploit the same flaw, Gispan said he was still able to take over Claude’s agent in some scenarios.

“Switching to ‘privileged’ mode, even without the user’s notification or consent, enabled circumventing these security checks and injecting prompts into the Claude extension, as before,” Gispan wrote.

Anthropic did not respond to a request for comment from CyberScoop on the research and mitigation efforts.

The post Flaw in Claude’s Chrome extension allowed ‘any’ other plugin to hijack victims’ AI appeared first on CyberScoop.

ChatGPT users beware: your browser extensions could be used to steal your accounts and identity.

LayerX Research has identified at least 16 Chrome browser extensions for ChatGPT floating around the internet that promise to enhance work productivity. All show signs of being built by the same threat actor and designed for the same purpose: to pilfer account credentials.

According to security researcher Natalie Zargarov, as legitimate AI browser extensions have become more widely used, “many of these extensions mimic known brands to gain users’ trust, particularly those designed to enhance interaction with large language models.”

“As these extensions increasingly require deep integration with authenticated web applications, they introduce a materially expanded browser attack surface,” Zargarov wrote.

That’s what the threat actor appears to have done in this case. The malicious extensions do not deploy malware or attack the model directly, they instead exploit vulnerabilities in the web-based authentication process used to verify ChatGPT users.

In order to work, many of these tools need access to authenticated AI sessions and high-level execution privileges within the browser itself. That combination of “high privilege, user trust and rapid adoption” makes them attractive targets to compromise for threat actors.

All but one of the extensions compromised their victims in the same way. A script injected into chatgpt.com monitors outbound requests coming from the ChatGPT web application. When a request goes out containing authorization details and the user’s session token data, the malicious extension extracts the information to a remote server.

With the user’s token in hand, the attackers can use them to authenticate ChatGPT sessions under the victim’s identity, access chat histories and applications that connect ChatGPT to other sensitive data sources, like Slack and GitHub.

Beyond token theft, the browser extensions also send metadata, usage telemetry and backend-issued access tokens used by the extension service to a third-party server.

The browsers share similar codebases used across different identities, consistent publisher characteristics across multiple listings and “highly similar icons, branding and descriptions.” In addition to their overlapping advertised functionality for enhancing productivity, they also displayed overlapping behaviors such as uploading batches of extensions on the same day, synchronized updates to several extensions at once, share backend infrastructure and web domains.

According to Zagarov’s blog, all 16 of the malicious extensions remain available on the Chrome Web Store today. CyberScoop has reached out to Google, which manages the Chrome browser, for comment.

All told, downloads have been low: about 900 total across the 16 browser extensions LayerX identified. Zagarov notes this is “a drop in the bucket” compared to other major browser extension campaigns like GhostPoster, which was downloaded more than 830,000 times and the Roly Poly VPN extension, which had over 31,000 documented installations.

But Zagarov said given the increasing popularity of AI browser extensions and the evidence that other actors are targeting the same weaknesses, time is not on defenders’  side.

“It just takes one iteration for a malicious extension to become popular,” Zargarov wrote. “We believe that GPT optimizers will soon become as popular as (not more than) VPN extensions, which is why we prioritized the publication of this analysis. Our goal is to shut it down BEFORE it hits critical mass.”

The post Some ChatGPT browser extensions are stealing your data appeared first on CyberScoop.

ON SECURITY By Susan Bradley The end of one year and the beginning of the next is a celebrated milestone, which is why some of us pundits recommend using some of that festive time to review your tech and tame it. Privacy is always top of mind, and it’s tempting to lock things down as […]