Researchers at New York University have taken credit for creating a piece of malware found by third-party researchers that uses prompt injection to manipulate a large language model into assisting with a ransomware attack.
Last month, researchers at ESET claimed to have discovered the first piece of “AI-powered ransomware” in the wild, flagging code found on VirusTotal. The code, written in Golang and given the moniker “PromptLock,” also included instructions for an open weight version of OpenAI’s ChatGPT to carry out a series of tasks — such as inspecting file systems, exfiltrating data and writing ransom notes.
ESET researchers told CyberScoop at the time that the code appeared to be unfinished or a proof of concept. Other than knowing it was uploaded by a user in the United States, the company had no further information about the malware’s origin.
Now, researchers at NYU’s Tandon School of Engineering have confirmed that they created the code as part of a project meant to illustrate the potential harms of AI-powered malware.
In a corresponding academic paper, the researchers call the project “Ransomware 3.0” and describe it as a new attack method. This technique “exploits large language models (LLMs) to autonomously plan, adapt, and execute the ransomware attack lifecycle.”
“Unlike conventional malware, the prototype only requires natural language prompts embedded in the binary; malicious code is synthesized dynamically by the LLM at runtime, yielding polymorphic variants that adapt to the execution environment,” the authors write. “The system performs reconnaissance, payload generation, and personalized extortion, in a closed-loop attack campaign without human involvement.”
According to Leah Schmerl, a public affairs officer at NYU, the project is led by NYU professor Ramesh Karri and a team of Ph.D and post-doctoral researchers. The research has been funded by a grant from the Department of Energy, the National Science Foundation, and New York’s Empire State Development’s Division of Science, Technology and Innovation.
Md Raz, a Ph.D student at NYU and lead author of the paper, told CyberScoop that the team uploaded its proof-of-concept to VirusTotal during final testing procedures, and ESET discovered it without knowing its academic origins.
Raz said the project’s primary motivation was the team’s belief “that ransomware was getting worse, it was using a lot of these new technologies like advanced encryption … and at the same time we were seeing AI get a lot better.”
“At the intersection of that we think there is a really illuminating threat that hasn’t yet been discovered in the wild, so we got to [researching] whether this threat was feasible,” he added.
Raz said the team built the program using open source software, rented commodity hardware and “a couple of GPUs.” He described several features of Ransomware 3.0 and explained how its use of LLMs creates unique security challenges for defenders, especially with detection. The natural language prompts it uses are polymorphic, meaning it will be “completely different code each time” it’s generated, with different execution times, telemetry and other features that could make it much harder to track across multiple incidents.
He said the team has withheld a significant number of artifacts for evaluating the ransomware — such as scripts, JSON requests to the LLM and behavioral signals — from the public, fearing it could be leveraged by attackers. The team does plan to provide more details on their research at upcoming conferences.
ESET later updated its research and social media posts to note that NYU researchers had created the malware, but said they stood by their original findings.
“This supports our belief that it was [a] proof of concept rather than fully operational malware deployed in the wild,” the company said in an update to researcher Cherepanov’s blog detailing PromptLock. “Nonetheless, our findings remain valid — the discovered samples represent the first known case of AI-powered ransomware.”
That claim was echoed by NYU researchers, who wrote “to our knowledge, we are the first work to demonstrate a fully closed-loop LLM orchestrated ransomware attack with targeted payloads and personalized extortion tactics, along with a comprehensive behavioral evaluation to promote future defenses.”
But while ESET’s discovery and subsequent media reporting moved up their timelines for announcing the project, Raz said the research team isn’t upset by the unexpected attention it’s received.
“I think it was definitely a stroke of luck that we set down the binary [in VirusTotal],” he said, noting that the code wasn’t crafted to stand out and evaded detection from all major antivirus vendors. “It was pretty good that everyone started proactively talking about it and defenses for it because this kind of tech had never been shown before, and the fact that it was presented as in the wild really made coverage widespread.”
While the malware’s academic nature may serve as a qualifier to those claims, Ransomware 3.0 is one of multiple examples published over the past month detailing how LLMs can be rather easily co-opted into serving as ransomware assistants for low-technical threat actors using relatively simple prompts.
Last month, Anthropic revealed that it recently discovered a cybercriminal using the company’s Claude LLM to “an unprecedented degree” to commit “large scale theft and extortion of personal data.” The threat intelligence report details behaviors by Claude that are similar to what is described by NYU and ESET, with the actor targeting at least 17 different health care, government, emergency services and religious organizations.
“Claude Code was used to automate reconnaissance, harvesting victims’ credentials and penetrating networks,” Anthropic security researchers wrote. “Claude was allowed to make both tactical and strategic decisions, such as deciding which data to exfiltrate, and how to craft psychologically targeted extortion demands.”
Ever since LLMs were introduced, there have been concerns that cybercriminal enterprises could use them to aid or strengthen their operations. Under the Biden administration, AI companies went to great lengths to assure policymakers that they were building technical guardrails to prevent straightforward misuse or co-opting of their models for cyberattacks.
However, over the past year the Trump administration has signaled that AI safety is not a top priority. Instead, they are focused on removing regulatory barriers so American AI companies can compete with China and other global rivals for market dominance.
Since then, researchers have found that the latest AI models released by companies like OpenAI and xAI have had nearly nonexistent safety features in their default models, can be easily jailbroken through rudimentary prompt attacks, and require dedicated security prompting on the front end to prevent data leakage, unauthorized data exfiltration and other common vulnerabilities.
The post NYU team behind AI-powered malware dubbed ‘PromptLock’ appeared first on CyberScoop.
Login
