On Wednesday, Microsoft released two new red teaming tools — Rampart and Clarity — meant to help developers design more secure agentic software and assist incident responders in the face of ongoing breaches.

Rampart is built on top of PyRIT, an existing open automation framework Microsoft developed for red teaming generative AI systems. But while PyRIT scans already-built systems for security flaws, Rampart is made to continuously test code for vulnerabilities during the development process, encoding both adversarial and benign testing scenarios into the software development pipeline to flag exploitable bugs and dependencies.

Microsoft said Rampart was built to focus on cross-prompt injection attacks, where “an agent retrieves or processes potentially poisoned content from documents, emails, tickets, and other data sources that manipulate behavior indirectly.” It also confirms fixes or exploits work as intended through multiple rounds of testing, as opposed to tools that perform “single shot validation.”

The second tool, Clarity, can be run as a desktop app, a web interface or directly embedded into a coding agent to provide real time security engineering guidance to developers at the outset of a project. It can categorize and track different business objectives related to the code and highlight downstream security implications along with more secure by design alternatives.

Ram Shankar Siva Kumar, who founded Microsoft’s AI red team in 2019, told CyberScoop that the company has seen internal security benefits from using the tools, but believesRampart and Clarity’s growth depends on contributions from other developers outside the Microsoft ecosystem.

In the fast-moving world of AI, where vibe coding, rogue AI agents and a steady churn of new model releases create fresh security implications nearly every week, Siva Kumar said it was important to begin building foundational, AI-centric security processes into the software development pipeline.

“When you hear a lot of talk about AI safety and security, it seems to be a lot of philosophical debates,” he said. “You’ll see frameworks, you’ll see white papers, and I think we’re really past that time, now. We really need to start thinking of AI safety as an engineering discipline and trying to bring security where the developers are.”

Rampart’s potential utility to defenders goes beyond just securing software development pipelines. It can also be used during an active incident response to speed up or automate red teaming for hot fixes, patching and remediation.

Microsoft has used Rampart when investigating reported vulnerabilities in their own products. Siva Kumar said the tool was able to help condense a week’s worth of manual work —  replicating the vulnerability, identifying different variants of the same bug, then patching and re-testing those variants to ensure they’re no longer exploitable — into hours.

Clarity, meanwhile, acts as a security adviser for software projects, prompting developers to consider potential risks in their design decisions and their downstream security consequences. With the rise of AI-generated code and agents, and execution becoming cheaper, this kind of proactive guidance is increasingly important.

“You’re going to be able to create apps, create MCP servers to pull things out from the internet,” said Siva Kumar. “The question is, ‘should you be doing it?’ And Clarity is a step in that direction. It is asking, ‘hey, should you be doing this in the first place?’”

The post Meet Rampart and Clarity, Microsoft’s new red team combo AI agents appeared first on CyberScoop.

Cybersecurity agencies from the United States, Australia, Canada, New Zealand and the United Kingdom jointly published guidance Friday urging organizations to treat autonomous artificial intelligence systems as a core cybersecurity concern, warning that the technology is already being deployed in critical infrastructure and defense sectors with insufficient safeguards.

The guidance focuses on agentic AI — software built on large language models that can plan, make decisions and take actions autonomously. In order for this software to function it needs to connect to external tools, databases, memory stores and automated workflows, allowing it to execute multi-step tasks without human review at each stage.

The guidance was co-authored by the U.S. Cybersecurity and Infrastructure Security Agency, the National Security Agency, the Australian Signals Directorate’s Australian Cyber Security Centre, the Canadian Centre for Cyber Security, New Zealand’s National Cyber Security Centre and the United Kingdom’s National Cyber Security Centre.

The agencies’ central message is that agentic AI does not require an entirely new security discipline. Organizations should fold these systems into the cybersecurity frameworks and governance structures they already maintain, applying established principles such as zero trust, defense-in-depth and least-privilege access.

The document identifies five broad categories of risk. The first is privilege: When agents are granted too much access, a single compromise can cause far more damage than a typical software vulnerability. The second covers design and configuration flaws, where poor setup creates security gaps before a system even goes live.

The third category covers behavioral risks, or cases where an agent pursues a goal in ways its designers never intended or predicted. The fourth is structural risk, where interconnected networks of agents can trigger failures that spread across an organization’s systems.

The fifth category is accountability. Agentic systems make decisions through processes that are difficult to inspect and generate logs that are hard to parse, making it difficult to trace what went wrong and why. The agencies also note that when these systems fail, the consequences can be concrete: altered files, changed access controls and deleted audit trails.

The guidance also flags prompt injection, where instructions embedded inside data can hijack an agent’s behavior to perform malicious tasks. Prompt injection has been a lingering problem with large language models, with some companies admitting that the problem may never be solved. 

Identity management gets significant attention throughout the document. The agencies recommend that each agent carry a verified, cryptographically secured identity, use short-lived credentials and encrypt all communications with other agents and services. For high-impact actions, a human should have to sign off, and the guidance is explicit that deciding which actions require that approval is a job for system designers, not the agent.

The agencies admit the security field has not fully caught up with agentic AI. Some risks unique to these systems are not yet covered by existing frameworks, and the guidance calls for more research and collaboration as the technology takes on a growing number of operational roles.

“Until security practices, evaluation methods and standards mature, organisations should assume that agentic AI systems may behave unexpectedly and plan deployments accordingly, prioritising resilience, reversibility and risk containment over efficiency gains,” the guidance reads. 

You can read the full guidance below.

The post US government, allies publish guidance on how to safely deploy AI agents appeared first on CyberScoop.

The tech giant found that many indirect prompt injection attempts are harmless, but some malicious exploits have also been identified.

The post Malicious AI Prompt Injection Attacks Increasing, but Sophistication Still Low: Google appeared first on SecurityWeek.

Security researchers at Noma Security have disclosed a new vulnerability they are calling GrafanaGhost, an exploit capable of silently stealing sensitive data from Grafana environments by chaining multiple security bypasses, including a method that circumvents the platform’s AI model guardrails without requiring any user interaction.

Grafana is widely deployed across enterprise organizations as a central hub for observability and data monitoring, typically housing real-time financial metrics, infrastructure health data, private customer records, and operational telemetry, among other uses. That concentration of sensitive information is what makes the platform a significant target. GrafanaGhost exploits how Grafana’s AI components process user-controlled input to bridge the gap between a private data environment and an external attacker-controlled server.

The attack requires no login credentials and does not depend on a user clicking a malicious link. It begins when an attacker crafts a specific URL path using query parameters originating outside the victim organization’s environment. Because Grafana handles entry logs, an attacker can gain access to an enterprise environment to which they have no legitimate connection. The attacker then injects hidden instructions that Grafana’s AI processes — a tactic known as prompt injection — using specific keywords to cause the model to ignore its own guardrails.

Grafana has built-in protections designed to prevent prompt injection, but Noma’s researchers found a flaw in the logic underlying that protection — one that could be exploited by formatting a web address in a way that Grafana’s security check misread as safe, while the browser treated it as a request to an external server the attacker controlled. The gap between what the security check believed it was allowing and what actually happened was enough to open the door for the attack.

The final obstacle was the AI model’s own instinct for self-defense. When researchers first attempted to slip malicious instructions past it, the model recognized the pattern and refused. After further study of how the model processed different types of input, they found a specific keyword that caused it to stand down, treating what was effectively an attack instruction as a routine and legitimate request.

With all three bypasses in place, the attack runs on its own. The AI processes the malicious instruction, attempts to load an image from the attacker’s server, and in doing so quietly carries the victim’s sensitive data along with that request in an image tag. The data is gone before anyone in the organization knows a request was ever made.

Noma’s researchers noted that multiple security layers were present in Grafana’s implementation, but each contained its own exploitable weakness. The domain validation logic, the AI model guardrails, and the content security controls all failed when approached in sequence. 

Because the exploit is triggered by indirect prompt injection rather than a suspicious link or an obvious intrusion, there is nothing for a user to notice, no access-denied error for an administrator to find, and no anomalous event for a security team to investigate. To a data team, a DevSecOps engineer, or a CISO, the activity is indistinguishable from routine processes.

“The payload sits inside what looks like a legitimate external data source. The exfiltration happens through a channel the AI itself initiates, which looks like normal AI behavior to any observer. Traditional SIEM rules, DLP tools, and endpoint monitoring aren’t designed to interrogate whether an AI’s outbound call was instructed by a user or by an injected prompt,” Sasi Levi, vulnerability research lead at Noma Labs, told CyberScoop. “Without runtime protection that understands AI-specific behavior, monitoring what the model was asked, what it retrieved, and what actions it took, this attack would be effectively invisible.”

The attack is another example of a broader shift in how adversaries are approaching enterprise environments that have integrated AI-assisted features. Rather than exploiting broken application code in the traditional sense, attackers are increasingly targeting weak AI security surfaces and indirect prompt injection methods that allow them to access and extract critical data assets while remaining entirely invisible to the security teams responsible for protecting them.

Noma has found similar issues over the past year, with Levi telling CyberScoop that researchers keep seeing the same fundamental gap: AI features are being bolted onto platforms that were never designed with AI-specific threat models in mind.

“The attack surface isn’t a misconfigured firewall or an unpatched library, rather it is the weaponization of the AI’s own reasoning and retrieval behavior. These platforms trust the content they ingest far too implicitly,” Levi said. 

The research is another example of how attackers can weaponize AI in a manner that current defenses cannot keep up with, making it extremely difficult for defenders to keep pace. 

“Offensive researchers and, increasingly, sophisticated threat actors are well ahead of most enterprise defenders on this,” Levi said. “The frameworks, detection signatures, and incident response playbooks for AI-native attacks simply don’t exist at scale yet. What gives us some optimism is that awareness is growing quickly, but awareness and readiness are very different things.”

Grafana Labs was notified through responsible disclosure protocols, worked with Noma to validate the findings, and issued a fix.

However, Joe McManus, CISO at Grafana Labs, told CyberScoop the company disputes “the claim that this finding constitutes either a ‘zero-click’ attack or that it could operate silently, autonomously, or in the background.”

“Any successful execution of this exploit would have required significant user interaction: specifically, the end user would have to repeatedly instruct our AI assistant to follow malicious instructions contained in logs, even after the AI assistant made the user aware of the malicious instructions,” McManus told CyberScoop via email. “We emphasize that there is no evidence of this bug having been exploited in the wild, and no data was leaked from Grafana Cloud.”

Update: April 7, 12:43 p.m.: This story has been updated with comment from Grafana.

The post ‘GrafanaGhost’ bypasses Grafana’s AI defenses without leaving a trace appeared first on CyberScoop.

In Part 2, we’re diving headfirst into one of the most critical attack surfaces in the LLM ecosystem - Prompt Injection: The AI version of talking your way past the bouncer.

The post Getting Started with AI Hacking Part 2: Prompt Injection appeared first on Black Hills Information Security, Inc..

Tim Fowler // Do you know what could be lurking in your Windows event logs? In May of 2022, I was sent a Threat Post article about a new technique […]

The post Windows Event Logs for Red Teams appeared first on Black Hills Information Security, Inc..

Marcello Salvati// During Red Team and penetration tests, it’s always important and valuable to test assumptions. One major assumption I hear from Pentesters, Red teamers and clients alike is that […]

The post Having Fun with ActiveX Controls in Microsoft Word appeared first on Black Hills Information Security, Inc..

Beau Bullock & Michael Felch // Source: https://chrome.google.com/webstore/detail/google-calendar-by-google/gmbgaklkmjakoegficnlkhebmhkjfich Overview Google Calendar is one of the many features provided to those who sign up for a Google account along with other popular […]

The post Google Calendar Event Injection with MailSniper appeared first on Black Hills Information Security, Inc..

Carrie Roberts // OS Command Injection is fun. I recently found this vulnerability on a web application I was testing (thanks to Burp Suite scanner). I was excited because I […]

The post OS Command Injection; The Pain, The Gain appeared first on Black Hills Information Security, Inc..