As security operations teams now use large language models (LLMs) and autonomous AI agents into their daily work, a new frontier is emerging: attackers deliberately manipulating AI agents. Prompt injection attacks—where an attacker hides malicious instructions that cause an AI agent to ignore its safety rules—pose a serious risk to enterprises. These attacks continue to grow in size and scale.  

Snyk’s security audit of the Agent Skills ecosystem, which includes Anthropic’s Claude, Vercel, and others, that 36% of all skills contained at least one critical-level security issue, including malware distribution, prompt injection attacks, and exposed secrets.

In June, researchers at Mozilla tested a prompt injection attack on Claude using indirect prompt injection—a technique that embeds malicious instructions in external content the AI agent processes. In this proof-of-concept, attackers took over developers’ systems by hiding indirect prompts in normal-looking repositories. When Claude Code executed them, the agent spawned a reverse shell.

AI agents often connect to more sensitive data than human employees do., A successful prompt injection can lead to catastrophic data loss or unauthorized system actions. Defending against prompt injection attacks requires multiple layers of protection. Security teams must monitor agent behavior for anomalies and prepare for agent containment, forensic preservation, and system remediation. Because AI agents execute tasks at machine speed, human responses must be able to match that pace.

The architecture of trust: Protocols and no “black box”

AI-native workflows need governed access rather than “black-box” autonomy. Modern governance frameworks use standardized protocols like the Model Context Protocol (MCP) to provide secure communication between AI clients and data sources. Visibility and transparency in agentic AI workflows matter, especially in cybersecurity. Autonomous agents perform complex tool executions and use independent logic, so they must show how they reached their decisions to meet regulatory requirements. Agents without transparency post serious risks: obscured reasoning can trigger unpredictable tool interactions, bypass governance controls, and create uncontrolled defensive gaps.

Implementing these protocols matters:

• Bounded Tenant Awareness: In a stable agentic AI architecture, multi-tenancy scales well. But if an AI tenant misbehaves, the entire system can fail. Bounded tenant awareness isolates any misbehaving AI agent to prevent cross-tenant contamination or data leakage.
• Strict Access Controls: By controlling connections to the platform, organizations can stop “ignore previous instructions” style bypasses. Maintain tight control over what the AI can see and do within a workflow.
• Standardized Telemetry: All telemetry must remain consistent and audit-ready. Even if an AI interaction is attempts to break rules, the underlying data movement gets tracked against established frameworks like MITRE ATT&CK and NIST.

Detecting the aftermath: UEBA and NDR as safeguards

A robust, unified SecOps platform can detect anomalous behavior even after prompt injection tricks an AI agent. Prompt injections often serve to steal credentials theft or extract data. When detected it’s important to act quickly. In agentic AI systems, misbehavior can escalate privileges, manipulate memory layers, create unauthorized identities, or alter shared reasoning components. Containment must be automatic and enforced at identity, authentication, and authorization layers.

These safeguards include:

• User and Entity Behavioral Analytics (UEBA): Identity-focused correlation and behavioral baselines to identify anomalous user activity or privilege escalation. If a compromised AI agent acts outside of its normal operational parameters, UEBA flags it in real-time and alerts a human security analyst.
• Network Detection and Response (NDR): Combining network traffic analytics with endpoint and cloud telemetry, NDR can identify data exfiltration or policy violations from a successful prompt injection.
• Multi-Layer AI Filtering: AI filters reduce raw alerts into high-fidelity incidents, cutting noise by up to 90%. This keeps the signals of an AI-driven attack from disappearing in a busy SOC.

Humans remain the strongest defense against AI agent social engineering. The human security analyst is still the one who makes the final decision. While AI handles triage and correlation, humans retain final control over response actions.

Moving beyond reactive guardrails

The traditional SOC model was never designed to handle machine-speed, AI-driven attacks. A human-augmented autonomous SOC approach moves from reactive alert handling to a proactive, verdict-first model. By combining a transparent, governed AI access with robust UEBA and NDR, organizations keep the SOC secure, transparent, and resilient as social engineering methods target machines.

The post Why transparent AI agents matter more than you think appeared first on CyberScoop.

On Wednesday, Microsoft released two new red teaming tools — Rampart and Clarity — meant to help developers design more secure agentic software and assist incident responders in the face of ongoing breaches.

Rampart is built on top of PyRIT, an existing open automation framework Microsoft developed for red teaming generative AI systems. But while PyRIT scans already-built systems for security flaws, Rampart is made to continuously test code for vulnerabilities during the development process, encoding both adversarial and benign testing scenarios into the software development pipeline to flag exploitable bugs and dependencies.

Microsoft said Rampart was built to focus on cross-prompt injection attacks, where “an agent retrieves or processes potentially poisoned content from documents, emails, tickets, and other data sources that manipulate behavior indirectly.” It also confirms fixes or exploits work as intended through multiple rounds of testing, as opposed to tools that perform “single shot validation.”

The second tool, Clarity, can be run as a desktop app, a web interface or directly embedded into a coding agent to provide real time security engineering guidance to developers at the outset of a project. It can categorize and track different business objectives related to the code and highlight downstream security implications along with more secure by design alternatives.

Ram Shankar Siva Kumar, who founded Microsoft’s AI red team in 2019, told CyberScoop that the company has seen internal security benefits from using the tools, but believesRampart and Clarity’s growth depends on contributions from other developers outside the Microsoft ecosystem.

In the fast-moving world of AI, where vibe coding, rogue AI agents and a steady churn of new model releases create fresh security implications nearly every week, Siva Kumar said it was important to begin building foundational, AI-centric security processes into the software development pipeline.

“When you hear a lot of talk about AI safety and security, it seems to be a lot of philosophical debates,” he said. “You’ll see frameworks, you’ll see white papers, and I think we’re really past that time, now. We really need to start thinking of AI safety as an engineering discipline and trying to bring security where the developers are.”

Rampart’s potential utility to defenders goes beyond just securing software development pipelines. It can also be used during an active incident response to speed up or automate red teaming for hot fixes, patching and remediation.

Microsoft has used Rampart when investigating reported vulnerabilities in their own products. Siva Kumar said the tool was able to help condense a week’s worth of manual work —  replicating the vulnerability, identifying different variants of the same bug, then patching and re-testing those variants to ensure they’re no longer exploitable — into hours.

Clarity, meanwhile, acts as a security adviser for software projects, prompting developers to consider potential risks in their design decisions and their downstream security consequences. With the rise of AI-generated code and agents, and execution becoming cheaper, this kind of proactive guidance is increasingly important.

“You’re going to be able to create apps, create MCP servers to pull things out from the internet,” said Siva Kumar. “The question is, ‘should you be doing it?’ And Clarity is a step in that direction. It is asking, ‘hey, should you be doing this in the first place?’”

The post Meet Rampart and Clarity, Microsoft’s new red team combo AI agents appeared first on CyberScoop.

Cybersecurity agencies from the United States, Australia, Canada, New Zealand and the United Kingdom jointly published guidance Friday urging organizations to treat autonomous artificial intelligence systems as a core cybersecurity concern, warning that the technology is already being deployed in critical infrastructure and defense sectors with insufficient safeguards.

The guidance focuses on agentic AI — software built on large language models that can plan, make decisions and take actions autonomously. In order for this software to function it needs to connect to external tools, databases, memory stores and automated workflows, allowing it to execute multi-step tasks without human review at each stage.

The guidance was co-authored by the U.S. Cybersecurity and Infrastructure Security Agency, the National Security Agency, the Australian Signals Directorate’s Australian Cyber Security Centre, the Canadian Centre for Cyber Security, New Zealand’s National Cyber Security Centre and the United Kingdom’s National Cyber Security Centre.

The agencies’ central message is that agentic AI does not require an entirely new security discipline. Organizations should fold these systems into the cybersecurity frameworks and governance structures they already maintain, applying established principles such as zero trust, defense-in-depth and least-privilege access.

The document identifies five broad categories of risk. The first is privilege: When agents are granted too much access, a single compromise can cause far more damage than a typical software vulnerability. The second covers design and configuration flaws, where poor setup creates security gaps before a system even goes live.

The third category covers behavioral risks, or cases where an agent pursues a goal in ways its designers never intended or predicted. The fourth is structural risk, where interconnected networks of agents can trigger failures that spread across an organization’s systems.

The fifth category is accountability. Agentic systems make decisions through processes that are difficult to inspect and generate logs that are hard to parse, making it difficult to trace what went wrong and why. The agencies also note that when these systems fail, the consequences can be concrete: altered files, changed access controls and deleted audit trails.

The guidance also flags prompt injection, where instructions embedded inside data can hijack an agent’s behavior to perform malicious tasks. Prompt injection has been a lingering problem with large language models, with some companies admitting that the problem may never be solved. 

Identity management gets significant attention throughout the document. The agencies recommend that each agent carry a verified, cryptographically secured identity, use short-lived credentials and encrypt all communications with other agents and services. For high-impact actions, a human should have to sign off, and the guidance is explicit that deciding which actions require that approval is a job for system designers, not the agent.

The agencies admit the security field has not fully caught up with agentic AI. Some risks unique to these systems are not yet covered by existing frameworks, and the guidance calls for more research and collaboration as the technology takes on a growing number of operational roles.

“Until security practices, evaluation methods and standards mature, organisations should assume that agentic AI systems may behave unexpectedly and plan deployments accordingly, prioritising resilience, reversibility and risk containment over efficiency gains,” the guidance reads. 

You can read the full guidance below.

The post US government, allies publish guidance on how to safely deploy AI agents appeared first on CyberScoop.