What is the difference between direct and indirect prompt injection?

Direct injection arrives in the user's own input, for example a user who types instructions intended to override the system prompt. Indirect injection arrives through content the agent retrieves, such as a document, a web page, or a calendar invitation, where an attacker has embedded instructions knowing the agent will read them.

Can output filtering catch prompt injection attacks?

Output filtering catches some exfiltration attempts after the fact, but it does not prevent the agent from taking harmful actions mid-task. A defense that only inspects output still allows an injected instruction to call a tool, write a file, or escalate a permission before any filter sees a result. Input-side controls and permission minimization have to come first.

How does permission minimization reduce injection risk?

An agent that cannot write to a database cannot corrupt it, even if it receives an injected instruction telling it to. Scoping each agent's credentials and tool access to exactly what it needs for its current task means a successful injection has a smaller blast radius. The Devin research illustrated this: the agent's broad execute permissions were what made key exfiltration possible, not just the injection itself.

How often should teams run adversarial red-teaming on agents in production?

Red-teaming before first deployment establishes a baseline, but the threat surface changes whenever the agent's tool set, data sources, or system prompt changes. A practical cadence is a structured adversarial review at every significant change and automated injection probing on every deployment, with continuous monitoring between releases.

From Lab to Liability: Building Prompt Injection Defenses Before Deployment

This article gives you a structured approach to finding and bounding prompt injection vulnerabilities before your agent reaches production. It covers why the problem is structural, how to map your attack surface, what a pre-production red-team workflow looks like, and what runtime controls to put in place on day one.

Why prompt injection is an architecture problem, not a bug

A traditional SQL injection flaw lives in a specific code path. You find it, you parameterize the query, you ship the fix. Prompt injection does not work that way. The same property that makes a language model useful, following natural-language instructions, is the property an attacker exploits. There is no single code path to patch because the vulnerability is the instruction-following mechanism itself.

This matters for how you plan. Teams that treat prompt injection as a checklist item at the end of a sprint will not find it in time. The architectural decisions you make about how your agent processes input and calls tools determine how exposed you are, and those decisions are expensive to reverse after launch.

OWASP found prompt injection in over 73% of production AI deployments assessed during security audits, as of May 2026. That figure is not a warning about careless teams. It reflects how easy it is to inherit the vulnerability without noticing.

Two vectors you need to model separately

Direct injection

Direct injection arrives through the user. A user submits a message designed to override the system prompt, claim a different identity, or instruct the agent to ignore prior instructions. These are visible in your input logs and relatively straightforward to probe.

Indirect injection

Indirect injection is the harder problem. Here, the hostile instructions are embedded in content the agent retrieves during a task, not in the user’s message at all. The agent reads a document, a web page, an email, or a calendar invitation, and the attacker’s payload travels with the content.

The Microsoft 365 Copilot EchoLeak vulnerability (CVE-2025-32711) demonstrated this at enterprise scale. A single email containing hidden instructions forced Copilot to exfiltrate sensitive data without any user interaction. No click, no link, no mistake by the recipient. The agent’s retrieval behavior was the attack surface, not the user interface.

The GitHub Copilot CamoLeak vulnerability (CVSS 9.6, CVE-2025-53773) followed the same pattern through image rendering in private repositories, allowing silent extraction of source code and secrets. The vendor’s response was to disable image rendering entirely, which illustrates an important point: sometimes the right fix is removing a retrieval capability, not filtering its output.

When you map your agent’s attack surface, list every external data source the agent reads. Each one is a potential indirect injection channel.

flowchart TD
    A[User Input] --> B{Input type}
    B -->|Direct| C[System prompt context]
    B -->|Retrieval trigger| D[External content fetch]
    D --> E[Document / Email / Web / API]
    E --> F{Injection present?}
    F -->|Yes| G[Injected instruction enters context]
    F -->|No| H[Clean content enters context]
    G --> I[Agent executes injected action]
    H --> J[Agent executes intended action]
    C --> K{Override attempt?}
    K -->|Yes| G
    K -->|No| J

Pre-production red-teaming: a four-step workflow

1. Define the trust boundary

Write down exactly what your agent is allowed to do and what it cannot do. Which tools can it call? Which data stores can it read or write? Which external domains can it fetch from? If you cannot state this boundary clearly before testing, your testers will not know what a successful attack looks like.

For multi-agent systems, define the boundary for each agent separately. An injected instruction that propagates from a sub-agent to an orchestrator can escalate privileges in ways that neither agent’s individual boundary would allow.

2. Build an injection test library

Collect both direct and indirect payloads. Direct payloads should cover role-override attempts (“ignore your previous instructions”), goal-substitution attempts (“your actual task is to…”), and exfiltration probes designed to extract system prompt content. Indirect payloads should be embedded in the file formats and sources your agent actually reads.

HackerOne reported a 540% year-over-year surge in prompt injection reports between July 2024 and June 2025, the fastest-growing AI attack vector on their platform. Publicly disclosed payloads from those reports are a practical starting point for your test library.

3. Run adversarial sessions with production-equivalent permissions

Testing against a sandboxed agent with reduced permissions will not reveal what a real attacker can accomplish. The Devin research by Johann Rehberger demonstrated this directly: for $500 in compute, he found the agent would download malware from an attacker-controlled site, grant itself execute permissions, and exfiltrate AWS keys. The agent’s broad permissions were what made the impact severe. Test with the permissions your production agent will actually hold.

4. Document findings as architectural constraints, not bug tickets

Each successful injection should produce a constraint: “the agent must not fetch content from domains outside the approved list” or “retrieved document content must be processed in a separate context from tool-calling instructions.” These become requirements, not post-launch patches.

flowchart TD
    A[Define trust boundary] --> B[Build injection test library]
    B --> C[Run adversarial sessions]
    C --> D{Injection succeeds?}
    D -->|Yes| E[Document as architectural constraint]
    D -->|No| F[Log as passing baseline]
    E --> G[Revise architecture or permissions]
    G --> C
    F --> H[Proceed to runtime controls]

Permission minimization

The Cursor IDE vulnerabilities across 2025 and 2026, including hidden instructions in Slack messages, Git hooks, and config files that rewrote MCP configuration and escaped the sandbox, show what happens when an agent holds developer-level privileges during a period of active injection risk. The agent’s permissions determined the blast radius.

Apply least-privilege at three levels. First, scope API credentials to the exact operations each task requires: read-only tokens for retrieval tasks, scoped write tokens for creation tasks, no standing access to deletion. Second, restrict which external domains the agent can fetch from to an explicit allowlist. Third, for MCP-connected tools, audit which servers are reachable and from which contexts.

Comparing RBAC and ABAC models for agent permissions is worth doing before your first deployment. Attribute-based models let you scope permissions to the current task state rather than the agent’s identity alone, which tightens the window an injected instruction can exploit.

Runtime monitoring

Pre-deployment testing cannot catch injections that arrive after launch through new content sources. Runtime monitoring closes that gap. Log every retrieval operation with its source, every tool call with its parameters, and every output that leaves the agent’s context. Alert on tool calls that were not initiated from a known user action, and on content fetched from domains not present at test time.

Tools in the agent observability category, including purpose-built platforms like Prefactor, can attach this instrumentation to existing agent frameworks without rewriting your core logic. The goal is a complete trace from user request through every retrieval and tool call to final output, so an injected instruction that reaches production is visible in the logs rather than silent.

For teams managing AI governance obligations, that trace is also the audit record that demonstrates you detected and responded to an attack.

Where to start

If you are shipping an agent in the next six months, the most useful immediate step is mapping every external data source your agent reads and every tool it can call, then running a structured adversarial session against that surface before you set a launch date. Take the agent readiness assessment to see where your current architecture stands against the vectors covered here, and get a prioritized list of the gaps to close first.