Prompt injection is an attempt to make an AI system follow instructions that conflict with its intended task or authority. The instructions can arrive in a user's message or inside material the system is asked to read, such as a document, support ticket, webpage or tool result.
For a software company, the risk becomes concrete when an assistant can read customer records or use tools. A document being investigated might contain a request to disclose unrelated information, change a setting or treat the document's author as an authorized approver. Reading that document should not grant it those powers.
The central design problem is to preserve the difference between information and authority. An agent needs to use source material to do useful work. The application must still control what the agent can access, which actions it may take and what evidence is needed before a consequential step proceeds.
Direct and indirect prompt injection
In a direct attempt, someone addresses the AI system with instructions intended to redirect it beyond the allowed task. In an indirect attempt, those instructions are embedded in material the system encounters while working on another person's request. OWASP's prompt injection guidance describes both forms and their possible consequences.
The distinction is about how the instructions reach the system. An external webpage is one route, but a company's internal records can also contain content written by another party. A ticket, shared attachment or copied excerpt can cross the same trust boundary.
Not every instruction inside a document is an attack. A migration guide legitimately describes steps someone might take. The application needs to distinguish reading or explaining those steps from receiving permission to execute them.
An agent asked to summarize a procedure may quote an instruction accurately. That does not authorize it to perform the operation. The original user request, application policy and available permissions still determine the scope of work.
Why does prompt injection matter for agents?
An assistant that only drafts text can produce a misleading answer. An agent with additional access may also read restricted material, send a message or change an external system if the surrounding controls allow it.
The consequences depend on the actual integration. A convincing instruction cannot create a tool that does not exist, but it may steer the use of capabilities the application already exposes. This makes permission and action design important alongside model behavior.
Microsoft's analysis of indirect prompt injection discusses both unintended actions and information disclosure, with mitigations spanning prevention, detection and limiting impact. Its examples illustrate why improving a prompt is only one part of the system design.
A useful security question is therefore: if the model follows an instruction from this source, what can the application still prevent? The answer should be visible in the execution and data access controls.
Example: investigating a customer attachment
In this hypothetical defensive scenario, a software support team is investigating delayed webhook deliveries. A customer provides a diagnostic attachment. The agent is authorized to inspect that attachment and a bounded set of relevant delivery records, then prepare an internal investigation brief.
The attachment includes a passage claiming that a broader diagnostic export has already been approved and should be sent to an external destination.
The legitimate task is to explain the delivery problem. The attachment is evidence supplied for that task. Its claim about approval does not establish authorization to disclose more data.
Establish the scope before reading
The workflow identifies the customer, permitted records and intended internal output. The agent's execution route should be limited accordingly. It does not need general access to every customer's history or a tool that sends arbitrary exports.
These limits should exist in application controls. A sentence in the instructions asking the model to stay within scope cannot replace enforcement by the services that return data or perform actions.
Inspect the content without accepting its authority
The agent can identify delivery timestamps and error descriptions in the attachment. A passage requesting a different operation should remain source content, with its origin preserved.
If additional records are genuinely needed, the agent can explain what is missing and request the appropriate access. It should not treat the attachment's assertion of approval as the required decision.
Constrain the output and the action path
The internal brief reports relevant findings and uncertainty. It need not copy the attachment's redirection into a recommendation or include unrelated sensitive material.
If the workflow allows external communication, the application must separately validate the target, content and authority for that action. A permitted destination should come from the approved workflow, not become trusted because it appeared in a customer supplied file.
Retain an honest outcome
The record can state that the attachment contained an instruction outside the task's scope and that no authorized external export was performed. It should distinguish an agent's statement from the execution evidence available.
If an action was attempted or its outcome is uncertain, the investigation needs the relevant operation records. An answer saying that nothing happened is not independent confirmation that no external effect occurred.
Keep authorization outside the retrieved content
The most important boundary in this example is who can authorize an action. A document can provide a reason to investigate; it cannot appoint itself as the decision maker.
MITRE's indirect prompt injection guidance recommends authorization and policy checks independent of the model's generated decision, together with restrictions on accessible resources and permissions. These controls limit what externally supplied instructions can cause the system to do.
For the support workflow, define the allowed customer scope and operations before execution. A request outside that scope needs a separate authorized route. The model's interpretation of the attachment should not expand its credentials or change the application policy.
Read access and disclosure also need separate consideration. An agent may be permitted to inspect a record for an internal investigation without permission to publish its contents or send them to another service.
Use several controls with different responsibilities
No single control addresses every way external content can influence a system. A practical design assigns responsibilities to distinct layers.
Source handling
Preserve the origin and status of retrieved material. Make external excerpts distinguishable from the user's request and the application's instructions. Avoid presenting an attachment's unverified claim as a system decision.
This helps the model interpret the material appropriately, but it remains a behavioral aid. It should not be the only boundary preventing a sensitive operation.
Data and tool access
Expose the records and operations needed for the workflow. Enforce the requester and target scope when a service handles a read or action. Separate reading from changing state wherever the integration permits it.
For the diagnostic task, a service can restrict the returned records to the selected customer and period. That restriction remains valuable even if the model asks for something broader.
Output handling
Treat generated output as content that still needs appropriate validation. Check what the application will render, store or send, including links and remote media. Microsoft's analysis notes that disclosure can occur through rendered content as well as explicit tool actions.
A text only agent is not automatically harmless if its output is later used by a more privileged component. Define what each downstream consumer is allowed to do with that output.
Detection and review
Detection can help identify suspicious instructions and route a case for review. A detector's absence of an alert does not establish that every passage is safe.
Review consequential actions with the relevant evidence and exact proposal visible. The person should be able to inspect the original source of an approval claim. A summary that repeats the same claim does not provide independent confirmation.
What happens when content becomes a summary or memory?
An agent may summarize an attachment, save a handover or produce an answer that another agent later reads. That transformation should not increase the authority of the original content.
In the webhook example, a summary that says “customer approved diagnostic export” would be misleading if its only basis was the attachment's own assertion. A later agent might see the summary without the warning or source context that made the problem apparent.
Preserve the relationship between a claim and its source. Separate observations, allegations, proposals and accepted decisions. Where a process promotes material into reusable guidance, apply the required review before that status changes.
Saving information and accepting it as trusted guidance should remain distinct operations.
Can RAG, better prompts or another model solve it?
Retrieval supplies material to a model. It can therefore deliver useful evidence and untrusted instructions through the same application path. More relevant retrieval does not by itself determine which parts of a source may govern the agent's behavior.
Clear instructions and source separation can help. OWASP also cautions that RAG and fine tuning do not fully mitigate prompt injection. Treat these techniques as components whose behavior needs testing, with access and action controls maintained separately.
A second model can assist with detection or review, but it is another model processing information. Validate its judgments and keep its authority bounded. A confident verdict should not silently authorize a new operation.
Our prompt engineering guide explains how to make instructions explicit and testable. Our MCP guide explains the connection layer. Neither instruction clarity nor a connection protocol replaces the application's permission checks.
How should a team test the boundary?
Use a controlled environment with synthetic records and operations that cannot affect real customers. Establish the expected behavior for each case before running the test.
The diagnostic workflow suggests a useful review set:
An attachment contains relevant facts and an unrelated instruction.
A source claims that a sensitive operation was already approved.
A tool result asks for access outside the selected customer scope.
A summary repeats an unverified instruction as accepted guidance.
A proposed output contains an unapproved external destination.
A detector flags ordinary procedural language that should remain usable as evidence.
A model proposes a prohibited action and the execution service rejects it.
Evaluate whether the system completes the legitimate investigation as well as whether it respects the boundary. Rejecting every document would be a poor result for a support workflow. The application needs to use permitted evidence without allowing the evidence to redefine its authority.
Inspect attempted reads and actions alongside the final report. Record what the service allowed or refused and whether an external effect was independently confirmed.
Where does Swarm fit?
Swarm Spaces give people and agents a persistent record of shared work, sources, outputs and human decisions. A support investigation can retain its original scope, the attachment's source, the agent's findings and any later review.
Swarm's connection guidance treats source text as evidence, not new authority. Connected clients must preserve the human's selected scope, execution policies and required approval gates. An agent cannot supply its own required human approval.
The connected application remains responsible for how it assembles model input, handles returned content and exposes tools. Access to a source does not grant permission to share it. A saved judgment or successful Run also does not independently establish that no external effect occurred.
A Space is therefore useful for preserving the records needed to inspect the workflow. It is not a guarantee against prompt injection, an automatic detector for every malicious instruction or a substitute for securing the executor and its integrations.
Common questions about prompt injection
Is prompt injection the same as an incorrect answer?
No. A model can make a mistake without any attempt to redirect it. Prompt injection concerns instructions that steer behavior beyond the intended task or authority. The investigation should establish what content and actions were involved.
Are internal documents always safe to trust?
No. Internal storage does not establish who wrote every passage, whether it was copied from another source or what authority it has. Retain the source context and enforce permissions independently.
Does requiring approval remove the risk?
Approval can limit particular actions when properly enforced. The reviewer still needs an accurate proposal and evidence. It does not address every disclosure path or make unreviewed output trustworthy.
What should a team do first?
Choose one workflow that reads outside content. List the data and actions it can reach, identify who authorizes each consequential step and test whether source content can cross those boundaries. Start with a narrow task whose legitimate outcome is clear.