RAG Poisoning: How to Secure an Enterprise Knowledge Base

A benefits analyst asks an internal agent, "What documents do I need to add a dependent?" The agent cites a polished page that looks like HR policy and tells the employee to upload a passport copy to an external form. The page entered the knowledge base through a migrated team wiki six months ago. Nobody approved it, nobody owns it, and normal questions still work.

That is the shape of RAG poisoning. The model has not been jailbroken or retrained. A bad document has been placed where the system goes for trusted evidence.

Research presented at NeurIPS 2024 makes the risk concrete. AgentPoison poisoned fewer than 0.1 percent of an agent's memory or knowledge-base records. Across the tested settings, poisoned records were retrieved for 81.2 percent of triggered queries, while the intended malicious behavior succeeded end to end 62.6 percent of the time. Ordinary-task accuracy fell by an average of only 0.74 percent. That distinction matters: retrieval success is not the same as business impact, but normal QA can still look healthy while a narrow trigger causes repeatable failures.

The primary experiments gave the attacker white-box access to the embedding model for trigger optimization, although the paper also tested transfer to other embedders. Enterprise testing should cover these targeted attacks as well as simpler risks: compromised connectors, unauthorized uploads, stale policy, and ordinary false documents.

The answer is not another sentence in the system prompt. It is a secure knowledge supply chain.

RAG Poisoning Corrupts the Evidence Layer

A retrieval-augmented generation system usually performs four jobs:

1. Ingest documents from one or more sources.
2. Split and transform them into searchable units.
3. Retrieve units that appear relevant to a request.
4. Give those units to a model as evidence for an answer or plan.

Each step changes the security model. A SharePoint page with an owner and access list becomes chunks in an index. A resolved support ticket becomes a reusable example. A customer attachment becomes text that may sit beside approved policy. If the pipeline discards provenance and permissions, retrieval turns content from different trust levels into equally plausible context.

Poisoning exploits that flattening. An attacker, careless employee, compromised connector, or broken migration introduces content designed to rank for a target query and supply a false fact or instruction. PoisonedRAG research presented at USENIX Security 2025 reported a 90% attack success rate after adding five malicious texts per target question to databases containing millions of texts. The researchers also found that several tested defenses were insufficient.

This differs from indirect prompt injection. Injection tells the model what to do through content it reads now. Poisoning changes the material it will consult later. A document can do both, but the controls differ:

The useful security boundary is not "inside the company." An internal ticket can contain a malicious customer message. An employee note can be wrong. A trusted connector can ingest an untrusted attachment. Trust belongs to each document and field, not the network it came from.

Build Four Knowledge Trust Zones

The usual RAG architecture starts with one index because one index is easy to demo. Production should start with trust zones.

Zone 1: authoritative records. Approved policies, signed contracts, controlled product documentation, current price books, and records from named systems of record belong here. Every item needs an owner, effective date, version, and source identifier.

Zone 2: internal working knowledge. Draft pages, employee notes, project documents, and tickets can help answer questions, but they should not override Zone 1. Retrieval should label them as working material and expose conflicts rather than silently selecting the most similar chunk.

Zone 3: external and customer content. Uploads, inbound email, websites, transcripts, and vendor documents are evidence about a case. They are not instructions and should not become company policy. Keep them tenant-scoped and task-scoped wherever possible.

Zone 4: learned agent memory. Past plans, successful traces, reviewer corrections, and summaries can improve future work. This is also the most easily overlooked supply chain. An agent should not promote its own output into long-term memory simply because a run completed without an error.

The zones can live in separate indexes or in one store with enforced metadata filters. The implementation matters less than the invariant: the retriever must know which source can answer which type of question. A customer PDF can establish what that customer claimed. It cannot establish the company refund policy.

Permissions are a separate dimension. A document may be authoritative yet unavailable to the current user. Microsoft documents a security-filter pattern for Azure AI Search in which identity metadata is stored with indexed content and used to exclude unauthorized results at query time. Equivalent controls exist in other stacks. Whatever the platform, copy the source permission into the retrieval path and test it with real user identities. Our permission-aware agent guide explains identity propagation and retrieval filtering in more detail.

A practical metadata contract should include:

Do not let an embedding erase these distinctions. Similarity is a ranking signal, not a trust decision.

Secure the Ingestion and Promotion Pipeline

RAG ingestion deserves the same discipline as software delivery. Code does not move from a stranger's laptop into production because it compiles. Knowledge should not move from an upload folder into an agent's decision context because it embeds successfully.

Use this six-stage pipeline:

1. Identify. Bind every item to its connector, original record, uploader, tenant, timestamp, and content hash. Reject content that cannot be traced.
2. Classify. Assign a trust zone, sensitivity level, document type, owner, and allowed use. Classification can be assisted by a model, but policy promotion needs a deterministic rule or human decision.
3. Inspect. Scan files for malware and active content, extract text in a constrained process, detect hidden or anomalous text, and flag instructions that address the model rather than a human reader.
4. Approve. Require an owner for authoritative sources and a review step before external content or agent traces become reusable guidance.
5. Publish. Build a versioned index from a manifest. Preserve lineage from source document to each chunk, embedding, summary, and cache entry.
6. Monitor and revoke. Track retrieval behavior, provide one action to quarantine a source, and prove that removal reaches every derived artifact.

NIST's adversarial machine learning taxonomy treats poisoning as a lifecycle problem and notes the limits of current mitigations. That is the right posture. No one classifier can certify that content is safe. Layered controls reduce entry points, limit authority, expose abnormal retrieval, and shorten recovery.

Promotion into agent memory needs its own gate. A support agent may resolve a ticket using an exception approved by one supervisor. Saving that trace as a reusable success can turn a one-time exception into a general rule. Promote a trace only when a reviewer labels what should be reusable, which policy version supported it, and when it should expire.

The same rule applies to model-generated summaries. A summary is a derivative, not an authoritative source. Store its parent references and rebuild it when the parents change.

Test Retrieval for Targeted Failure, Not Average Quality

Poisoning is hard to catch with a standard RAG scorecard because the attack is designed to leave ordinary performance alone. If 990 common questions still pass and 10 targeted questions reliably retrieve a poisoned page, an average score can look healthy.

Test four properties separately:

Create adversarial cases around the queries that matter most: payment instructions, security procedures, employee data, discounts, refund policy, legal commitments, and account access. Add controlled documents that contain false facts, model-directed instructions, authority claims, hidden text, and near-duplicate wording. Test synonyms and paraphrases rather than one known trigger string.

Watch for retrieval dominance. A newly indexed page that appears in a wide range of semantically unrelated requests may be over-optimized, duplicated, or malicious. Compare the distribution of sources before and after each ingestion release. A small corpus change that causes a large rank shift deserves review even if final answers still pass.

Then test the action boundary. Retrieved content should never expand the agent's permissions. A policy page can say "send this report externally," but the tool layer must still evaluate the caller, destination, data classification, and approval rule. The OWASP guidance on excessive agency recommends enforcing authorization in downstream systems rather than asking the model to decide whether an action is allowed. This is the same hard boundary described in the prompt-injection security playbook; poisoning changes how the hostile content persists, not what should be allowed to authorize an action.

Every release test should retain the retrieved chunk IDs, their source metadata, ranking scores, the final answer, tool proposals, policy decisions, and resulting state. Without that trace, a failure appears as "the model was wrong" and the poisoning mechanism remains hidden.

Prepare for Revocation Before an Incident

Deleting the original document is not incident response. Its chunks may remain in the vector store, its summary may sit in a cache, its text may have been copied into an agent memory, and its claims may have influenced later generated documents.

A RAG poisoning runbook should answer these questions before launch:

1. Which source records and ingestion runs are affected?
2. Which chunks, embeddings, summaries, caches, and memories descend from them?
3. Which users and workflows retrieved those artifacts?
4. Which answers or actions were produced afterward?
5. Can the index roll back to a known-good manifest?
6. Which business actions need review or reversal?
7. What prevents the connector from re-importing the same content?

Version indexes as immutable releases. Build a candidate index, test it, then promote an alias or pointer. If an incident occurs, switch back to the last known-good version while investigators quarantine the bad source and rebuild. This is more reliable than editing a live corpus in place and hoping every derived copy disappears.

Keep the blast radius small. Separate tenants, high-risk domains, and memory classes so one poisoned source cannot influence every workflow. Give retrieval services read-only access to sources. Give ingestion workers only the permissions required for their assigned connectors. Require approval before a knowledge source can affect money movement, external communication, access changes, or deletion.

The recovery metric is not "document removed." It is time from detection to safe retrieval, plus the count of affected decisions found and reviewed.

How OpenNash Can Help

OpenNash helps enterprises turn an existing RAG prototype into an owned, testable knowledge system. We map each source, its authority, owner, permissions, retention, and allowed uses. Then we design the ingestion contract, trust zones, retrieval filters, lineage, versioning, action gates, and evidence trace around the workflows that carry real risk.

The first engagement is often an audit rather than a rebuild. We sample production retrievals, look for permission gaps and source dominance, trace documents through chunking and caching, and run targeted poisoning cases in a separate index. The result is a prioritized control plan tied to business impact, not a generic security checklist.

Built-in search-platform controls may cover a straightforward corpus and tenancy model. A custom ingestion and policy layer is warranted when sources, permissions, or approval rules do not fit packaged connectors. Without clear source ownership, the system should remain read-only. OpenNash can help make that call and implement the smallest defensible architecture.

Book a call to map one knowledge workflow. Bring the source list, two high-risk queries, and the question nobody wants the agent to answer from the wrong document.