Securing RAG pipelines: a practical checklist

Retrieval-augmented generation (RAG) is the most common way enterprises put large language models to work: index your documents, retrieve the relevant passages for each question, and let the model answer from them. It is also a new data system — and most of the security risk sits in the plumbing, not the model.

This checklist walks through a RAG pipeline stage by stage.

1. Ingestion: know what you are indexing

Every document you index becomes something the assistant can repeat. Before connecting a source:

• Classify first. Run discovery for personal, health, financial and confidential data. Decide whether it should be indexed at all, masked, or kept out.
• Carry permissions with the content. Store the access-control list (or the group memberships) alongside each chunk, so retrieval can enforce it later.
• Track lineage. Record source system, document ID and version for every chunk, so you can delete or re-index when the source changes.
• Treat documents as untrusted input. A PDF or web page can contain hidden instructions. Ingestion is where you can strip active content and flag suspicious text.

2. Retrieval: enforce access at query time

The most common RAG failure is not a hallucination — it is an assistant that answers from documents the user was never allowed to see.

• Filter by the caller's identity, not the service account's. Retrieval should run with the user's permissions, ideally as a filter in the vector or search query itself.
• Avoid a single shared index for content with different audiences unless permission filtering is enforced and tested.
• Test it. Build a small suite of "should not retrieve" cases — a junior employee asking about executive compensation, for example — and run it in CI.

3. Generation: assume the context is hostile

Retrieved text is untrusted. Prompt injection — instructions hidden in content the model reads — is the top risk in the OWASP Top 10 for LLM Applications.

• Separate instructions from data. Put retrieved passages in clearly delimited sections and tell the model they are reference material, not instructions.
• Limit what the model can do. If a RAG assistant can also send emails or call APIs, a successful injection becomes an action. Scope tools tightly and require approval for anything irreversible.
• Filter outputs for sensitive data patterns and for links or content that should never be returned.
• Require citations and check that each cited passage was actually retrieved for this request.

4. Logging and evaluation: keep evidence, not liabilities

• Trace every run — question, retrieved chunk IDs, model version, answer — so you can investigate a bad answer.
• Protect the logs. Prompts and answers often contain the very data you were protecting. Apply the same retention and access rules as the source.
• Red-team before launch and after changes. Maintain a set of adversarial prompts and injection payloads and run them whenever prompts, models or sources change.

A minimum bar before go-live

1. Sensitive sources classified and either excluded, masked or permission-tagged.
2. Retrieval filtered by user identity, with automated negative tests.
3. Tools scoped to least privilege; irreversible actions behind human approval.
4. Output filtering and citation checks in place.
5. Traces retained under your data policy.
6. A red-team pass against the OWASP LLM Top 10.

None of this is exotic. It is ordinary data security, applied to a system that reads and writes natural language. If you want a second pair of eyes before launch, our AI security service covers exactly this.