Prepare a document corpus for a RAG assistant
A document corpus is the set of documents an assistant may use. Usefulness depends on relevance, versions and access rather than just imported file count.
Start with questions to answer
Prepare authorised real questions or representative fictional cases. Identify documents that should support each answer and cases where the answer must remain unknown.
Fictional example: a support assistant retrieves a current procedure and an archived version. The corpus must distinguish their roles instead of mixing instructions.
Inventory versions and owners
Retain title, identifier, owner, version, validity date and original location. Distinguish file date from content applicability. Specify how corrections propagate.
A recent procedure may cite an older appendix that remains valid. Selection should not merely prefer whichever document has the latest date.
Define authorised documents
Determine who may read each document and whether system use is authorised. Separate public, internal and restricted material. The assistant must not bypass restrictions through answers or citations.
Prepare a case where a person must not see information. Use fictional documents initially without transferring sensitive data to an unapproved tool.
Evaluate retrieval and answers
For each case, check whether correct passages are retrieved and answers remain grounded in them. Record retrieval errors separately from answer errors. A displayed reference may not support a claim.
Retain passage, version and review conclusion. Test cases lacking suitable documents to ensure the system signals limits instead of inventing answers.
A record to keep with the decision
| Item | Evidence |
|---|---|
| Questions | Covered cases and expected or unknown answers. |
| Documents | Identifier, version, owner and validity. |
| Access | Authorised audience and tested restrictions. |
| Evaluation | Retrieved passage, answer and review evidence. |
Download the worksheet to fill in (CSV)
Frequently asked questions
Should every document be imported?
Select documents useful to permitted questions with context and versions. A larger corpus does not prove better answers.
Does a citation guarantee the answer?
No. Check that the passage actually supports the answer and applies to the case.
Reference material
The practical checklist is an editorial synthesis to adapt to your service. It does not constitute a certification or an audit result.