RAG source and evaluation planner
Map source versions, access boundaries, owners and evaluation questions. Download a practical source plan for your RAG project.
A RAG project starts with information that can actually support an answer. This planner describes one application, its sources and the checks needed before using its output. It collects the decisions for an investigation without requiring document uploads.
Choose a bounded audience and a recognisable type of question. A website visitor asking about a public product needs different sources and access from an employee searching a customer file. Prepare separate plans for those applications so ownership and exceptions remain visible.
Prepare your plan
Your answers stay in this page. Download the draft to keep it; closing or resetting the page clears your input.
What belongs in a source plan?
For each source type, record its content owner, authoritative version and permitted audience. A folder name or file format does not answer those questions. Describe withdrawal too, including what should happen to outdated passages already present in a search index. Use field names and document categories instead of confidential source content.
Distinguish finding a source from supporting an answer. Retrieval may miss the relevant passage; an answer may make unsupported claims even when a useful passage was retrieved. Assessing those stages separately helps identify which part needs investigation or improvement.
Prepare a small, varied evaluation set
Illustrative example: an internal assistant helps employees find a maintenance procedure. The content owner selects current procedures and records an expected supporting passage for each evaluation question. The set also includes a question with missing information, two conflicting versions and a document restricted to another department.
A useful trial looks beyond a well-written answer. Check which source is retrieved, which version is used, whether the user can open the source link and what happens when evidence is missing. Agree who assesses failures; an aggregate score alone does not explain why a particular question fails.
Turn the draft into maintenance arrangements
The download contains your answers and unresolved topics. Agree with the content owner which changes require another evaluation. Examples include new documents, changed permissions, retrieval settings or an answer model. A successful trial on one source collection does not automatically establish the quality of an expanded application.
Use the plan to select sources for the first release and explicitly defer others. Then agree access, updates, evaluation and handover. The tool does not build an index or assess real answers; it makes the investigation question and responsibilities concrete.
Understand the decisions behind your plan
RAG & business knowledge ↗RAG, search or a knowledge base: what does your team need?
Compare document search, a managed knowledge base and RAG against tasks, source quality, review effort and everyday use.
Read the guide ↗02 / RAG & business knowledgePrepare company documents for RAG: build a managed source collection
Make documents useful for a RAG trial through ownership, validity, structure, removal rules and representative checks.
Read the guide ↗03 / RAG & business knowledgeRAG access control: who may retrieve which company information?
Define users, organisations, source permissions, caches and revocation before making internal knowledge available through AI.
Read the guide ↗04 / RAG & business knowledgeRAG retrieves the wrong passage: investigate retrieval before rewriting the answer
Diagnose retrieval quality with expected passages, product versions, document structure, filters and controlled comparisons.
Read the guide ↗05 / RAG & business knowledgeCitations in RAG: does the passage actually support the claim?
Design reviewable AI answers using appropriate passages, versions, conditions and useful next steps when evidence is missing.
Read the guide ↗06 / RAG & business knowledgeTest a RAG system: assess sources, answers and refusals separately
Build a useful RAG evaluation with representative questions, expected passages, permission cases, clear criteria and repeatable comparisons.
Read the guide ↗07 / RAG & business knowledgeWhat does a RAG system cost? Budget for sources, retrieval, evaluation and updates
Examine RAG costs through source preparation, processing, search infrastructure, evaluation, permissions and everyday content maintenance.
Read the guide ↗08 / RAG & business knowledgeKeep RAG current: source changes, withdrawal and everyday knowledge maintenance
Organise source freshness through change detection, processing checks, ownership, withdrawal and targeted reassessment.
Read the guide ↗