Start with source quality, permissions and freshness.
AI knowledge base →OXIAE / INTEGRATIONS & DATA
RAG system development.
Build a RAG system that adds relevant business information to AI answers.
Discuss your projectManage knowledge, chat with documents or build RAG?
Staff need to ask questions about a bounded document collection.
Chat with business documents →You need custom source selection, integration and evaluation.
RAG system development (this service)UNDERSTAND THE SERVICE
What is a RAG system?
RAG stands for retrieval-augmented generation. It retrieves relevant information from selected sources and supplies it to a language model to formulate an answer. Quality depends on both finding the right information and using it correctly. Oxiae tests those stages separately.
When it is a good fit
RAG suits questions about your documents or frequently changing information when answers must be traceable to sources.
When to take a different approach
RAG cannot repair missing knowledge or solve every calculation or reporting task. Exact figures may be better served by a controlled database query or conventional report.
PRACTICAL DECISION GUIDE
What should a RAG system deliver?
A RAG system includes ingestion, text processing, indexing, passage retrieval and answer generation. We make the implementation and measurement of each part explicit. An incorrect answer can arise from a missing source, a missed passage or a mistaken interpretation; each requires a different correction. [1]
Delivery therefore includes a source register, update path, retrieval configuration and an assessed question set. Access filters apply before context reaches the model. Relevant sources still require tests for manipulated document instructions. RAG alone does not guarantee correct or secure answers. [2][3]
Compare knowledge search, document chat and access requirements in the AI search and business knowledge overview. Use that comparison to decide which part of the information process needs work first.
Questions to settle first
- Where do errors arise: sources, search results or answer generation?
- Which source and user permissions must retrieval enforce?
Make acceptance measurable
- Separate assessment of retrieved passages and supported answers.
- A delivery matrix covering ingestion, updates, permissions, evaluation and operations.
FICTIONAL DIAGNOSTIC PROTOCOL — NO RAG EVALUATION WAS CARRIED OUT
A wrong RAG answer: which stage should you investigate?
This fictional protocol uses procedure questions to distinguish different failures. Before evaluation, record the requesting user, authoritative source version, required passage and an acceptable answer or abstention for each question. The situations below are invented examples of possible findings. No search results, answer rates or system performance were measured.
On a narrow screen, scroll the table horizontally to read every column.
| Fictional test situation | What to check separately | Possible diagnosis | Targeted next step |
|---|---|---|---|
| The question concerns the return period for custom products; the approved source contains no rule for it. | Is there actually a supporting passage in the permitted, current sources? Have the source owner assess this before changing search settings. | Missing business knowledge. Fluent wording or a similar document does not supply the missing rule. | Have the application state what is missing and refer the question to the authorised source owner. Add a rule only after it has been established and published. |
| A current procedure contains the correct exception, but that passage is absent from the retrieved results. | Check ingestion, readable text, passage identifier, metadata and access filter. Then examine the query, search method and selection of passages offered to the model. | A possible processing or retrieval problem. The answer generator cannot correctly use a passage it never receives. | Repair the affected stage and rerun the same question. Check other questions too, so improving this exception does not displace relevant passages elsewhere. |
| The correct passage is retrieved, but the answer omits its approval condition. | Keep the context actually provided and compare each relevant answer claim with the intended passage and its conditions. | The source was found; wording or use of context fails. A valid source link does not establish that the content is correct. | Investigate answer instructions and output checks. Reassess the original case and similar conditions rather than changing only search settings. |
| A user without HR permissions asks about a restricted personnel policy. | Before answer generation, check that restricted passages are absent from results and model context. Also assess existing session context and source links. | An access failure if the content is offered anyway. A refusal in the final text does not undo earlier disclosure to the model. | Block the affected path and repair identity and permission enforcement. Repeat with permitted and excluded roles before releasing access. |
| Two permitted documents specify different periods; metadata does not establish which is authoritative. | Can the source owner establish validity and precedence? Check whether the system finds both relevant passages and makes the conflict visible. | An unresolved source conflict. The model must not decide policy based on writing style or an arbitrary search order. | Show the conflict and request a content decision. Then update versions, metadata and the evaluation question; reassess against the established source. |
| A test document is changed or deleted, but its old text still appears in a new answer. | Check the confirmed source change, synchronisation, index, caches and any reused context. Examine the source version attached to the passage actually retrieved. | A possible update or context problem. Uploading a new file alone does not establish that old content was withdrawn everywhere. | Repair the processing path and check a new question and an existing session under the agreed update and retention periods. Do not allow outdated content to pass as confirmed current information. |
What this helps you decide
Assess source availability and access first, then passage retrieval, then answer support. Record findings per question with source version, passage identifier, supplied context and relevant configuration, within agreed data and retention constraints. This gives each failure a targeted owner and correction. A small example set provides useful diagnoses, but does not establish general accuracy or safety.
Apply this to your own project
- Does every evaluation question have an authorised reviewer, permitted source selection and expected evidence or desired abstention defined in advance?
- Can you find the passages actually retrieved and supplied, separately from the sources cited in the final answer?
- Does a reviewer assess support for individual claims, including exceptions and conditions, rather than checking only that a source link works?
- Are unanswerable questions, conflicting documents and users without access included as separate cases?
- After a source change or deletion, are new questions and existing context checked within the agreed update and retention periods?
Compare your options
Choose based on the task, the people involved and what you can maintain.
| Approach | Useful when | What to consider |
|---|---|---|
| Document search | The goal is to find relevant sources. | Users combine and interpret the results. |
| RAG | The goal is an answer based on retrieved sources. | Retrieval and formulation can each introduce errors. |
| Model adaptation | The intended change concerns behaviour, style or task execution. | Not a substitute for current, verifiable document access. |
What I need to start
- Readable documents and version, owner and access metadata.
- Questions with known correct answers and sources.
- Rules for updates, deletion and access restrictions.
How to assess the result
- Measure whether relevant passages appear in retrieval results.
- Check that the cited source actually supports the answer.
- Test unanswerable questions, conflicting sources and access restrictions.
SCOPE & INVESTMENT
What determines the cost?
A useful estimate starts with the scope. I distinguish development work from recurring usage and operating costs. These are the main drivers:
- Document volume, formats and preprocessing.
- Search methods, permissions and refresh frequency.
- Indexing, model usage, hosting and maintaining evaluations.
APPLICATION AND SCOPE
Your challenge.
A practical application.
Illustrative scenario; not a published client case.
A language model does not adequately know your products, procedures or documentation.
RAG connects source retrieval with a language model, but does not replace source management. An AI knowledge base defines which information is available and current. The LLM integration then determines how passages are supplied to the model and how answers and uncertainty are assessed.
THE QUESTION I ASK FIRST
How will we measure whether the right source is found and used correctly?
Agree the first step
Use the prerequisites, acceptance checks and cost factors above to define your assignment. Agree who can approve the outcome and who will maintain the solution. The project approach explains discovery, delivery and handover.
Read the project approachQUESTIONS ABOUT THIS SERVICE
Your questions.
Clear answers.
Does RAG always produce correct answers?
No. It may retrieve the wrong passage or misinterpret a good source. I assess source selection and answer quality separately and design for uncertainty.
Do I always need a vector database?
No. The appropriate search method depends on your documents, questions and infrastructure. Keyword search, semantic search or a combination may fit.
How can you tell whether a RAG failure comes from search or the answer?
Inspect the context the model actually received. If the correct permitted passage is absent, investigate source ingestion and retrieval. If it is present but the answer loses a condition or invents information, investigate use of context and answer checks. Both problems can occur together.
Which questions belong in a RAG evaluation set?
Include representative task questions with reviewed supporting passages, plus unanswerable questions, exceptions, conflicting versions and access differences. Keep examples used to adjust settings separate from cases used to assess the subsequent change. Choose the scope and review process according to the tasks and consequences of error; no universal minimum establishes reliability.
Is a correct answer with a source reference enough for acceptance?
No. The passage must actually support the claims and conditions, the user must be permitted to see the content and the source version must be current. Also check missing information, changed permissions and deleted sources. One good answer does not establish that the entire chain works correctly.
Sources and technical basis
These primary sources explain the technical principles behind this guide. The project recommendations are OXIAE’s interpretation; they do not imply endorsement by the publishers. Product documentation applies to the named platform.
- Retrieval augmented generation in Azure AI SearchMicrosoft Learn
Technical basis for the components of RAG.
Link and relevance checked: - Security filter pattern in Azure AI SearchMicrosoft Learn
Platform example of user-scoped search filters.
Link and relevance checked: - LLM01:2025 Prompt InjectionOWASP GenAI Security Project
Why retrieval does not fully prevent instruction manipulation.
Link and relevance checked:
PREPARE YOUR PROJECT
Prepare a brief for this service.
Describe the scope below and download a draft for your project conversation. Unknown details remain open for discussion.
For your exploration of: RAG system development.
Make your request concrete
These optional fields turn the download into a brief for this service. Use general descriptions. Do not enter personal data, passwords, API keys or other secrets.
Name source types and approximate volume, such as manuals and policy documents.
Describe roles or departments; do not include names or real case data.
How often do they change and who handles replacements, deletion and versions?
Describe test questions, required citations and when the system should abstain.
Your answers stay in this browser and are not saved. Refreshing clears them. The download is created locally.