Internal knowledge often sits across manuals, shared folders, old notes and colleagues’ experience. The question is not merely how to turn it into an AI answer. You need to establish what is valid, who may receive it and what assistance helps somebody complete a concrete task. That takes substantive agreements as well as an appropriate technical design.
This collection provides independent guides for important decisions around RAG and company knowledge. Start with the choice of approach, investigate a specific failure or prepare maintenance arrangements. Worked situations are illustrative; they do not claim delivered customer outcomes or universally dependable automatic answers.
Start with the information problem behind the question
A team spending too long searching may have several causes for the same visible difficulty. An instruction is undocumented, terminology differs or a colleague cannot open the necessary source. A new answer interface cannot resolve those situations in the same way. Begin with a recognisable work example that the responsible owner and developer can examine together before choosing a technical approach.
Ask what happens after information is found. Does the employee need to select a document, understand an exception or prepare a reply? That distinction determines useful assistance. A source passage with a direct reference may be sufficient, while a question spanning several instructions needs more context. The intended next action provides the first practical boundary around the assignment and the outcome worth evaluating.
Choose an entry point for your present situation
If no technique has been selected, begin with the comparison between search, a managed knowledge base and RAG. If sources already exist, investigate whether structure, validity and ownership are adequate. For a trial producing incorrect answers, first establish whether the right passages are retrieved before focusing exclusively on rewriting response instructions or changing the language model.
The guides address separate decisions of that kind. They need not all be read sequentially. Choose one question, work through a concrete example and use the matching preparation tool to record assumptions and unresolved points. This turns a general conversation about AI into information useful for investigation, a scoped proposal or a targeted correction to an existing application.
Make sources recognisable to people and systems
A folder does not establish which information is valid. For selected material, record the content owner, intended audience and version relevant to the task. Include sources deliberately excluded and why. This keeps the collection explainable when somebody later asks why a familiar historical note is absent from its results despite remaining available elsewhere in the organisation.
Check the handling of different file types. Tables, attachments and scanned passages may need particular attention because extracted text can differ from the visual source. The original information remains the reference. Successful technical processing is only part of the check; the content still needs to preserve the conditions and meaning on which a user will base the next step in their work.
Design retrieval around real question types
RAG combines retrieving information with preparing an answer from context. An exact product code and an everyday problem description are not necessarily equivalent retrieval tasks. Test both. Also investigate variants with different meanings but similar wording. A relevant resemblance between texts is not the same as evidence that a particular rule applies to the user’s situation.
The retrieval guide breaks a failure down into expected and returned passages. That gives developers a clearer basis for examining filters, document splitting or search design. Keep questions for which no source exists as well. Such gaps should remain recognisable instead of being filled with plausible general information that may not describe the company’s actual product version or agreed procedure.
Microsoft Learn: Retrieval-augmented generation in Azure AI Search ↗
Preserve permissions throughout the information chain
A connector able to read a broad collection and a user entitled to receive a narrow subset have different roles. Keep those permissions separate. Define users, source areas and organisations so the application selects appropriate information before generating an answer. The same boundary needs to remain meaningful in citations, source opening and reuse of previously prepared responses.
Permissions also change during normal operation. A colleague moves projects or a document is temporarily withdrawn. Prepare a trial exercising that transition. A working account during a demonstration provides little evidence about revocation or missing identity context. The access-control guide explores those cases without suggesting a single standard setting necessarily covers every source, connector and platform combination.
Check the relationship between claim and evidence
A citation makes an answer open to investigation, not automatically correct. Ask which statement the passage supports, which conditions it contains and whether the source version fits. Material discussing the same subject may still be insufficient evidence for the conclusion. This helps reviewers separate convincing language from support that actually applies to the decision a user is preparing.
A fictional product appendix might apply only to one series. If the answer removes that condition, a valid link does not support the resulting general claim. The citation guide uses such cases to build a practical review method. Users should recognise what is established, which part remains unsupported and what clarification or human judgement is still necessary before they can rely on the information.
Keep the trial small enough to explain failures
Select one audience, a managed source area and representative questions. Include normal cases, exceptions, missing information and refused access. A limited collection can still reveal important distinctions. The objective is not an impressive test count, but an investigation whose outcomes lead understandably to a decision about improvement, narrower scope or the next stage of work.
Assess retrieval, substantive support and user experience separately. A fast response requiring substantial correction may be less useful than a clear reference. An appropriate refusal may be the desired result. Preserve reasons behind judgements so later changes are not assessed only through a shifted overall score without understanding which kind of task became easier, harder or less dependable.
Discuss costs as a set of responsibilities
Budget depends on more than requests to a language model. Source preparation, access boundaries, indexing, review and updates belong in the assignment. A demonstration proposal may therefore purchase something different from a proposal including maintenance and handover. Clarify the first release and the work retained by your team before comparing apparently similar totals from several providers.
The cost guide helps model normal and busier scenarios without presenting fictional rates as market evidence. It makes unresolved assumptions and recurring work visible. Assess time over the full process, including source checking and recovery. Released capacity and an actual reduction in spending remain different business outcomes that should be discussed separately instead of being treated as interchangeable measures of value.
Keep changes visible after launch
A source area grows, a product receives a new version or a procedure changes. Agree how revisions are nominated, processed and reviewed. Investigate missed and partial processing too. An application can remain technically available while some answers still reflect older information. Availability monitoring and source freshness therefore answer different operating questions and need responsibilities that reflect their separate meanings.
The maintenance guide distinguishes renewal, replacement and withdrawal. The preparation tool records ownership, source state and evaluation questions. Useful handover includes an exercise where an owner performs a change and reviews its result. The knowledge application then remains connected to everyday work, with recognisable boundaries and an executable route when relevant information is not yet correctly available through the system.
Explore each decision
RAG, search or a knowledge base: what does your team need?
Compare document search, a managed knowledge base and RAG against tasks, source quality, review effort and everyday use.
Read the guide ↗02 / RAG & business knowledgePrepare company documents for RAG: build a managed source collection
Make documents useful for a RAG trial through ownership, validity, structure, removal rules and representative checks.
Read the guide ↗03 / RAG & business knowledgeRAG access control: who may retrieve which company information?
Define users, organisations, source permissions, caches and revocation before making internal knowledge available through AI.
Read the guide ↗04 / RAG & business knowledgeRAG retrieves the wrong passage: investigate retrieval before rewriting the answer
Diagnose retrieval quality with expected passages, product versions, document structure, filters and controlled comparisons.
Read the guide ↗05 / RAG & business knowledgeCitations in RAG: does the passage actually support the claim?
Design reviewable AI answers using appropriate passages, versions, conditions and useful next steps when evidence is missing.
Read the guide ↗06 / RAG & business knowledgeTest a RAG system: assess sources, answers and refusals separately
Build a useful RAG evaluation with representative questions, expected passages, permission cases, clear criteria and repeatable comparisons.
Read the guide ↗07 / RAG & business knowledgeWhat does a RAG system cost? Budget for sources, retrieval, evaluation and updates
Examine RAG costs through source preparation, processing, search infrastructure, evaluation, permissions and everyday content maintenance.
Read the guide ↗08 / RAG & business knowledgeKeep RAG current: source changes, withdrawal and everyday knowledge maintenance
Organise source freshness through change detection, processing checks, ownership, withdrawal and targeted reassessment.
Read the guide ↗Turn the explanation into a project decision
Record your assumptions, unknowns and acceptance checks in the matching tool. Use the service guides to discuss delivery and scope.
Open the preparation tool ↗Questions about this topic
Where should we start without a source inventory?
Choose one question type and a small source area a responsible owner can review. Record validity, ownership and access, then try representative questions. Use the results to decide whether source organisation, search improvement or a RAG trial should come first.
Can this collection help investigate an existing AI trial?
Yes. Preserve a concrete failure and compare the retrieved passage with the expected source. Then examine the claim and its conditions. The separate guides help distinguish retrieval, answer behaviour, permissions and source freshness instead of treating every issue as one general AI problem.
What makes an initial assignment well bounded?
A recognisable audience, concrete task, reviewable sources, evaluation questions and agreements about access, updates and handover. Also state what the first release excludes and what investigation should deliver, including the possibility of choosing a simpler approach.
Sources and technical background
- Microsoft Learn: Retrieval-augmented generation in Azure AI Search ↗
Architecture overview; the examples and project choices here are original design guidance.
- OWASP: Authorization Cheat Sheet ↗
Authorisation principles; the worked evaluation cases are illustrative.
- NIST: AI Risk Management Framework ↗
Framework for organising AI risk management, not product certification.