When an employee cannot find an answer, an AI chatbot is not automatically the next step. A document may lack a clear title, the same instruction may exist in several locations, or nobody may know which version applies. RAG can retrieve relevant passages for a question and use them when generating an answer. Its usefulness still depends on the task, the sources and the way a person checks the result.
This guide helps you define the information problem before choosing a solution. The situations are illustrative. They are not measured customer outcomes or a claim that one standard product suits every organisation without investigation.
Describe the work behind the question
Ask employees to reconstruct a recent situation rather than simply list documents they find difficult to locate. What question arrived, what information was missing and what decision followed? A service colleague may need an exception for one product, while a new employee wants to know where a procedure starts. They may use the same folder but need different forms of support.
Define a useful result for each situation. It could be a relevant passage, a short explanation with a citation, or a list of facts still needed before an answer is possible. Generated prose helps only when it makes the next action clearer. For straightforward lookup, a direct link to an instruction may be faster and easier to verify than an additional conversational layer.
Investigate the existing search route
Give several colleagues the same representative question and observe their normal route. Record search terms, folders, filters and moments when they ask another person. This helps distinguish a content problem from a terminology or access problem. Improving retrieval will not solve a question whose underlying rule has never been documented by the organisation.
Try a small improvement before a larger experiment. For one topic, clarify its title, owner and applicable version, then repeat the task. If that is enough to make the answer findable, it provides a useful first step. It does not rule out later AI assistance; it prevents a new interface from reproducing the same uncertainty about the underlying information.
Separate three forms of assistance
Search helps people locate sources. A managed knowledge base organises approved information with ownership and revision arrangements. A RAG application combines retrieved material with answer generation. These can coexist. They are not a mandatory progression in which the most technically elaborate option automatically offers the best outcome for the team.
An exact part number may be a search task. A question requiring several instructions to be combined may benefit from a supported explanation. A missing policy decision still requires an accountable person to decide the rule. Give each problem a distinct outcome in the investigation instead of placing every requirement under an expansive promise to understand all company knowledge.
| Assistance | Useful outcome | Investigate first |
|---|---|---|
| Search | Locate the relevant source or passage | Titles, terminology, filters and valid versions |
| Managed knowledge base | Approved information with an owner | Missing rules and editorial maintenance |
| RAG | An answer with appropriate source support | Retrieval, support for claims and permissions |
Choosing a search or answer layer starts with the material you can actually use: Prepare company documents for RAG: build a managed source collection.
Microsoft Learn: Retrieval-augmented generation in Azure AI Search ↗
Prepare questions before the demonstration
Collect normal questions, ambiguous questions and questions the organisation cannot answer from the proposed sources. Include questions with a mistaken assumption. If guidance applies only to certain products, the application needs to preserve that condition. A convincing demonstration using only convenient preselected questions tells you little about everyday behaviour and the work users will still have to perform.
For each question, record the expected supporting source and any information that remains missing. Avoid grading an answer solely by matching the wording of a prepared response. A valid paraphrase may be useful, while a familiar sentence can omit a decisive condition. The collection becomes a testable agreement about work rather than a competition in producing polished language.
Choose an initial source area with an owner
An illustrative installation company wants help with maintenance questions. Its first collection contains approved instructions for one product family. General sales emails, historical proposals and personal notes remain outside the scope. A technical owner decides which versions apply and which exceptions should be referred to a person instead of answered by the application.
This boundary makes it possible to investigate whether answer assistance is useful. The pilot need not interpret every company document. If even the small collection contains contradictory instructions, resolving them takes priority. A modest, well-managed topic can yield better learning than a broad import whose contents have no clear owner or agreed meaning.
Compare equivalent tasks in different designs
Use the same questions with the current process, an improved search route and a limited RAG trial. Keep sources and review criteria comparable. Measure not just time to an initial answer but time spent opening citations, checking conditions and correcting mistakes. Otherwise a fast response may appear efficient while transferring additional work to the person responsible for checking it.
Also examine when employees still need specialist help. An appropriate refusal can be preferable to a complete but incorrect answer. Distinguish missing information, a passage that was not found and a passage interpreted incorrectly. Those failures require different improvements. An overall satisfaction score alone makes it hard to see which part of the proposed solution needs attention.
Keep access separate from interpretation
Content being technically readable does not mean every employee is entitled to receive it. Define which users and organisations participate in the comparison. Check whether the same question produces appropriately different results under different permissions. A generated summary must not become an alternative route around restrictions on the original document or a way to expose information through a citation.
Consider what follows the answer as well. An explanation that assists a colleague is different from an agent that sends a customer message using it. Do not silently expand an information pilot into an action-taking system. First establish what source assistance is useful and sufficiently dependable for the bounded task and the review arrangements already agreed.
Discuss changes after the first version
A knowledge application depends on document updates, new terminology and new questions. Agree who investigates a report of an incorrect answer and how an updated instruction becomes available again. A system whose sources are current only during a demonstration can lose practical value quickly once colleagues begin to depend on it for recurring work.
Make maintenance specific in the proposal. Ask which changes require another import, an index update or a fresh evaluation. A different language model is not the only change that matters. A relocated document, altered permission or new product release can change the information available to the application and therefore requires an appropriate, targeted check before wider use.
Make a decision that allows different conclusions
The investigation may justify a RAG pilot, show that source organisation needs attention first, or establish that better search is enough. Record the observations behind the decision. A proposal should say what the first version deliberately excludes and which assumptions remain uncertain. This keeps the outcome connected to the information task rather than to a predetermined technology purchase.
Choose one concrete next step: a managed source inventory, a trial using representative questions or an improvement to the existing search experience. The investment then addresses an observed information problem. The supporting technique follows from that task and can be reconsidered when the experiment shows that another design would serve users more effectively.
Turn the explanation into a project decision
Record your assumptions, unknowns and acceptance checks in the matching tool. Use the service guides to discuss delivery and scope.
Open the preparation tool ↗Questions about this topic
Is RAG the same as training a model on our documents?
No. In a typical RAG design, relevant sources are retrieved while preparing a response. This differs from model training. The actual processing, storage and any additional techniques should be documented in the particular architecture.
Can we connect every company document at once?
A broad import may look technically possible, but a bounded source area with an accountable owner is easier to assess in a first useful trial. Investigate gaps, permissions, versions and representative questions before expanding the collection.
What should an initial investigation deliver?
A reasoned choice with a bounded task, reviewed source collection, useful evaluation questions and a concrete next step. Deciding to improve ordinary search first can also be a valuable and well-supported result.
Continue with a related question
Prepare company documents for RAG: build a managed source collection
Make documents useful for a RAG trial through ownership, validity, structure, removal rules and representative checks.
RAG & business knowledgeTest a RAG system: assess sources, answers and refusals separately
Build a useful RAG evaluation with representative questions, expected passages, permission cases, clear criteria and repeatable comparisons.
AI agents & chatbotsAI agent or chatbot: which approach fits your process?
Compare chatbots, fixed workflows and AI agents by task, permissions, reliability and operating effort, with a practical decision example.
Sources and technical background
- Microsoft Learn: Retrieval-augmented generation in Azure AI Search ↗
Architecture overview; the examples and project choices here are original design guidance.
- OWASP: Authorization Cheat Sheet ↗
Authorisation principles; the worked evaluation cases are illustrative.
- NIST: AI Risk Management Framework ↗
Framework for organising AI risk management, not product certification.