LLM hallucinations in business: how to reduce incorrect AI answers

To reduce LLM hallucinations, define where the assistant gets its facts, how it handles missing information and which answers need checking before use. RAG, which retrieves material before an answer is prepared, can help. It does not replace checking sources or testing the full customer service process.

A hallucination occurs when a language model presents false information, or information unsupported by the supplied material, as a fact. A confident tone and correct grammar do not establish that it is true. NIST also describes this problem in relation to fabricated citations and explanations. For a business, the consequence depends on whether someone uses the answer in their work or passes it on to a customer.

Define what the assistant may say and do

Rephrasing a checked message calls for a different level of review from answering a question about product availability, delivery dates or the terms of a particular order. For those questions, identify the source that establishes each fact. The model's general knowledge of how sales usually work is not enough.

Separate drafting a response from acting in a system. An assistant that can write about a refund should not automatically have authority to approve it. Define data access, permission to send messages and permission to change orders separately in the scope of the implementation. Also assign responsibility for decisions the assistant cannot make.

If an answer could create a commitment to a customer, establish how it will be approved. The employee reviewing the text should have access to the information behind it and know exactly what to check. An “Approve” button alone does not provide that review.

RAG helps find material but does not establish its truth

In a RAG solution, the system first retrieves relevant passages from documents and then supplies them to the model to prepare an answer. Microsoft's documentation explains the importance of retrieval relevance and access to the right material. Connecting a company knowledge base alone does not guarantee that the right document will reach the model.

Assign responsibility for keeping the material current. If the knowledge base contains two conflicting versions of the terms, identify which applies and decide how the system should handle the discrepancy. The same applies to changes in your offering: new content on the website does not automatically mean the copy used by the assistant has been refreshed.

Details of a particular order are best retrieved from the system that manages it, within the user's access permissions. A document describing the delivery procedure does not confirm the dispatch date of an individual parcel. Also check what happens when a connection fails: missing information should not become a convincing guess.

Decide how missing information will be handled

The model's instructions should explicitly allow it to say that the material does not contain the information needed. This approach also appears in Anthropic's guidance on reducing hallucinations in Claude. Test the instruction on your own questions, including cases where a customer presses for a definite answer.

Decide what happens next: a request for missing details, referral to an employee or an answer limited to the parts supported by the information available. If the assistant promises contact from a person, the process must actually transfer the case for handling, for example by creating and assigning a ticket. Do not add that promise simply to make the conversation end politely.

An example instruction might be: “Give a delivery date only if it is confirmed in the data for this order. If it is missing, state that the information is unavailable and follow the agreed handling procedure.” Test this instruction together with data retrieval and case transfer.

Example: a customer asks for a delivery date

Suppose, hypothetically, that a customer asks which day their order will arrive. The order system returns “being prepared” but contains no confirmed delivery date. The assistant has grounds to report the status. It has no grounds to add a date based on the usual delivery time.

The response can therefore say that the order is being prepared and that no delivery date has been confirmed yet. The next step depends on the company's actual procedure. If an employee is expected to establish the date, check that they receive a case with the information needed to act. This is an example of designing the service process, not a description of a client implementation or result.

Change the conditions in your test: supply a confirmed date, remove it, return conflicting information and simulate the system being unavailable. A good answer in one of those cases does not tell you how the assistant will behave in the others.

Check the content of the answer, not just its format

JSON is a format for structured data. Correct syntax does not establish that a date or price in the response is true. A link to a document is not enough either: check whether the document actually contains information supporting the statement. The review should cover both the existence of the source and its relationship to the answer.

A second model may help find contradictions, but its agreement does not confirm a fact. The final check of a date or amount should refer to the appropriate system record or document. Before adding another model, compare which errors it actually catches, which it misses, and the cost of using it and handling corrections.

Use everyday questions to check the system before sign-off

Prepare a set of common and difficult questions and define how each should be handled. Include cases with no answer in the sources, outdated material and ambiguous questions. Anthropic's guidance on evaluating LLM applications likewise ties criteria to the task and includes missing information. No single quality threshold suits every company.

Assess incorrect facts, appropriate acknowledgements of missing information and unnecessary refusals separately. An assistant that refuses every question may rarely provide false information, but it still does not help the team. Also check the time an employee needs to correct an answer and whether transferring a case to a person works.

After deployment, keep a way to report errors and assign responsibility for investigating them. The sources used, model and instruction versions, and the answer itself will help reproduce a problem; what you record and who can access it should follow the company's data rules. Add the cases you find to tests for subsequent changes. Decisions about continued use will then rest on the assistant's actual behaviour rather than a claim that it uses RAG or “better instructions”.