LLM hallucination and decision risk: how to measure the exposure

What does LLM hallucination mean for decision risk?

An LLM, or large language model, predicts text, and Wikipedia describes a hallucination as a response that contains false or misleading information presented as fact (Wikipedia, Hallucination (artificial intelligence), consulted October 2026). Decision risk adds a second question: what happens if the false claim is used? A wrong sentence in an email draft and a wrong threshold in a covenant carry very different consequences.

Measuring exposure means attaching numbers to those consequences. A measure lets a team rank its use cases, spend review time where the risk is highest, and show a board that controls follow the stakes. A measure replaces a uniform check with a check sized to the stakes.

How do you list the decisions where an LLM is used?

Start with an inventory. Ask each team where it uses an LLM: drafting, research, summarising, translating, scoring. Record the task, the tool, the owner and the destination of the output. A short sheet with 5 columns is enough. Teams often find uses they did not expect, such as a junior analyst who pastes figures from a chat answer into a model.

Then mark which outputs reach a decision. Text for internal reading sits at the low end. Figures that enter a model, a contract or an investor document sit at the high end. The inventory gives the team its list of use cases, and each use case receives its own exposure score in the steps that follow.

Which factors set the exposure of an LLM hallucination?

Rate 4 factors on a scale of 1 to 5, where 5 means higher exposure: the stakes of the decision, how hard the output is to verify, how hard the action is to reverse, and whether the audience sits outside the organisation. The scale is a working convention that a team can adapt, and it keeps the discussion concrete.

Each rating needs one sentence of evidence. A threshold copied into a contract is hard to reverse once signed, so it earns a 4 or a 5. A figure that links to an open government source is easy to verify, so it earns a 1 or a 2. The table lists the 4 factors with the meaning of a 5.

How do you measure the LLM hallucination rate on your own tasks?

Build a test set from your own work: 50 questions whose answers a colleague has already verified from a primary document. Run the assistant on the set, mark each answer correct or incorrect, and divide the incorrect answers by 50. The rate belongs to one task, one tool and one date, so label it that way and repeat the test after each model update.

Add a line for answers that admit uncertainty. Wikipedia notes that OpenAI researchers proposed rescoring benchmarks to reward acknowledging uncertainty rather than guessing (Wikipedia, Hallucination (artificial intelligence), consulted October 2026). A test set that values an honest answer of not sure shows how a tool behaves at the edge of its knowledge, which matters for decisions.

How do you turn the measures into a decision risk score?

Add the 4 ratings to obtain an exposure score from 4 to 20, then read the measured error rate beside it. An illustrative slogan draft scores 1 + 1 + 1 + 2 = 5. A contract threshold review scores 5 + 4 + 4 + 4 = 17. Board memo figures score 4 + 3 + 3 + 4 = 14. The figure lists the 3 examples.

The exposure score and the error rate answer different questions. The score says how costly a mistake would be, and the rate says how often mistakes occur. A task with a high score and a measured error rate above the team’s tolerance goes to a calculation or a human reviewer. A task with a low score can run with spot checks.

What tolerance should a team set for LLM hallucination risk?

Set tolerance bands in advance. For example, an exposure score up to 8 allows spot checks, a score from 9 to 14 requires a cited source and a reviewer, and a score above 14 requires the number to come from a calculation that can be rerun. The bands are a team choice, and writing them down turns a debate into a rule.

Revisit the bands each quarter. Tools change, tasks change, and a band that suited an earlier tool may need adjusting after a model update. A short review of 5 or 6 sampled outputs per band keeps the rules honest. The sample also shows whether the owners apply the bands the way the team intended.

How does a decision engine lower the LLM hallucination exposure score?

A decision engine lowers the score by moving the number into a calculation. The EQOS Asia engine computes φ(S) between 0 and 1 on 9 dimensions with a deterministic core, and its method page states that scores and numerical verdicts come exclusively from that engine, never from a generative model (EQOS Asia method page, as of October 2026). Verifiability rises, because a reviewer can recompute the result.

The model can be re-measured monthly through Sentinel, which keeps the reading current as facts change (EQOS Asia method page, as of October 2026). A reader can try the public simulator for a first score, read the documented method, and see the Reading tier at US$358 on the pricing page (as of October 2026). This page is general information, not legal, tax or investment advice.

Frequently asked questions

About the author

Stéphane Jambu

President, EQOS Forecast Asia

Stéphane Jambu leads EQOS Forecast Asia and works as a consultant in e-commerce, search visibility, and AI. LinkedIn profile