AI Hallucination Business Risk: What the Data Shows

A man in Vancouver booked a last minute flight to Toronto in November 2022 after his grandmother died. Before booking, he asked the airline's website chatbot about bereavement fares. The bot told him he could buy the ticket now and apply for the reduced rate within 90 days. That was not the policy. Fifteen months later, the BC Civil Resolution Tribunal ordered Air Canada to pay him $812.02.

That story is the whole shape of AI hallucination business risk in one paragraph. The system didn't crash. It didn't refuse. It gave a fluent, confident, plausible answer that happened to be wrong, and the business that deployed it owned the outcome.

A BC tribunal already decided who is responsible

The decision is Moffatt v. Air Canada, 2024 BCCRT 149, issued February 14, 2024 by Tribunal Member Christopher C. Rivers. Air Canada argued it could not be held liable for what its chatbot said. Rivers called that "a remarkable submission" and wrote: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot."

He also rejected the fallback argument that the customer should have checked the correct policy page elsewhere on the site. The decision notes Air Canada "does not explain why customers should have to double-check information found in one part of its website on another part of its website."

The award breaks down as $650.88 in damages, $36.14 in pre-judgment interest and $125 in tribunal fees. Small money. The precedent is the expensive part, and it applies to any business in Burnaby, New Westminster or anywhere else in BC that puts an automated answer in front of a customer.

What the published error rates actually look like

The most useful public number comes from OpenAI's own benchmark paper. SimpleQA, by Jason Wei and colleagues at OpenAI (November 2024), is a set of 4,326 short fact-seeking questions where each has one indisputable answer. Models get graded correct, incorrect, or not attempted.

In the paper's Table 3, GPT-4o answered 38.2% correct, 60.8% incorrect, and declined only 1.0% of the time. Claude 3.5 Sonnet scored 28.9% correct and 36.1% incorrect, but declined 35.0% of the time. OpenAI o1-preview came out highest at 42.7% correct with 48.1% incorrect.

Read that carefully, because the honest reading is narrower than the headline. SimpleQA was built adversarially: questions were kept only if at least one of four OpenAI model completions got them wrong. It is a hard set by construction, so 60.8% is not the error rate you should expect from a well built support bot. It also measures unaided recall of obscure trivia, not a grounded assistant answering from your own documents. And these are 2024 model versions. I have not verified current model scores, so I'm not quoting any.

What the table does establish, and what still holds, is the ratio between wrong and declined. GPT-4o said something wrong sixty times for every one time it said it didn't know. That gap is the risk. A system that fails by refusing is annoying. A system that fails by inventing a 90 day refund window is a liability.

The model does not know when it is wrong

The same paper measured calibration by asking models to state their confidence. The authors found "models consistently overstate their confidence," with accuracy sitting well below stated confidence across the board.

That is why "we'll just tell staff to double check it" isn't a control. There is no visible signal separating the answers that are right from the ones that aren't. Both arrive in the same tone.

Design patterns that actually move the number

Anthropic's own documentation is blunt about the ceiling. Its guide to reducing hallucinations notes that while these techniques significantly reduce hallucinations, "they don't eliminate them entirely," and advises validating critical information for high-stakes decisions. Design accordingly.

• Restrict the model to your documents. Anthropic's guide recommends explicitly instructing the model to use only provided source material and not its general knowledge, and having it pull word-for-word quotes before answering.
• Make it cite, then check its own citation. The same guide suggests having the model find a supporting quote for each claim after drafting, and retract any claim it cannot support.
• Give it permission to say "I don't know." Listed first in Anthropic's guide, and it directly targets that 60 to 1 wrong-versus-declined ratio.
• Whitelist the topics that can be answered automatically. Prices, refund windows, warranty terms, medical or legal specifics, and anything with a deadline should route to a human or to a fixed, hand-written response.
• Build the kill switch before launch. The NIST AI Risk Management Framework subcategory Manage 2.4 asks that mechanisms exist "to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use."
• Monitor in production, not just before it. NIST Measure 2.4 covers monitoring system behaviour once deployed. Sampling and reading real transcripts weekly is the cheapest control on this list.

Illustrative example, not a client result: if a shop handles 400 chats a month and you sample 40 of them, you'll reliably catch a failure mode that shows up in 10% of conversations. You will not catch one that shows up in 0.5%. Set your sample size against the cost of one bad answer.

Where this doesn't apply

Some businesses should not automate customer answers yet, and the evidence supports saying so.

Statistics Canada's study of AI adoption and productivity in Canadian firms by Jiang Li and Huju Liu, released April 22, 2026, found the apparent productivity premium for AI adopters shrank from 16.8% to 10.2% once pre-existing productivity was controlled for, and to 5.1% once complementary capabilities were included, at which point it was no longer statistically significant. Their conclusion is that "AI adoption in isolation is likely insufficient to deliver transformative productivity gains." Firms that were already good at data got the gains. The tool alone did not deliver them.

Adoption is still a minority position. Statistics Canada's second quarter 2026 release, published June 11, 2026, reports 19.2% of Canadian businesses used AI to produce goods or deliver services in the previous 12 months, up from 12.2% a year earlier. Among those users, 28.2% used virtual agents or chatbots, and 13.4% of businesses named cybersecurity or privacy concerns as a barrier.

So: if your answers change weekly and nobody has written them down, fix the documentation first, because a retrieval system with no reliable source to retrieve from will hallucinate more, not less. If a wrong answer carries regulatory or health consequences, keep a human in the loop. If you handle fewer than roughly twenty repetitive inquiries a week, the monitoring overhead probably costs more than the automation saves. NIST Manage 1.1 frames this as a decision point: determine whether the system achieves its intended purpose and whether deployment should proceed at all.

The businesses across Metro Vancouver getting real value out of this are the ones treating an AI employee as a system with a known failure rate, boundaries and an off switch, rather than as a hire that never makes mistakes.

Sources

• Civil Resolution Tribunal of British Columbia. *Moffatt v. Air Canada*, 2024 BCCRT 149 (February 14, 2024). canlii.org
• Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. OpenAI. "Measuring short-form factuality in large language models" (SimpleQA), 2024. arxiv.org
• Anthropic. Claude platform documentation, "Reduce hallucinations." platform.claude.com
• National Institute of Standards and Technology. *AI Risk Management Framework (AI RMF 1.0)*, Core functions, 2023. airc.nist.gov
• Statistics Canada. "Analysis on artificial intelligence use by businesses in Canada, second quarter of 2026," catalogue 11-621-M, June 11, 2026. statcan.gc.ca
• Li, J. and Liu, H. Statistics Canada. "Artificial intelligence adoption and productivity in Canadian firms," April 22, 2026. statcan.gc.ca

If you want an honest read on which of your customer conversations are safe to automate and which ones aren't, that's a conversation worth having before you buy anything. Autana Solutions builds AI employees for businesses around Burnaby and New Westminster, with the boundaries and monitoring built in from day one. Book a free call and we'll map it out with you.