Skip to content

AI Fundamentals for Marketers & Creators · How generative AI actually works · lesson 3 of 16 · 11 min

Hallucinations: why AI gets things confidently wrong

What a hallucination is

A hallucination is when an AI produces information that sounds right but is false or unsupported: a made-up statistic, a fake quote, a non-existent study, a wrong product feature, or a URL that leads nowhere. It is not a rare bug; it is a natural side effect of how language models work. They are optimized to produce plausible text, and plausible is not the same as true.

Why it happens

  • Pattern completion over facts. If most marketing articles cite a percentage at this point, the model may produce a percentage, whether or not one exists.
  • Gaps in knowledge. When the model lacks information (a niche local regulation, your client's pricing, last week's news), it may fill the gap rather than admit uncertainty.
  • Pressure from your prompt. "Give me five statistics proving influencer marketing works" invites the model to supply five statistics, real or not.
  • Ambiguity. Vague questions produce confident guesses.

Modern models hallucinate less than earlier ones, and tools with web search or document grounding can cite sources, but no current tool is hallucination-free. Even cited sources can be misread or misquoted.

High-risk zones for marketers

Be most careful with:

  1. Numbers and statistics such as market sizes, conversion benchmarks and follower counts.
  2. Quotes and attributions such as "As Gary Vee said..." or client testimonials.
  3. Legal, medical and financial claims such as health benefits, investment returns and regulatory rules.
  4. Product details such as ingredients, specifications, prices and delivery times.
  5. People and brands: biographies, partnerships and awards.
  6. Links and references: book titles, study names and URLs.

A made-up claim in a sponsored post can mislead consumers, breach advertising rules in markets like the UK, US, UAE and KSA, and damage the trust your brand depends on.

A five-step verification routine

Use this routine whenever AI output contains facts:

  1. Highlight every factual claim. Numbers, names, dates, quotes, features.
  2. Ask where it came from. If the source is "the AI said so", it is unverified.
  3. Check against a primary source. The brand's official page, the regulator, the original study, the client's brief.
  4. Remove or rewrite anything you cannot verify. "Many customers tell us..." beats a fake "87% of customers...".
  5. Keep a note of the source for important claims, especially in sponsored content, in case you are asked.

Prompting to reduce hallucinations

You can lower the risk, though not eliminate it:

  • "Use only the information in the text below. If the answer is not in the text, say 'Not provided'."
  • "Do not invent statistics, quotes or sources. Where a statistic would help, insert [STAT NEEDED] instead."
  • "List any assumptions you made at the end."
  • "Rate your confidence for each claim and flag anything I should verify."

A worked example

Hassan, a freelance copywriter in Manchester, asks for a blog post on "why small businesses should use email marketing". The draft cites a specific return-on-investment figure attributed to a named research firm. Hassan searches for it and finds no such report, only similar-sounding figures from different sources with different methods.

He rewrites the paragraph: "Email remains one of the most cost-effective channels for many small businesses because you own the list and pay little per message." Qualitative, true and still persuasive. Where he wants a number, he uses one from a report he has actually opened, and links to it.

Hands-on: turn a risky prompt into a safe one

Before (invites hallucination):

Give me five statistics proving influencer marketing works, for my pitch deck.

Typical result: five precise-looking percentages with vague or missing sources, at least some of which you cannot find anywhere.

After (grounded and honest):

I'm preparing a pitch deck for a skincare brand in the UAE.
Search for recent (last 24 months) research on influencer marketing effectiveness from reputable
sources (industry bodies, platforms' own published research, peer-reviewed studies).
For each finding give: the claim, who published it, the date, the sample or method if stated, and the link.
If you can't find a reliable source, say so. Do not estimate or round numbers.

Typical result: three findings with publishers, dates and links, one "couldn't find reliable data for the UAE specifically", and no invented percentages. Then run the five-step verification routine on the ones you'll use.

A hallucination-resistant brief template

Add these lines to any content prompt where facts matter:

Use only the facts in the brief below. If something isn't in the brief, write [CHECK] instead of guessing.
Do not add statistics, quotes, awards, prices, dates or health, financial or legal claims unless they're in the brief.
At the end, list every factual claim you made and where it came from in the brief.

Grounding tools reduce, not remove, the risk

Web search, deep research and source-grounded notebooks (such as Gemini Notebook, formerly NotebookLM) cite sources, which makes checking faster. But sources can be misread, outdated or low quality. Treat citations as where to check, not as proof.

Do and don't

  • Do ask for placeholders instead of invented numbers.
  • Do verify anything that would embarrass you if wrong.
  • Don't ask AI to "find statistics that prove X" and paste them straight in.
  • Don't treat a confident tone or a neat citation as evidence.

Video lecture: Hallucinations: why AI gets things confidently wrong

Lecture coming soon · 10 chapters · about 8 minutes. Read the full transcript below.

  1. Hallucinations
  2. What a hallucination is
  3. Why it happens
  4. High-risk zones
  5. The five-step verification routine
  6. Example 1: the entrepreneur quote
  7. Example 2: the clinic's health claim
  8. Watch me do it
  9. Common mistakes
  10. Recap and try this now

Lecture transcript

Hallucinations

Imagine presenting to a client and quoting a statistic that the AI gave you. Seventy three point four percent of Gulf shoppers trust influencer recommendations. The client's analyst asks, great, where's that from? And you realize you have no idea, because it came from nowhere. That's a hallucination, and it's one of the most common ways AI embarrasses marketers. In this lecture you'll learn what hallucinations are, why they happen, where they're most likely, and a simple routine to catch them. You'll see two examples, watch me turn a risky prompt into a safe one, and leave with a brief template that prevents most of them.

What a hallucination is

So what is a hallucination? It's when an AI produces information that sounds right but is false or unsupported. A made up statistic. A quote someone never said. A study that doesn't exist. A product feature your client doesn't have. A link that goes nowhere. Here's the key idea. It's not a rare bug. It's a natural side effect of how language models work. They're optimized to produce plausible text, and plausible isn't the same as true. Modern models hallucinate less than older ones, and search tools can cite sources, but no current tool is hallucination free. So the skill isn't avoiding AI. It's knowing where to look and how to check.

Why it happens

Why does it happen? Four reasons. Pattern completion: if most marketing articles cite a percentage at a certain point, the model may produce one, whether or not a real one exists. Gaps in knowledge: when the model lacks information, like a niche local regulation or your client's pricing, it may fill the gap rather than admit uncertainty. Pressure from your prompt: give me five statistics proving influencer marketing works invites the model to supply five statistics, real or not. And ambiguity: vague questions produce confident guesses. Notice that two of those four are in your control. You write the prompt, and you decide how clear it is.

High-risk zones

Where should you look hardest? Numbers and statistics: market sizes, conversion benchmarks, follower counts. Quotes and attributions: as a famous entrepreneur once said, or a customer testimonial. Legal, medical and financial claims: health benefits, investment returns, what a regulation requires. Recent events and platform changes. Your own product details, like features, prices, ingredients and dates. And links and references. Think of it like proofreading with a highlighter. General phrasing is low risk. Anything specific and checkable is where hallucinations hide, and it's also where they do the most damage.

The five-step verification routine

Here's a five step routine for anything you'll publish or present. One, highlight every factual claim: numbers, names, quotes, dates, product details. Two, trace each one to a source: your brief, the client's materials, or a cited link. Three, open the source and find the exact sentence. Four, check the date and the authority. Is it current, and is it someone credible? Five, remove or replace anything that fails. Rewrite the sentence without the number, or find a real figure from a reputable source and quote it accurately with who published it and when. It takes a few minutes, and it's the difference between confident and correct.

Example 1: the entrepreneur quote

A simple example. A consultant asks for a LinkedIn post about customer retention. The draft opens with a punchy quote attributed to a well known entrepreneur. It sounds exactly like something he'd say. The consultant searches for it. Nothing. No interview, no book, no talk. It's a plausible quote, stitched together from that person's style. Publishing it would be misattribution, and it's easy for someone to call out. The fix is to remove it, or replace it with a real, sourced quote, or better still, replace it with the consultant's own insight from a real client. Which, honestly, makes a better post anyway.

Example 2: the clinic's health claim

Now a business scenario, with illustrative details. An agency in Dubai writes social ads for a dental clinic. An AI draft for a whitening promotion says: guaranteed results in one visit, painless for everyone. Neither claim is in the clinic's brief, and health related claims are regulated in most markets, including by health authorities in the UAE. The account manager catches it on review. But more importantly, she changes the process. Every content prompt for health clients now includes three lines: use only the facts in the brief, write check instead of guessing, and don't add health, financial or legal claims. Then list every factual claim you made and where it came from. The next drafts arrive with check tags instead of invented promises.

Watch me do it

Let me show you the difference. First, the risky prompt: give me five statistics proving influencer marketing works. Here are five neat percentages. Two have vague sources, three have none. Now the grounded version. I'm preparing a pitch deck for a skincare brand in the UAE. Search for recent research on influencer marketing effectiveness from reputable sources. For each finding give the claim, the publisher, the date, the method if stated, and the link. If you can't find a reliable source, say so, and don't estimate or round numbers. This time I get three findings in a table, each with a publisher and date, and one row that says no reliable UAE specific data found. That honesty is worth more than five made up numbers. Now I open the first source and find the exact sentence. It matches. That one goes in the deck.

Common mistakes

The common mistakes. Asking for proof, like statistics proving my point, instead of asking what the evidence says. Trusting citations blindly. A link tells you where to check, not that the claim is right, and sources can be misread or out of date. Skipping checks under deadline pressure, which is exactly when mistakes slip through. Build the check into your workflow so it's automatic. And letting regulated claims through: health, financial and legal statements need your client's approved wording, not the model's creativity. One more: never let AI invent testimonials or reviews. Fake reviews are illegal in many markets.

Recap and try this now

Let's recap. Hallucinations are plausible but false or unsupported outputs, and they come from how models work. Look hardest at numbers, quotes, regulated claims, recent events, product details and links. Reduce them with grounded prompts: use only the brief, write check instead of guessing, don't add claims, list your claims at the end. And catch the rest with the five step verification routine. Here's your try this now. Take a piece of AI assisted content you've published or are about to publish. Highlight every factual claim, and trace each one to a source. Then add the hallucination resistant lines from the lesson text to your go to prompt, so next time there's less to fix.

Key takeaways

  • A hallucination is plausible but false or unsupported output; it's a side effect of how language models work.
  • Highest-risk zones: numbers, quotes, legal/medical/financial claims, recent events, product details and links.
  • Prompts that demand facts ('give me five statistics') invite invention; ask for sourced findings and permission to say 'not found'.
  • Use [CHECK] placeholders, ask for a claims list, and verify with the five-step routine; citations are where to check, not proof.

Try it

Take an AI-written piece you have used before. Highlight every factual claim and label each one as verified, unverified or removed.