Analytics, heatmaps and A/B testing

Video lecture

Analytics, heatmaps and A/B testing

13 chapters · about 9 min · full transcript

Coming soon

Chapter 1 of 13

Analytics, heatmaps and A/B testing

'Conversion went up!' Compared to what?
Right questions for data
Tracking, recordings, experiments
A reliable improvement cycle

The narrated lecture is in production

Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.

Chapters

Transcript of the narration, chapter by chapter.

0:00 Analytics, heatmaps and A/B testing

A founder shows you a dashboard. Conversion went up four percent after the redesign! Great, you say. Compared to when? Was it the redesign, or the sale that started the same week? Did mobile improve, or only desktop? And did refunds go up too? Silence. Data is powerful, but only when you ask it the right questions. In this lecture, you'll learn the key analytics for UX, how to set up useful tracking in GA4 with privacy in mind, how to use heatmaps and session recordings responsibly, how A/B tests work, their pitfalls, when not to test, and how to combine evidence into a reliable improvement cycle.

0:47 Why it matters

Why does this matter? Usability testing tells you why people struggle. Analytics tells you what happens at scale. Where people come from, where they drop off, what they click. You need both. Think of analytics like a map of traffic flow in a city. It shows you which roads are jammed and which exits nobody takes. It doesn't tell you that a sign is missing at the junction. For that, you go and stand at the junction, which is usability testing. Put the two together and you know where the problem is and why it happens.

1:29 Key metrics for UX

Here are the key metrics for UX. Conversion rate, the share of visitors who complete the goal, which depends heavily on traffic quality. Funnel drop-off, which step loses the most people. Engagement and scroll depth, how far people read. Remember, scrolling isn't understanding. Device and browser split, because problems are often device-specific. Check in-app browsers separately. Traffic source performance, though attribution is always imperfect. Site search terms, which show what people can't find. And form analytics, which show the fields that cause abandonment. Each metric answers a different question, so start with your question.

2:10 Useful tracking in GA4

Setting up useful tracking. Define your conversion events, like form submitted, purchase or booking confirmed. In GA4, you mark these as key events. Then define micro-conversions, like call to action clicked, pricing viewed or video played. Name events consistently. GA4's recommended event generate lead is a good choice for lead forms. Tag your campaign links with UTM parameters, so you know which ad or post drove the visit. And respect privacy. Follow consent requirements for cookies and tracking in the markets you serve, never send personal data like emails or phone numbers in events, and document what you collect.

2:53 Heatmaps and recordings

Heatmaps and session recordings. Behavior analytics tools like Microsoft Clarity, Hotjar or Contentsquare show click maps, including clicks on things that aren't clickable, which signals a false affordance. Scroll maps show how far people get. And session recordings show individual journeys, where you'll spot rage clicks and people going back and forth in confusion. Use them responsibly. Mask all form inputs and sensitive text. Disclose use in your privacy notice. Get consent where it's required. And keep recordings only as long as you need. A simple weekly routine: watch ten recordings of people who dropped off.

3:35 A/B testing basics

Now A slash B testing. An A slash B test shows different versions to randomly split groups of visitors and compares outcomes. A good test starts with a written plan. The hypothesis: because of an insight, changing an element to a variant will improve a metric for an audience. A primary metric. Guardrail metrics that must not get worse, like refunds, lead quality or page speed. The audience. A sample size and duration calculated in advance, running in whole weeks. And a decision rule agreed before you look at results. Here's the key idea. The plan protects you from fooling yourself.

4:19 Pitfalls and when not to test

A slash B testing pitfalls. Too little traffic: small sites often can't detect small changes reliably, so test bolder changes or use qualitative methods. Stopping early: checking daily and stopping when one version looks ahead inflates false winners. Testing many things at once without the right design. Ignoring segments and seasonality, because results can differ by device, channel, or time, like during Ramadan or a big sale. Optimizing the wrong metric: a manipulative change can lift clicks while increasing refunds. And treating one test as universal truth. Also, don't A slash B test obvious bug fixes or accessibility improvements. Just fix them.

5:03 Worked example 1: newsletter form

Worked example one, simple. A newsletter landing page. GA4 shows many CTA clicks but fewer form submissions. Clarity recordings show people tapping the email field, getting a letter keyboard without an at sign, and giving up. That's a form bug, not a test question. The team changes the field to type email, so phones show the right keyboard, and monitors before and after, comparing similar weeks and the same traffic sources. Form completion on mobile improves. No A slash B test needed. Some fixes are simply fixes.

5:41 Worked example 2: Dubai store (illustrative)

Worked example two, a business scenario with illustrative numbers. A Dubai e-commerce store sees mobile checkout conversion far below desktop. Recordings show shoppers repeatedly tapping a tiny promo code field, then leaving to search for codes. In usability tests, people say they feel they're missing a discount. The fix: collapse the promo field behind a small have a code link, and show automatic discounts clearly. With enough traffic, they run an A slash B test over three full weeks, pre-registered, with customer service contacts as a guardrail. Mobile checkout completion improves, with no increase in contacts.

6:23 Watch me do it: events → funnel → recordings

Watch me do it. I add two events. A CTA click event on every button with a data attribute, and generate lead, fired only from the form's success handler, never on button click. No personal data in either. In GA4 admin, I mark generate lead as a key event. Then I build a funnel exploration: page view on the landing page, CTA click, generate lead. I break it down by device and filter to paid social. Mobile loses most people between click and lead. In Clarity, I filter recordings to mobile, paid social, landing page, and watch ten sessions that dropped off. Three show the same confused behavior on the phone number field.

7:12 Sanity-check winners, then combine evidence

When a test tool shows a winner, sanity-check it. Did it run for the planned duration, in whole weeks? Are the two groups similar in size? A big mismatch can mean a setup problem, often called a sample ratio mismatch. Did the guardrails hold? Is the result consistent across major segments like device? If not, re-run it, or treat the result as a hint rather than a fact. And combine evidence in a cycle. Analytics shows where the problem is. Recordings and tests show why. You design a fix, validate it quickly, and launch, as an A slash B test if traffic allows, otherwise with careful before and after monitoring.

8:00 Common mistakes and measures

Common mistakes. Collecting data without clear questions. Tracking without consent where it's required, or sending personal data to analytics. Firing the lead event on button click instead of on success. Running A slash B tests on low traffic and trusting noisy results. Peeking and stopping early. And using analytics to confirm opinions instead of to learn. How do you measure success? Whether you can answer, in minutes, where do mobile visitors from paid social drop off, and what did they do just before? Plus the share of tests with written plans, and how many shipped changes hold their gains after launch.

8:44 Recap and try this now

Let's recap. Analytics shows what happens at scale. Testing shows why. Define key events and micro-conversions, tag your links, and respect consent and privacy. Use heatmaps and recordings responsibly to see behavior. Run A slash B tests with a clear hypothesis, a pre-set sample size, guardrails and a decision rule. Don't test obvious fixes. Just fix them. Try this now. Map the funnel for one conversion path in your analytics. Find the biggest drop-off, watch ten recordings of people who left there, and write either an A slash B test plan, or a before and after plan if your traffic is low.

Two kinds of evidence

Usability testing tells you why people struggle. Analytics tells you what happens at scale — where people come from, where they drop off, what they click. You need both.

Key analytics for UX

Metric	What it reveals	Caution
Conversion rate	Share of visitors completing the goal	Depends heavily on traffic quality
Funnel drop-off	Which step loses the most users	Needs correct event tracking
Engagement/scroll depth	How far people read	High scroll doesn't mean understanding
Bounce/engagement rate	Leaving without interaction	Definitions differ between tools
Device and browser split	Where problems may be device-specific	Check in-app browsers separately
Traffic source performance	Which channels bring converting visitors	Attribution is imperfect
Site search terms	What people can't find	Great source of label ideas
Form analytics	Which fields cause abandonment	Need privacy-safe setup

Setting up useful tracking

Define conversion events (form submitted, purchase, booking confirmed) and micro-conversions (CTA clicked, pricing viewed, video played).
Name events consistently.
Tag campaign links (e.g. with UTM parameters) so you know which ad or post drove visits.
Respect privacy: follow consent requirements for cookies and tracking (e.g. under GDPR/UK GDPR and applicable local laws), avoid collecting personal data in analytics, and document what you collect.

Heatmaps and session recordings

Behavior analytics tools show:

Click maps: where people click — including on things that aren't clickable (a sign of false affordance).
Scroll maps: how far people get.
Session recordings: individual journeys — helpful for spotting confusion ("rage clicks", back-and-forth).

Use responsibly: mask sensitive inputs, disclose use in your privacy policy, obtain consent where required, and don't record more than you need.

A/B testing basics

An A/B test shows different versions of a page (or element) to randomly split groups of visitors and compares outcomes.

A/B test plan
Hypothesis:  Because [insight], changing [element] to [variant] will improve [metric]
             for [audience].
Example:     Because 4/5 test users missed per-staff pricing, adding a price calculator
             will increase booking-call conversions on mobile.
Primary metric: Booking-call conversion rate
Guardrail metrics: Refund requests, lead quality, page speed
Audience/traffic: Mobile visitors from paid social
Sample size & duration: Calculated in advance; run full weeks
Decision rule: Ship if variant beats control with pre-agreed confidence

A/B testing pitfalls

Too little traffic: small sites often can't reach reliable results for small changes. Use qualitative testing and make bolder changes instead.
Stopping early: checking results daily and stopping when one version looks ahead inflates false positives. Decide sample size or duration in advance.
Testing many things at once without an appropriate design, making results hard to interpret.
Ignoring segments and seasonality: results can differ by device, channel or time (e.g. during Ramadan or sales periods).
Optimizing the wrong metric: a manipulative change can raise clicks while increasing refunds or complaints — use guardrail metrics.
Treating one test as universal truth: results depend on context.

When not to A/B test

Fixing obvious usability bugs (just fix them).
Accessibility improvements (required, not optional).
Very low-traffic pages.
Changes that are legally or ethically necessary.

Combining evidence

A reliable improvement cycle:

Analytics shows where the problem is (e.g. 60% drop at step 2 on mobile).
Recordings and usability tests show why (users can't find the delivery date field).
Design a fix based on the insight.
Validate with a quick usability test.
Launch — as an A/B test if traffic allows, otherwise monitor before/after with care.

Worked example: a Dubai e-commerce store

Analytics: mobile checkout conversion far below desktop. Recordings: users repeatedly tapping a tiny promo-code field and leaving to search for codes. Usability test: users felt they were "missing a discount". Fix: collapsed the promo field behind a small "Have a code?" link and showed automatic discounts clearly. A/B test over three full weeks showed improved mobile checkout completion, with no increase in customer service contacts.

Common mistakes

Collecting data without clear questions.
Tracking without consent where it's required.
Running A/B tests on low traffic and trusting noisy results.
Using analytics to confirm opinions rather than to learn.

Hands-on: track the events that matter in GA4

Many form and CTA interactions aren't tracked by default. If you use Google Tag Manager, create tags for the events you need; if you add code directly, the pattern with the Google tag (gtag.js) looks like this. Fire events only after the user has given any consent your jurisdiction requires, and never send personal data (names, emails, phone numbers) in event parameters.

<script>
  // CTA click (micro-conversion)
  document.querySelectorAll('[data-cta]').forEach(function (btn) {
    btn.addEventListener('click', function () {
      gtag('event', 'cta_click', { cta_id: btn.dataset.cta, page_type: 'landing' });
    });
  });

  // Successful form submission (conversion) — call this from your form's success handler
  function trackLead(formId) {
    gtag('event', 'generate_lead', { form_id: formId });
  }
</script>

generate_lead is one of GA4's recommended event names; you can mark it as a key event (GA4's term for a conversion) in the admin settings. Then build a Funnel exploration (Explore → Funnel exploration) with steps such as page_view on the landing page → cta_click → generate_lead, and break it down by device category and session source.

Tag your campaign links consistently so traffic sources are clear:

https://example.com/webinar?utm_source=instagram&utm_medium=paid_social&utm_campaign=sep26_webinar&utm_content=story_v2

Hands-on: behavior analytics setup checklist

Tools such as Microsoft Clarity (free), Hotjar or Contentsquare provide heatmaps and session recordings.

[ ] Installed only after consent where required; mentioned in the privacy notice
[ ] Masking enabled for all form inputs and any sensitive text
[ ] Recording retention set to the minimum you need
[ ] Filters saved: device = mobile, source = paid social, page = landing
[ ] Weekly routine: watch 10 recordings of drop-offs; check rage/dead-click reports

Checking an A/B result responsibly

If your testing tool shows a winner, sanity-check it: Did the test run for the pre-set duration in whole weeks? Were the groups similar in size (a large mismatch can signal a setup problem, often called a sample ratio mismatch)? Did guardrail metrics hold? Are results consistent across major segments such as device? When in doubt, re-run or treat the result as a hint, not a fact. The sample-size calculator in the "Landing page UX testing" lesson helps you decide in advance whether a test is feasible.

How to measure success

A good analytics setup is one where you can answer, within minutes, "Where do mobile visitors from paid social drop off, and what did they do just before?" Measure the number of decisions made from evidence each month, the share of tests with pre-registered plans, and the proportion of shipped changes that held their gains after launch.

Summary

Use analytics to find where problems occur, heatmaps and recordings (responsibly) to see behavior, usability tests to understand why, and A/B tests with clear hypotheses, pre-set sample sizes and guardrail metrics to validate changes.

Analytics shows what happens at scale; usability testing shows why.
Define conversion and micro-conversion events and respect consent and privacy rules.
A/B tests need a clear hypothesis, pre-set sample size or duration and guardrail metrics.
Don't A/B test obvious bug fixes or accessibility improvements — just fix them.

Map the funnel for one conversion path using your analytics, find the biggest drop-off, and write an A/B test plan (or a before/after plan if traffic is low) using the template.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.

Analytics, heatmaps and A/B testing