Quantitative research: analytics, forms and speed

Video lecture

Quantitative research: analytics, forms and speed

13 chapters · about 8 min · full transcript

Coming soon

Chapter 1 of 13

Quantitative research

Where and how often, not why
Analytics questions that matter
Forms, speed and site search
Avoiding analytical traps

The narrated lecture is in production

Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.

Chapters

Transcript of the narration, chapter by chapter.

0:00 Quantitative research

Numbers don't lie, but they do mislead, especially when you ask them the wrong questions. Quantitative research is how you use analytics, form data, speed data and site search to find where and how often problems happen. In this lecture you'll learn the questions that matter for CRO, how to analyse forms and speed, why site search is the most honest data you have, and the analytical traps that fool smart teams. Then you'll watch me pull real-user speed data for key pages with a short script.

0:38 Why quant first

Why start with quantitative data? Because it covers everyone, not just the handful of people you interview. It tells you which pages matter most, which segments struggle, and how big each problem is. It's the map before the expedition. But it has limits. Analytics tells you that sixty percent of mobile visitors leave the delivery page. It can't tell you whether they left because of the price, the delivery date, or a broken button. That's what the next lessons are for. Quantitative research narrows the search. Qualitative research explains what you find.

1:18 Questions before dashboards

Here are the analytics questions that matter for CRO. Which landing pages get the most traffic, and how do they convert? Where in the funnel do people drop, by device and source? How do new and returning visitors differ? Which products or services get viewed but not bought? Where do people exit, and from which step? What's the revenue per visitor by segment? Write these as questions before you open the tool. Otherwise, you'll browse dashboards for hours and find nothing actionable.

1:54 Form analytics

Form analytics are gold for lead generation and checkouts. Track which fields people start, which they abandon, how long they spend on each, and which trigger errors. A field with a long dwell time and high abandonment is a warning. Maybe it asks for something people don't have at hand, like a company registration number, or something that feels too personal too early, like a phone number. Error rates matter too. If thirty percent of submissions trigger a phone format error, the problem is the validation, not the visitor.

2:33 Speed: Core Web Vitals

Now speed. Slow pages lose visitors, especially on mobile networks. Google's Core Web Vitals measure three things users feel. Largest Contentful Paint: how quickly the main content appears. Interaction to Next Paint, which replaced First Input Delay in March twenty twenty-four: how quickly the page responds when you tap. And Cumulative Layout Shift: whether things jump around while loading. Google's good thresholds, at the seventy-fifth percentile of real users, are two and a half seconds, two hundred milliseconds, and zero point one. Always look at field data from real users, not just a lab test on a fast office connection.

3:16 Site search

Site search is the most honest data you have. People type exactly what they want, in their own words. Look at the most common search terms: are those products easy to find in navigation? Look at zero-result searches: are you missing products, synonyms or spelling variants, like abaya versus abaya dress, or Urdu and English spellings? And compare conversion for sessions with and without search. If searchers convert worse than browsers, your search is a leak.

3:49 Simple example: UK homeware site search

A simple example. A homeware store in the UK notices that 'curtain lengths' is one of its top site searches, and many searches return zero results because products are listed by drop in centimetres, not by length. Customers also search for 'blackout', but the filter is called 'light-blocking'. The fixes are simple: add synonyms to search, rename the filter to use customers' words, and add a size guide link on curtain pages. Nothing here needed a test. The data pointed straight at the fix.

4:26 Realistic example: Lahore umrah packages (illustrative)

Now a realistic scenario with illustrative details. A travel agency in Lahore sells umrah packages online. Analytics shows most traffic lands on package pages from paid social on mobile, and conversion to enquiry is low. Form analytics show that the passport number field has the longest dwell time and the highest abandonment. Speed data from real users shows the package pages have a poor Largest Contentful Paint on mobile, because of large uncompressed hotel photos. Two findings, both quantitative. First, ask for passport details later in the process, after the initial enquiry. Second, compress and properly size images. Then qualitative research can explore what else is holding people back.

5:13 Watch me do it: field speed data

Watch me pull real-user speed data. I have a short Python script that calls Google's PageSpeed Insights API for my key templates: home, a product page and the cart. The API key lives in an environment variable, never in the code. For each page, I request the mobile strategy and read the field metrics: Largest Contentful Paint, Interaction to Next Paint, and Cumulative Layout Shift, at the seventy-fifth percentile. The script handles errors, like timeouts, and prints a neat table. The product page shows a poor Interaction to Next Paint. That's my lead. The likely cause is heavy scripts, so I list every third-party tag on that template, including testing and chat tools, and ask which ones we really need.

6:05 Analytical traps

Now the traps. Averages hide problems, so segment. Correlation isn't causation: people who use the size guide convert more, but that doesn't mean forcing everyone to open it would help. They might be more committed buyers already. Small numbers mislead: a thirty percent conversion rate on a page with ten visits means nothing. Tracking breaks quietly after site releases. And consent-mode modelling means some numbers are estimates. Always ask, what else could explain this?

6:37 AI with analytics

AI helps here too. Many analytics tools now let you ask questions in plain language, and general assistants can analyse exported tables. Keep exports free of personal data. Ask the assistant to show its calculations. Verify a couple of numbers by hand, because models can misread columns or confuse sessions with users. And remember that an AI insight, like Android users convert less, is a starting point for research, not a conclusion. The model doesn't know that your Android app released a buggy update last week. You do.

7:15 Common mistakes

Common mistakes. Opening dashboards without questions. Trusting averages. Ignoring form and search data. Looking only at lab speed tests. Drawing causal conclusions from correlations. Not checking tracking after releases. And treating analytics as the whole truth when consent, ad blockers and modelling mean some data is missing.

7:35 Recap

Recap. Quantitative research tells you where and how often, across everyone. Start with written questions. Use form analytics, real-user speed data and site search, which are some of the richest sources you have. Watch out for averages, correlations, small numbers and tracking gaps. And use AI to speed up analysis, with verification. Try this now: write five analytics questions for your site, run the speed script from the lesson text on three key templates, and export your top fifty site search terms, marking every zero-result search.

What quantitative research tells you

Quantitative research answers what, where and how much: which pages leak, which segments struggle, which form fields cause errors, how fast pages load. It rarely tells you why — that is the job of qualitative research. Use numbers to decide where to look closely.

Analytics questions for CRO

Work through a structured set of questions in your analytics tool:

1. Which landing pages have high traffic but low key event rates?
2. Which funnel steps have the largest absolute drop-off?
3. Which devices, browsers and countries under-perform the average?
4. Which traffic sources send high-intent vs low-intent visitors?
5. Where do users exit most often during the funnel?
6. What do converters do differently (pages viewed, search use, content consumed)?
7. How does performance differ for new vs returning visitors?

Question 6 is particularly valuable: comparing converters and non-converters often reveals the content that tips decisions — for example, visitors who view the delivery information page or a size guide may convert at much higher rates. That suggests surfacing that information earlier.

Form analytics

Forms are where intent meets friction. Field-level analysis reveals:

Field drop-off — the last field touched before abandonment.
Time per field — fields that take unusually long (often unclear labels or formats).
Refills and errors — fields users return to or that trigger validation messages.

Dedicated form-analytics tools provide this, or you can track form_start, form_error (with field name) and form_submit events yourself. Common culprits: phone number formats (international numbers with or without country codes), postcode fields in countries where postcodes are not widely used, mandatory "company" fields for individuals, and password rules that are revealed only after failure.

Speed and technical performance

Slow pages cost conversions, especially on mobile networks. Check:

Core Web Vitals (Largest Contentful Paint, Interaction to Next Paint, Cumulative Layout Shift) using field data where available, not just lab tests on a fast office connection.
Heavy scripts: chat widgets, multiple analytics and ad tags, video backgrounds.
Image sizes and formats on product and landing pages.
Performance on mid-range Android devices, which are common in many markets.

Treat significant speed problems as fixes, not tests.

Site search queries are unfiltered voice-of-customer data. Look for:

High-volume searches with zero results (missing products or synonyms — including local spellings or transliterations).
Searches for information (delivery, returns, "COD", "installments") indicating unanswered questions.
Searches followed by exits.

Worked example: a UK homeware store

Analysis shows (illustratively) that sessions viewing the delivery page convert at a much higher rate than those that do not, and that "delivery" is a top site-search query. Mobile product pages hide delivery information in a collapsed tab near the bottom. Hypothesis: surfacing delivery cost and timing near the add-to-cart button will reduce uncertainty and increase add-to-cart and purchase rates. The team now has a data-backed test instead of a hunch.

Avoiding common analytical traps

Correlation is not causation. Converters might view the delivery page because they already intend to buy. The test, not the correlation, proves impact.
Small samples. A landing page with 40 visits and 0 conversions is not proven broken.
Tracking errors. An "amazing" conversion rate on one browser may be double-firing tags. Verify the data before drawing conclusions.
Averages across different intents. Blending brand search and cold social traffic hides both.

A step-by-step quantitative audit

Confirm tracking accuracy for every funnel step (see the analytics course).
Export 60–90 days of funnel data, segmented by device, source and new vs returning.
Rank landing pages by traffic and key event rate; flag high-traffic, low-rate pages.
Pull field-level form data or error events for the main forms.
Check Core Web Vitals field data for the top templates.
Export site-search terms and flag zero-result and information-seeking queries.
Log each finding with its evidence strength, then decide which ones need qualitative follow-up.

Research log template

Record every finding so it can be linked to hypotheses later:

| ID  | Source      | Finding                                            | Segment      | Evidence strength |
|-----|-------------|----------------------------------------------------|--------------|-------------------|
| Q01 | Analytics   | Mobile payment step drop-off much higher than desk | Mobile       | Strong            |
| Q02 | Form data   | Phone field has highest abandonment                | All          | Medium            |
| Q03 | Site search | "installments" top query, 0 content                | New visitors | Medium            |

Hands-on: pull real-user Core Web Vitals for your key templates

Speed problems hurt conversion most on mobile. Google's PageSpeed Insights API returns field data from the Chrome UX Report (real Chrome users) when enough data exists, plus a lab test. Use an API key from Google Cloud stored in an environment variable:

import os
import requests

API = "https://www.googleapis.com/pagespeedonline/v5/runPagespeed"
KEY = os.environ["PSI_API_KEY"]            # never hard-code keys
PAGES = {
    "home": "https://example.com/",
    "product": "https://example.com/products/linen-shirt",
    "checkout-cart": "https://example.com/cart",
}

for name, url in PAGES.items():
    try:
        r = requests.get(API, params={"url": url, "strategy": "mobile", "key": KEY}, timeout=90)
        r.raise_for_status()
    except requests.RequestException as e:
        print(f"{name}: request failed - {e}")
        continue
    field = r.json().get("loadingExperience", {}).get("metrics", {})
    def p75(metric):
        return field.get(metric, {}).get("percentile", "n/a")
    print(f"{name:14s} LCP(ms) {p75('LARGEST_CONTENTFUL_PAINT_MS'):>6}  "
          f"INP(ms) {p75('INTERACTION_TO_NEXT_PAINT'):>5}  "
          f"CLS(x100) {p75('CUMULATIVE_LAYOUT_SHIFT_SCORE'):>4}")

Google's "good" thresholds at the 75th percentile are LCP ≤ 2.5 s, INP ≤ 200 ms and CLS ≤ 0.1 (INP replaced FID as a Core Web Vital in March 2024). If field data is missing (low traffic), rely on lab data and your own real-user monitoring.

Site search: the most honest data you have

Visitors who search tell you, in their own words, what they want. In GA4, enable site search tracking in enhanced measurement (it reads query parameters such as q or s), then review:

Question	Where to look	What it suggests
What do people search for most?	`view_search_results` event, `search_term`	Navigation or merchandising gaps
Which searches return zero results?	Your site search tool's logs	Missing products, synonyms (e.g. "abaya" vs "abaya dress"), spelling variants
Do searchers convert better or worse?	Segment: sessions with search vs without	If worse, search quality is a leak

Using AI with analytics data

Many analytics tools now include natural-language querying and automated insights, and general assistants can analyse exported CSVs. Keep exports free of personal data, ask for calculations to be shown, and verify one or two numbers by hand. An "insight" such as "Android users convert less" is the start of research, not its conclusion.

Quantitative research shows what and where; qualitative research explains why.
Compare converters with non-converters to find decision-tipping content.
Field-level form analytics and site search reveal specific friction.
Verify tracking and sample sizes before trusting surprising numbers.

Answer the seven analytics questions for a site you know and record at least five findings in a research log.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.

Quantitative research: analytics, forms and speed