Writing strong hypotheses

0:00 Writing strong hypotheses

Let's test a new product page. Sound familiar? That's not a hypothesis. It's a wish. And tests built on wishes usually end in a shrug: inconclusive, nobody learned anything. In this lecture you'll learn why hypotheses matter, a template that forces clarity, the qualities of a strong one, how to turn one insight into several hypotheses, and how to choose primary, secondary and guardrail metrics. Then you'll watch me use AI to widen my idea pool without letting it invent evidence.

0:35 Why hypotheses matter

Why do hypotheses matter? Three reasons. They connect a test to evidence, so you're testing something research suggests could work. They define success in advance, so nobody can move the goalposts after the results come in. And they make every result useful, including losses. If a test built on a clear mechanism loses, you learn the mechanism was wrong, or the execution was weak. If a test built on a wish loses, you learn nothing. Hypotheses turn tests into learning.

1:10 The template

Here's the template. Because we observed this insight, from this evidence, we believe that making this change, for this audience, will cause this outcome, measured by this metric. Every part matters. Because forces evidence. Change forces specificity. Audience forces focus. Outcome and metric force a definition of success. Think of it like a doctor's prescription. It names the diagnosis, the treatment, the patient, and how you'll know it worked.

1:40 Qualities of a strong hypothesis

What makes a hypothesis strong? It's grounded in evidence from more than one source where possible. It's specific enough that a designer could build it without guessing. It names a mechanism, why it should work, like reducing anxiety or removing friction. It's testable with your traffic. And it's worth doing: if it wins, the impact would matter to the business. Tagging the mechanism is especially useful, because over time your learning library tells you which mechanisms work for your customers.

2:15 One insight, several hypotheses

One insight can produce several hypotheses. Say research shows mobile visitors don't trust that delivery will arrive on time. You could add a delivery date estimate on the product page. You could show a delivery promise in the header. You could add reviews that mention delivery. You could offer a delivery guarantee with compensation if you can honour it. Each addresses the same cause with a different change. Write several, then prioritise. The first idea is rarely the best one.

2:50 Three kinds of metric

Now metrics. The primary metric decides the test: choose the one closest to business value that you can measure within the test, often revenue per visitor or purchase conversion. Secondary metrics help you understand how it worked, like add-to-cart rate. Guardrail metrics protect you from harmful wins: return rate, refund requests, page speed, support contacts. And decide your minimum worthwhile effect, the smallest improvement that would justify shipping. That becomes the input for your sample size calculation in the next module.

3:25 Simple example: agency audit form

A simple example. An agency sells a free SEO audit through a landing page. The weak hypothesis: let's make the form shorter. The strong one: because fourteen of twenty form recordings show people pausing at the website revenue field, and sales says that field rarely changes the audit, removing it for first-time visitors will increase qualified audit requests, measured by sales-accepted leads, with lead quality as a guardrail. Notice the primary metric isn't form submissions. A shorter form could double submissions and halve quality.

4:02 Realistic example: UK fashion fit guide (illustrative)

Now a realistic scenario with illustrative details. A UK fashion brand found that eleven of twenty-five filtered mobile recordings showed repeated taps on the size selector before exit, and thirty-eight percent of poll answers mentioned fit. The hypothesis: because mobile visitors from paid social can't judge fit, adding a find-my-size guide with model height and a fit summary from reviews above the size selector will increase revenue per visitor. Secondary metric: add-to-cart rate. Guardrails: thirty-day return rate and mobile page speed. Minimum worthwhile effect: five percent relative. That's a hypothesis a designer can build, an analyst can evaluate, and a finance director can understand.

4:47 Watch me do it: AI to widen ideas

Watch me use AI to widen my idea pool. I paste the finding, with its evidence counts, into my assistant. I ask for six different changes that could address the underlying cause, from small copy tweaks to bigger UX changes. For each, I want the mechanism, what would have to be true for it to work, and one risk or guardrail. And I add a rule: no dark patterns, no fake urgency, no hidden costs, no pre-ticked boxes. It suggests a fit guide, a review-based fit summary, a size-exchange promise, a try-on video, a fit quiz and a size recommendation from past purchases. I keep the ones that trace back to evidence, and write them up as full hypotheses in the card format.

5:40 Store hypotheses as structured cards

Store hypotheses in a structured format, not scattered across slides. A spreadsheet row, a ticket in your project tool, or a small YAML file in a repository. Each gets an ID, links to the findings that support it, the page and audience, the because, change and expect statements, a mechanism tag, the primary, secondary and guardrail metrics, the minimum worthwhile effect, and a status. Structure makes hypotheses searchable, easy to prioritise, and easy to link to results later. It also makes them readable by tools, including AI assistants that help you search your learning library.

6:21 Hypotheses for non-test changes

Hypotheses aren't only for A/B tests. Some changes you'll simply build, like fixing a broken payment method, and some you'll drop. Write a short hypothesis for those too. It records why you made the change and what you expect, so you can check afterwards whether the metric moved in the expected direction, even without a controlled test. It keeps your decisions honest and your learning library complete.

6:50 Practice: rewrite a weak hypothesis

Let's practise rewriting a weak hypothesis, because this is where most teams improve fastest. Weak: adding trust badges will increase conversions. What's missing? Evidence: why do we think trust is the problem? Specificity: which badges, where? Audience: everyone, or first-time mobile visitors? Mechanism: trust in what, exactly, the payment or the product? And the metric: conversions of what? Strong: because exit poll answers on the payment step mention card safety, and recordings show pauses at the card field, adding the payment provider's security explanation and accepted wallets next to the card field for first-time visitors will increase checkout completion, with payment errors as a guardrail. Same idea, but now it can be built, tested and learned from.

7:41 Common mistakes

Common mistakes. Hypotheses without evidence. Vague changes, like improve the page. Missing audience. Clicks or form fills as the primary metric when the business cares about revenue or qualified leads. No guardrails. Too many primary metrics. And letting AI-generated ideas skip the evidence step. Each of these produces tests that are hard to interpret, or wins that don't matter.

8:07 Recap

Recap. A hypothesis connects evidence to a specific change, audience, outcome and metric. Strong hypotheses name a mechanism and define success before the test. One insight can produce many hypotheses, and AI can help you widen the pool, as long as ideas trace back to evidence. Choose a primary metric close to business value, add secondary and guardrail metrics, and set a minimum worthwhile effect. Try this now: take your strongest research finding, write three hypotheses in the card format from the lesson text, and tag each with its mechanism.